2024-03-26T06:00:59.902 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-26T06:00:59.902 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi --owner scheduled_rishabh@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --archive /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740 --verbose -- /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740/orig.config.yaml 2024-03-26T06:00:59.961 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:01:00.055 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740 branch: rishabh-2024mar8-2 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7622740' kernel: client: sha1: distro flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-2024mar8-2 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 7e98f5f6b535defc7fa715b456904ffc0500aa9f subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 7e98f5f6b535defc7fa715b456904ffc0500aa9f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-2024mar8-2 sha1: 7e98f5f6b535defc7fa715b456904ffc0500aa9f owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1249 sha1: 7e98f5f6b535defc7fa715b456904ffc0500aa9f sleep_before_teardown: 0 subset: 481/512 suite: fs suite_branch: rishabh-2024mar8-2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_7e98f5f6b535defc7fa715b456904ffc0500aa9f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 7e98f5f6b535defc7fa715b456904ffc0500aa9f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMUJWwIeV2tDbDTKU5oQJSoans6FdnapWWLfkeVDtMx8cGd93Rk8ud8orz2qvUwStrzULIm8g8053PXmu7XgR2c= smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFwwNqCYxhAGGT+xm8s18CqV9Mc4RWhKijKL3dAJLl5sNFuuc60XCA8QAvx6bkqmHSpPku2RnzSvrx7p6ThgJto= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBdzvzIWPethCrJJLKbvNhiYcu7Gg3Swk7r3Ihyf8xqRLwwSJDi4QMFI1gM/s+MFEOObjxH4RrGyOvK4iYlLwsE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-26_05:47:39 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-26T06:01:00.056 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_7e98f5f6b535defc7fa715b456904ffc0500aa9f/qa; will attempt to use it 2024-03-26T06:01:00.057 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_7e98f5f6b535defc7fa715b456904ffc0500aa9f/qa/tasks 2024-03-26T06:01:00.057 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-26T06:01:00.058 INFO:teuthology.task.internal:Checking packages... 2024-03-26T06:01:00.082 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7e98f5f6b535defc7fa715b456904ffc0500aa9f' 2024-03-26T06:01:00.082 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-26T06:01:00.082 INFO:teuthology.packaging:ref: None 2024-03-26T06:01:00.082 INFO:teuthology.packaging:tag: None 2024-03-26T06:01:00.082 INFO:teuthology.packaging:branch: rishabh-2024mar8-2 2024-03-26T06:01:00.083 INFO:teuthology.packaging:sha1: 7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:01:00.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-2024mar8-2 2024-03-26T06:01:00.303 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1964.g7e98f5f6 2024-03-26T06:01:00.304 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-26T06:01:00.311 INFO:teuthology.task.internal:no buildpackages task found 2024-03-26T06:01:00.311 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-26T06:01:00.333 INFO:teuthology.task.internal:Saving configuration 2024-03-26T06:01:00.354 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-26T06:01:00.361 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T06:01:00.387 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 05:54:25.167422', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMUJWwIeV2tDbDTKU5oQJSoans6FdnapWWLfkeVDtMx8cGd93Rk8ud8orz2qvUwStrzULIm8g8053PXmu7XgR2c='} 2024-03-26T06:01:00.410 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 05:54:25.166051', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFwwNqCYxhAGGT+xm8s18CqV9Mc4RWhKijKL3dAJLl5sNFuuc60XCA8QAvx6bkqmHSpPku2RnzSvrx7p6ThgJto='} 2024-03-26T06:01:00.435 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 05:54:25.168524', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBdzvzIWPethCrJJLKbvNhiYcu7Gg3Swk7r3Ihyf8xqRLwwSJDi4QMFI1gM/s+MFEOObjxH4RrGyOvK4iYlLwsE='} 2024-03-26T06:01:00.435 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-26T06:01:00.441 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-26T06:01:00.441 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-26T06:01:00.441 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-26T06:01:00.442 INFO:teuthology.run_tasks:Running task console_log... 2024-03-26T06:01:00.579 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fde629c40d0>, signals=[15]) 2024-03-26T06:01:00.580 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-26T06:01:00.585 INFO:teuthology.task.internal:Opening connections... 2024-03-26T06:01:00.585 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-03-26T06:01:00.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:01:00.697 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-03-26T06:01:00.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:01:00.802 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-03-26T06:01:00.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:01:00.891 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-26T06:01:00.906 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-03-26T06:01:00.924 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-03-26T06:01:00.925 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-03-26T06:01:00.982 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-03-26T06:01:00.982 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-03-26T06:01:00.982 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-03-26T06:01:00.982 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-03-26T06:01:00.982 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-03-26T06:01:00.982 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-03-26T06:01:00.983 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T06:01:00.983 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-03-26T06:01:00.983 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-03-26T06:01:00.983 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T06:01:00.983 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-03-26T06:01:00.983 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T06:01:00.983 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T06:01:00.983 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T06:01:00.984 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-03-26T06:01:01.022 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-03-26T06:01:01.041 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-03-26T06:01:01.041 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-03-26T06:01:01.099 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream" 2024-03-26T06:01:01.099 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="9" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="9" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el9" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:LOGO="fedora-logo-icon" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T06:01:01.100 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T06:01:01.101 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-03-26T06:01:01.114 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-03-26T06:01:01.129 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-03-26T06:01:01.130 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-03-26T06:01:01.185 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-03-26T06:01:01.185 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-03-26T06:01:01.185 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-03-26T06:01:01.185 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-03-26T06:01:01.186 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-03-26T06:01:01.186 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-03-26T06:01:01.186 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T06:01:01.186 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-03-26T06:01:01.186 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-03-26T06:01:01.186 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T06:01:01.186 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-03-26T06:01:01.186 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T06:01:01.186 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T06:01:01.186 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T06:01:01.187 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-03-26T06:01:01.205 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-26T06:01:01.218 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-26T06:01:01.252 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-26T06:01:01.252 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-03-26T06:01:01.255 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-03-26T06:01:01.257 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-03-26T06:01:01.273 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-26T06:01:01.283 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-26T06:01:01.283 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-03-26T06:01:01.312 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-03-26T06:01:01.315 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-03-26T06:01:01.343 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-26T06:01:01.412 INFO:teuthology.run_tasks:Running task kernel... 2024-03-26T06:01:01.429 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-26T06:01:01.430 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-26T06:01:01.430 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-26T06:01:01.430 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T06:01:01.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:01:01.449 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-03-26T06:01:01.507 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-361.el9.x86_64 2024-03-26T06:01:01.508 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.14.0-361.el9.x86_64 2024-03-26T06:01:01.508 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum install -y kernel 2024-03-26T06:01:03.689 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-26T06:01:05.104 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2024-03-26T06:01:07.153 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-03-26T06:01:09.904 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-03-26T06:01:14.271 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-03-26T06:01:15.714 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-03-26T06:01:18.365 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 8.2 MB/s | 21 MB 00:02 2024-03-26T06:01:23.474 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-03-26T06:01:25.681 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-26T06:01:25.681 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-26T06:01:25.773 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-26T06:01:25.780 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-26T06:01:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-03-26T06:01:25.780 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-26T06:01:25.780 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-26T06:01:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-03-26T06:01:25.780 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-26T06:01:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-03-26T06:01:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-03-26T06:01:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-03-26T06:01:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:01:25.781 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-26T06:01:25.781 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-26T06:01:25.781 INFO:teuthology.orchestra.run.smithi037.stdout:Install 4 Packages 2024-03-26T06:01:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:01:25.782 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 76 M 2024-03-26T06:01:25.782 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 123 M 2024-03-26T06:01:25.782 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-26T06:01:25.950 INFO:teuthology.orchestra.run.smithi037.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 502 kB/s | 66 kB 00:00 2024-03-26T06:01:27.634 INFO:teuthology.orchestra.run.smithi037.stdout:(2/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 18 MB/s | 33 MB 00:01 2024-03-26T06:01:27.909 INFO:teuthology.orchestra.run.smithi037.stdout:(3/4): kernel-core-5.14.0-432.el9.x86_64.rpm 7.1 MB/s | 15 MB 00:02 2024-03-26T06:01:32.742 INFO:teuthology.orchestra.run.smithi037.stdout:(4/4): kernel-modules-core-5.14.0-432.el9.x86_6 4.0 MB/s | 27 MB 00:06 2024-03-26T06:01:32.742 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-26T06:01:32.743 INFO:teuthology.orchestra.run.smithi037.stdout:Total 11 MB/s | 76 MB 00:06 2024-03-26T06:01:33.438 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-26T06:01:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-26T06:01:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-26T06:01:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-26T06:01:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-26T06:01:37.509 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-26T06:01:38.151 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-03-26T06:01:38.251 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T06:01:41.150 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T06:01:41.296 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T06:01:46.372 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T06:01:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T06:01:51.517 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T06:02:22.149 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T06:02:22.150 INFO:teuthology.orchestra.run.smithi037.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T06:02:22.150 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:02:22.196 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-03-26T06:02:22.603 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T06:02:22.603 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-03-26T06:02:22.603 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T06:02:22.603 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T06:02:23.043 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T06:02:23.043 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:02:23.044 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-26T06:02:23.044 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-5.14.0-432.el9.x86_64 2024-03-26T06:02:23.044 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-03-26T06:02:23.044 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-03-26T06:02:23.044 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-03-26T06:02:23.044 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:02:23.044 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-26T06:02:23.374 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T06:02:23.482 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T06:02:23.482 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T06:02:23.483 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-03-26T06:02:23.483 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-26T06:02:23.483 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-26T06:02:23.483 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-26T06:02:23.483 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi037.front.sepia.ceph.com, path=None, version=distro) 2024-03-26T06:02:23.483 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T06:02:23.566 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T06:02:23.566 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T06:02:23.566 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -qi grub2-tools 2024-03-26T06:02:23.603 INFO:teuthology.orchestra.run.smithi037.stdout:Name : grub2-tools 2024-03-26T06:02:23.603 INFO:teuthology.orchestra.run.smithi037.stdout:Epoch : 1 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Version : 2.06 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Release : 68.el9 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: x86_64 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Group : Unspecified 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Size : 8264254 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:License : GPLv3+ 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Packager : builder@centos.org 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Vendor : CentOS 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T06:02:23.604 INFO:teuthology.orchestra.run.smithi037.stdout:Summary : Support tools for GRUB. 2024-03-26T06:02:23.605 INFO:teuthology.orchestra.run.smithi037.stdout:Description : 2024-03-26T06:02:23.605 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:02:23.605 INFO:teuthology.orchestra.run.smithi037.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T06:02:23.605 INFO:teuthology.orchestra.run.smithi037.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T06:02:23.605 INFO:teuthology.orchestra.run.smithi037.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T06:02:23.605 INFO:teuthology.orchestra.run.smithi037.stdout:hardware devices. 2024-03-26T06:02:23.605 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:02:23.605 INFO:teuthology.orchestra.run.smithi037.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T06:02:23.606 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T06:02:23.606 INFO:teuthology.task.kernel:Updating grub on smithi037 to boot 5.14.0-432.el9.x86_64 2024-03-26T06:02:23.606 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T06:02:24.245 INFO:teuthology.orchestra.run.smithi037.stderr:Generating grub configuration file ... 2024-03-26T06:02:25.756 INFO:teuthology.orchestra.run.smithi037.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T06:02:25.784 INFO:teuthology.orchestra.run.smithi037.stderr:done 2024-03-26T06:02:25.786 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-03-26T06:02:25.801 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.xGOPF8UuyG 2024-03-26T06:02:25.801 DEBUG:teuthology.orchestra.run.smithi037:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xGOPF8UuyG 2024-03-26T06:02:25.876 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.xGOPF8UuyG 2024-03-26T06:02:26.016 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.xGOPF8UuyG is 6KB 2024-03-26T06:02:26.065 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.xGOPF8UuyG 2024-03-26T06:02:26.079 DEBUG:teuthology.orchestra.run.smithi037:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T06:02:26.148 INFO:teuthology.orchestra.run.smithi037.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T06:02:26.148 INFO:teuthology.orchestra.run.smithi037.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T06:02:26.148 INFO:teuthology.orchestra.run.smithi037.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T06:02:26.148 INFO:teuthology.orchestra.run.smithi037.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-03-26T06:02:26.149 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-03-26T06:02:26.296 DEBUG:teuthology.orchestra.run.smithi037:> sudo shutdown -r now 2024-03-26T06:02:56.299 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-26T06:02:56.300 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-26T06:02:56.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:03:14.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-03-26T06:03:23.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-26T06:03:23.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:03:55.689 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-03-26T06:03:56.213 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-26T06:03:56.213 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-432.el9.x86_64"... 2024-03-26T06:03:56.214 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-03-26T06:03:56.227 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-432.el9.x86_64 2024-03-26T06:03:56.227 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-03-26T06:03:56.227 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T06:03:56.227 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T06:03:57.228 DEBUG:teuthology.parallel:result is None 2024-03-26T06:03:57.228 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-26T06:03:57.236 INFO:teuthology.task.internal:Creating test directory... 2024-03-26T06:03:57.236 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T06:03:57.239 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T06:03:57.241 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T06:03:57.259 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-26T06:03:57.285 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-26T06:03:57.317 INFO:teuthology.task.internal:Creating archive directory... 2024-03-26T06:03:57.317 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T06:03:57.319 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T06:03:57.322 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T06:03:57.495 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-26T06:03:57.679 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-26T06:03:57.680 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T06:03:57.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:03:57.697 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T06:03:57.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:03:57.713 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T06:03:57.715 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T06:03:57.739 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T06:03:57.789 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T06:03:57.803 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T06:03:57.804 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T06:03:57.813 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T06:03:57.824 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T06:03:57.856 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T06:03:57.858 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-26T06:03:57.866 INFO:teuthology.task.internal:Configuring sudo... 2024-03-26T06:03:57.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T06:03:57.868 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T06:03:57.870 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T06:03:57.901 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-26T06:03:57.914 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-26T06:03:57.915 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T06:03:57.935 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T06:03:57.944 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T06:03:57.960 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T06:03:58.030 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T06:03:58.085 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T06:03:58.167 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T06:03:58.208 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:03:58.208 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T06:03:58.301 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T06:03:58.347 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T06:03:58.410 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T06:03:58.453 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T06:03:58.503 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-26T06:03:58.504 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T06:03:58.577 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T06:03:58.621 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T06:03:58.682 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T06:03:58.725 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T06:03:58.769 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-26T06:03:58.769 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T06:03:58.838 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-03-26T06:03:58.840 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-03-26T06:03:58.842 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-03-26T06:03:58.883 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T06:03:58.885 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T06:03:58.920 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T06:03:59.375 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-26T06:03:59.383 INFO:teuthology.task.internal:Starting timer... 2024-03-26T06:03:59.383 INFO:teuthology.run_tasks:Running task pcp... 2024-03-26T06:03:59.438 INFO:teuthology.run_tasks:Running task selinux... 2024-03-26T06:03:59.469 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-26T06:03:59.469 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-03-26T06:03:59.537 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-03-26T06:03:59.539 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2024-03-26T06:03:59.599 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2024-03-26T06:03:59.601 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-03-26T06:03:59.650 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-03-26T06:03:59.652 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-26T06:03:59.652 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-03-26T06:03:59.671 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-03-26T06:03:59.671 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2024-03-26T06:03:59.689 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2024-03-26T06:03:59.689 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-03-26T06:03:59.710 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-03-26T06:03:59.710 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive'} 2024-03-26T06:03:59.710 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T06:03:59.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:03:59.744 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T06:03:59.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:03:59.776 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T06:03:59.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:03:59.804 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-26T06:03:59.804 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-03-26T06:03:59.873 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2024-03-26T06:03:59.908 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-03-26T06:03:59.953 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-26T06:03:59.963 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-26T06:03:59.969 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-26T06:04:00.121 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-26T06:04:00.138 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-26T06:04:00.139 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi066.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-26T06:08:55.091 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2024-03-26T06:08:55.092 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-26T06:08:55.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:08:55.186 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-03-26T06:08:55.265 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-26T06:08:55.265 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-26T06:08:55.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:08:55.368 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-03-26T06:08:55.438 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-03-26T06:08:55.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-26T06:08:55.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:08:55.527 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-03-26T06:08:55.602 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-26T06:08:55.603 INFO:teuthology.run_tasks:Running task clock... 2024-03-26T06:08:55.613 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-26T06:08:55.614 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T06:08:55.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T06:08:55.617 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T06:08:55.617 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T06:08:55.619 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T06:08:55.619 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T06:08:55.649 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T06:08:55.653 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T06:08:55.667 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T06:08:55.673 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T06:08:55.674 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T06:08:55.691 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T06:08:55.699 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-03-26T06:08:55.709 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2024-03-26T06:08:55.713 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-03-26T06:08:55.721 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-03-26T06:08:55.726 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2024-03-26T06:08:55.730 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T06:08:55.735 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-03-26T06:08:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T06:08:55.747 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T06:08:55.751 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T06:08:55.766 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T06:08:55.767 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T06:08:55.800 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-03-26T06:08:55.803 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T06:08:55.803 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-03-26T06:08:55.803 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.803 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.803 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.803 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.811 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-03-26T06:08:55.814 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-03-26T06:08:55.815 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T06:08:55.815 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-03-26T06:08:55.815 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.815 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.815 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.815 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.817 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T06:08:55.817 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-03-26T06:08:55.817 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.817 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.817 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.817 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:08:55.818 INFO:teuthology.run_tasks:Running task install... 2024-03-26T06:08:55.829 DEBUG:teuthology.task.install:project ceph 2024-03-26T06:08:55.830 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7e98f5f6b535defc7fa715b456904ffc0500aa9f'}} 2024-03-26T06:08:55.830 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7e98f5f6b535defc7fa715b456904ffc0500aa9f'} 2024-03-26T06:08:55.830 INFO:teuthology.task.install:Using flavor: default 2024-03-26T06:08:55.837 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-26T06:08:55.837 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-26T06:08:55.838 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7e98f5f6b535defc7fa715b456904ffc0500aa9f', 'tag': None, 'wait_for_package': False} 2024-03-26T06:08:55.838 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:08:55.840 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7e98f5f6b535defc7fa715b456904ffc0500aa9f', 'tag': None, 'wait_for_package': False} 2024-03-26T06:08:55.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:08:55.841 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7e98f5f6b535defc7fa715b456904ffc0500aa9f', 'tag': None, 'wait_for_package': False} 2024-03-26T06:08:55.841 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:08:55.991 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/ 2024-03-26T06:08:55.991 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1964.g7e98f5f6 2024-03-26T06:08:56.062 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/ 2024-03-26T06:08:56.062 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1964.g7e98f5f6 2024-03-26T06:08:56.078 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/ 2024-03-26T06:08:56.078 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1964.g7e98f5f6 2024-03-26T06:08:56.164 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T06:08:56.165 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-26T06:08:56.165 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T06:08:56.196 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T06:08:56.196 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7e98f5f6b535defc7fa715b456904ffc0500aa9f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T06:08:56.232 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T06:08:56.232 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-26T06:08:56.232 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T06:08:56.245 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T06:08:56.245 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:08:56.245 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T06:08:56.261 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T06:08:56.261 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7e98f5f6b535defc7fa715b456904ffc0500aa9f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T06:08:56.267 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T06:08:56.273 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T06:08:56.274 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7e98f5f6b535defc7fa715b456904ffc0500aa9f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T06:08:56.329 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T06:08:56.343 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T06:08:56.355 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T06:08:56.385 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2024-03-26T06:08:56.386 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2024-03-26T06:08:56.409 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T06:08:56.424 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T06:08:56.475 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-03-26T06:08:56.477 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-03-26T06:08:56.491 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-03-26T06:08:56.493 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-03-26T06:08:56.633 INFO:teuthology.orchestra.run.smithi066.stdout:74 files removed 2024-03-26T06:08:56.660 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T06:08:56.710 INFO:teuthology.orchestra.run.smithi149.stdout:74 files removed 2024-03-26T06:08:56.737 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T06:08:56.757 INFO:teuthology.orchestra.run.smithi037.stdout:74 files removed 2024-03-26T06:08:56.785 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T06:08:57.494 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 156 kB/s | 82 kB 00:00 2024-03-26T06:08:57.508 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 173 kB/s | 82 kB 00:00 2024-03-26T06:08:57.515 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 193 kB/s | 82 kB 00:00 2024-03-26T06:08:57.853 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-26T06:08:57.892 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-03-26T06:08:57.940 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-26T06:08:58.279 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-03-26T06:08:58.301 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-03-26T06:08:58.349 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-03-26T06:08:58.704 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-26T06:08:58.765 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-26T06:08:58.852 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-26T06:09:00.129 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-03-26T06:09:00.223 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-03-26T06:09:00.303 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-26T06:09:01.127 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-26T06:09:02.122 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-03-26T06:09:04.194 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.0 MB 00:03 2024-03-26T06:09:17.843 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-03-26T06:09:20.255 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-03-26T06:09:26.348 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.1 MB 00:02 2024-03-26T06:09:27.807 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-03-26T06:09:28.861 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-03-26T06:09:32.844 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 555 kB/s | 6.1 MB 00:11 2024-03-26T06:09:33.958 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-03-26T06:09:34.545 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-03-26T06:09:35.608 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 637 kB/s | 19 MB 00:30 2024-03-26T06:09:35.888 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-26T06:09:36.481 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T06:09:36.482 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T06:09:36.495 INFO:teuthology.orchestra.run.smithi066.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T06:09:36.496 INFO:teuthology.orchestra.run.smithi066.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T06:09:36.497 INFO:teuthology.orchestra.run.smithi066.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T06:09:36.498 INFO:teuthology.orchestra.run.smithi066.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T06:09:36.500 INFO:teuthology.orchestra.run.smithi066.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T06:09:36.503 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T06:09:36.504 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T06:09:36.590 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-03-26T06:09:36.616 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================ 2024-03-26T06:09:36.617 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-03-26T06:09:36.617 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================ 2024-03-26T06:09:36.617 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-03-26T06:09:36.617 INFO:teuthology.orchestra.run.smithi066.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T06:09:36.617 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 6.6 k 2024-03-26T06:09:36.617 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 5.4 M 2024-03-26T06:09:36.617 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 889 k 2024-03-26T06:09:36.617 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 145 k 2024-03-26T06:09:36.617 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 1.5 M 2024-03-26T06:09:36.617 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 139 k 2024-03-26T06:09:36.617 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 1.8 M 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 7.4 M 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 49 k 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 9.8 M 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 47 M 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 264 k 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 749 k 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 219 k 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 27 k 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 18 k 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T06:09:36.618 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T06:09:36.619 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 31 k 2024-03-26T06:09:36.619 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 745 k 2024-03-26T06:09:36.619 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 127 k 2024-03-26T06:09:36.619 INFO:teuthology.orchestra.run.smithi066.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T06:09:36.619 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T06:09:36.619 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T06:09:36.619 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T06:09:36.619 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T06:09:36.619 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T06:09:36.619 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 162 k 2024-03-26T06:09:36.620 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 321 k 2024-03-26T06:09:36.620 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 298 k 2024-03-26T06:09:36.620 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 99 k 2024-03-26T06:09:36.620 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 86 k 2024-03-26T06:09:36.620 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 3.1 M 2024-03-26T06:09:36.620 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 170 k 2024-03-26T06:09:36.620 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T06:09:36.620 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2024-03-26T06:09:36.620 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T06:09:36.620 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T06:09:36.620 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 3.4 M 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 3.1 M 2024-03-26T06:09:36.621 INFO:teuthology.orchestra.run.smithi066.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T06:09:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T06:09:36.623 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 20 M 2024-03-26T06:09:36.623 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 34 k 2024-03-26T06:09:36.623 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 2.1 M 2024-03-26T06:09:36.623 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 248 k 2024-03-26T06:09:36.623 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 4.7 M 2024-03-26T06:09:36.623 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 17 M 2024-03-26T06:09:36.623 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 16 k 2024-03-26T06:09:36.623 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 25 k 2024-03-26T06:09:36.623 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T06:09:36.623 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T06:09:36.623 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 163 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T06:09:36.624 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 503 k 2024-03-26T06:09:36.625 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T06:09:36.625 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 5.1 M 2024-03-26T06:09:36.625 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T06:09:36.625 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T06:09:36.625 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T06:09:36.625 INFO:teuthology.orchestra.run.smithi066.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T06:09:36.625 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T06:09:36.625 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T06:09:36.625 INFO:teuthology.orchestra.run.smithi066.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T06:09:36.625 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T06:09:36.625 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 45 k 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 131 k 2024-03-26T06:09:36.626 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T06:09:36.627 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T06:09:36.628 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T06:09:36.629 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T06:09:36.630 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout:Installing weak dependencies: 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================ 2024-03-26T06:09:36.631 INFO:teuthology.orchestra.run.smithi066.stdout:Install 150 Packages 2024-03-26T06:09:36.632 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 24 Packages 2024-03-26T06:09:36.632 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:09:36.632 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 223 M 2024-03-26T06:09:36.632 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-03-26T06:09:38.134 INFO:teuthology.orchestra.run.smithi066.stdout:(1/174): ceph-19.0.0-1964.g7e98f5f6.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-03-26T06:09:38.435 INFO:teuthology.orchestra.run.smithi066.stdout:(2/174): ceph-fuse-19.0.0-1964.g7e98f5f6.el9.x8 2.9 MB/s | 889 kB 00:00 2024-03-26T06:09:38.560 INFO:teuthology.orchestra.run.smithi066.stdout:(3/174): ceph-immutable-object-cache-19.0.0-196 1.1 MB/s | 145 kB 00:00 2024-03-26T06:09:38.918 INFO:teuthology.orchestra.run.smithi066.stdout:(4/174): ceph-mds-19.0.0-1964.g7e98f5f6.el9.x86 5.9 MB/s | 2.1 MB 00:00 2024-03-26T06:09:39.086 INFO:teuthology.orchestra.run.smithi066.stdout:(5/174): ceph-mgr-19.0.0-1964.g7e98f5f6.el9.x86 9.0 MB/s | 1.5 MB 00:00 2024-03-26T06:09:39.277 INFO:teuthology.orchestra.run.smithi066.stdout:(6/174): ceph-base-19.0.0-1964.g7e98f5f6.el9.x8 3.3 MB/s | 5.4 MB 00:01 2024-03-26T06:09:39.869 INFO:teuthology.orchestra.run.smithi066.stdout:(7/174): ceph-mon-19.0.0-1964.g7e98f5f6.el9.x86 6.0 MB/s | 4.7 MB 00:00 2024-03-26T06:09:39.892 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.1 MB 00:00 2024-03-26T06:09:40.853 INFO:teuthology.orchestra.run.smithi066.stdout:(8/174): ceph-radosgw-19.0.0-1964.g7e98f5f6.el9 9.9 MB/s | 9.8 MB 00:00 2024-03-26T06:09:40.911 INFO:teuthology.orchestra.run.smithi066.stdout:(9/174): ceph-selinux-19.0.0-1964.g7e98f5f6.el9 434 kB/s | 25 kB 00:00 2024-03-26T06:09:41.046 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-26T06:09:41.418 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-03-26T06:09:42.270 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 28 MB/s | 21 MB 00:00 2024-03-26T06:09:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:(10/174): ceph-osd-19.0.0-1964.g7e98f5f6.el9.x8 4.3 MB/s | 17 MB 00:03 2024-03-26T06:09:43.237 INFO:teuthology.orchestra.run.smithi066.stdout:(11/174): cephfs-mirror-19.0.0-1964.g7e98f5f6.e 3.2 MB/s | 219 kB 00:00 2024-03-26T06:09:43.287 INFO:teuthology.orchestra.run.smithi066.stdout:(12/174): cephfs-shell-19.0.0-1964.g7e98f5f6.el 541 kB/s | 27 kB 00:00 2024-03-26T06:09:43.337 INFO:teuthology.orchestra.run.smithi066.stdout:(13/174): libcephfs-devel-19.0.0-1964.g7e98f5f6 625 kB/s | 31 kB 00:00 2024-03-26T06:09:43.496 INFO:teuthology.orchestra.run.smithi066.stdout:(14/174): libcephfs2-19.0.0-1964.g7e98f5f6.el9. 4.6 MB/s | 745 kB 00:00 2024-03-26T06:09:43.566 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T06:09:43.567 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T06:09:43.579 INFO:teuthology.orchestra.run.smithi066.stdout:(15/174): libcephsqlite-19.0.0-1964.g7e98f5f6.e 1.9 MB/s | 163 kB 00:00 2024-03-26T06:09:43.582 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T06:09:43.583 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T06:09:43.584 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T06:09:43.585 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T06:09:43.587 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T06:09:43.591 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T06:09:43.592 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T06:09:43.630 INFO:teuthology.orchestra.run.smithi066.stdout:(16/174): librados-devel-19.0.0-1964.g7e98f5f6. 2.5 MB/s | 127 kB 00:00 2024-03-26T06:09:43.677 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-26T06:09:43.707 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================================ 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================================ 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 6.6 k 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 5.4 M 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 889 k 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 145 k 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 1.5 M 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 139 k 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 1.8 M 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 7.4 M 2024-03-26T06:09:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 49 k 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 9.8 M 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 47 M 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 264 k 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 749 k 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 219 k 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 27 k 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 18 k 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 31 k 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 745 k 2024-03-26T06:09:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 127 k 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 162 k 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 321 k 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 298 k 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 99 k 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 86 k 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 3.1 M 2024-03-26T06:09:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 170 k 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T06:09:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 3.4 M 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 3.1 M 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T06:09:43.712 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 20 M 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 34 k 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 2.1 M 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 248 k 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 4.7 M 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 17 M 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 16 k 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 25 k 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T06:09:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 163 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T06:09:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 503 k 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 5.1 M 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T06:09:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 45 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 131 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T06:09:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T06:09:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T06:09:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T06:09:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T06:09:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T06:09:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T06:09:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T06:09:43.721 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-03-26T06:09:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T06:09:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:09:43.721 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-26T06:09:43.721 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================================ 2024-03-26T06:09:43.721 INFO:teuthology.orchestra.run.smithi037.stdout:Install 150 Packages 2024-03-26T06:09:43.721 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 24 Packages 2024-03-26T06:09:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:09:43.722 INFO:teuthology.orchestra.run.smithi066.stdout:(17/174): libradosstriper1-19.0.0-1964.g7e98f5f 5.4 MB/s | 503 kB 00:00 2024-03-26T06:09:43.722 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 223 M 2024-03-26T06:09:43.722 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-26T06:09:44.472 INFO:teuthology.orchestra.run.smithi066.stdout:(18/174): librgw2-19.0.0-1964.g7e98f5f6.el9.x86 6.8 MB/s | 5.1 MB 00:00 2024-03-26T06:09:44.522 INFO:teuthology.orchestra.run.smithi066.stdout:(19/174): python3-ceph-argparse-19.0.0-1964.g7e 906 kB/s | 45 kB 00:00 2024-03-26T06:09:44.572 INFO:teuthology.orchestra.run.smithi066.stdout:(20/174): python3-ceph-common-19.0.0-1964.g7e98 2.6 MB/s | 131 kB 00:00 2024-03-26T06:09:44.623 INFO:teuthology.orchestra.run.smithi066.stdout:(21/174): python3-cephfs-19.0.0-1964.g7e98f5f6. 3.2 MB/s | 162 kB 00:00 2024-03-26T06:09:44.698 INFO:teuthology.orchestra.run.smithi066.stdout:(22/174): python3-rados-19.0.0-1964.g7e98f5f6.e 4.2 MB/s | 321 kB 00:00 2024-03-26T06:09:44.773 INFO:teuthology.orchestra.run.smithi066.stdout:(23/174): python3-rbd-19.0.0-1964.g7e98f5f6.el9 3.9 MB/s | 298 kB 00:00 2024-03-26T06:09:44.815 INFO:teuthology.orchestra.run.smithi066.stdout:(24/174): python3-rgw-19.0.0-1964.g7e98f5f6.el9 2.3 MB/s | 99 kB 00:00 2024-03-26T06:09:44.857 INFO:teuthology.orchestra.run.smithi066.stdout:(25/174): rbd-fuse-19.0.0-1964.g7e98f5f6.el9.x8 2.0 MB/s | 86 kB 00:00 2024-03-26T06:09:45.122 INFO:teuthology.orchestra.run.smithi037.stdout:(1/174): ceph-19.0.0-1964.g7e98f5f6.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-03-26T06:09:45.274 INFO:teuthology.orchestra.run.smithi066.stdout:(26/174): rbd-mirror-19.0.0-1964.g7e98f5f6.el9. 7.4 MB/s | 3.1 MB 00:00 2024-03-26T06:09:45.341 INFO:teuthology.orchestra.run.smithi066.stdout:(27/174): rbd-nbd-19.0.0-1964.g7e98f5f6.el9.x86 2.5 MB/s | 170 kB 00:00 2024-03-26T06:09:45.383 INFO:teuthology.orchestra.run.smithi066.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1964.g 815 kB/s | 34 kB 00:00 2024-03-26T06:09:45.673 INFO:teuthology.orchestra.run.smithi037.stdout:(2/174): ceph-fuse-19.0.0-1964.g7e98f5f6.el9.x8 1.6 MB/s | 889 kB 00:00 2024-03-26T06:09:45.765 INFO:teuthology.orchestra.run.smithi037.stdout:(3/174): ceph-base-19.0.0-1964.g7e98f5f6.el9.x8 5.4 MB/s | 5.4 MB 00:00 2024-03-26T06:09:45.790 INFO:teuthology.orchestra.run.smithi037.stdout:(4/174): ceph-immutable-object-cache-19.0.0-196 1.2 MB/s | 145 kB 00:00 2024-03-26T06:09:45.800 INFO:teuthology.orchestra.run.smithi066.stdout:(29/174): ceph-test-19.0.0-1964.g7e98f5f6.el9.x 9.7 MB/s | 47 MB 00:04 2024-03-26T06:09:45.817 INFO:teuthology.orchestra.run.smithi066.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1964.g7e98f5f 320 kB/s | 139 kB 00:00 2024-03-26T06:09:46.009 INFO:teuthology.orchestra.run.smithi066.stdout:(31/174): ceph-common-19.0.0-1964.g7e98f5f6.el9 2.4 MB/s | 20 MB 00:08 2024-03-26T06:09:46.048 INFO:teuthology.orchestra.run.smithi037.stdout:(5/174): ceph-mds-19.0.0-1964.g7e98f5f6.el9.x86 7.5 MB/s | 2.1 MB 00:00 2024-03-26T06:09:46.184 INFO:teuthology.orchestra.run.smithi066.stdout:(32/174): ceph-mgr-dashboard-19.0.0-1964.g7e98f 4.7 MB/s | 1.8 MB 00:00 2024-03-26T06:09:46.209 INFO:teuthology.orchestra.run.smithi066.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1964.g7e 1.2 MB/s | 248 kB 00:00 2024-03-26T06:09:46.276 INFO:teuthology.orchestra.run.smithi066.stdout:(34/174): ceph-mgr-rook-19.0.0-1964.g7e98f5f6.e 536 kB/s | 49 kB 00:00 2024-03-26T06:09:46.293 INFO:teuthology.orchestra.run.smithi066.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1964.g7 197 kB/s | 16 kB 00:00 2024-03-26T06:09:46.385 INFO:teuthology.orchestra.run.smithi066.stdout:(36/174): ceph-volume-19.0.0-1964.g7e98f5f6.el9 2.4 MB/s | 264 kB 00:00 2024-03-26T06:09:46.432 INFO:teuthology.orchestra.run.smithi037.stdout:(6/174): ceph-mgr-19.0.0-1964.g7e98f5f6.el9.x86 2.3 MB/s | 1.5 MB 00:00 2024-03-26T06:09:46.477 INFO:teuthology.orchestra.run.smithi066.stdout:(37/174): cephfs-top-19.0.0-1964.g7e98f5f6.el9. 199 kB/s | 18 kB 00:00 2024-03-26T06:09:46.619 INFO:teuthology.orchestra.run.smithi066.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2024-03-26T06:09:46.661 INFO:teuthology.orchestra.run.smithi066.stdout:(39/174): cephadm-19.0.0-1964.g7e98f5f6.el9.noa 2.0 MB/s | 749 kB 00:00 2024-03-26T06:09:46.682 INFO:teuthology.orchestra.run.smithi037.stdout:(7/174): ceph-mon-19.0.0-1964.g7e98f5f6.el9.x86 7.4 MB/s | 4.7 MB 00:00 2024-03-26T06:09:46.703 INFO:teuthology.orchestra.run.smithi066.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 920 kB 00:00 2024-03-26T06:09:46.728 INFO:teuthology.orchestra.run.smithi066.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-26T06:09:46.762 INFO:teuthology.orchestra.run.smithi066.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-03-26T06:09:46.779 INFO:teuthology.orchestra.run.smithi066.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 639 kB/s | 32 kB 00:00 2024-03-26T06:09:46.846 INFO:teuthology.orchestra.run.smithi066.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 36 MB/s | 3.0 MB 00:00 2024-03-26T06:09:46.862 INFO:teuthology.orchestra.run.smithi066.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 208 kB/s | 17 kB 00:00 2024-03-26T06:09:46.888 INFO:teuthology.orchestra.run.smithi066.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 241 kB/s | 10 kB 00:00 2024-03-26T06:09:46.913 INFO:teuthology.orchestra.run.smithi066.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-26T06:09:46.947 INFO:teuthology.orchestra.run.smithi066.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 751 kB/s | 44 kB 00:00 2024-03-26T06:09:46.990 INFO:teuthology.orchestra.run.smithi066.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.5 MB/s | 661 kB 00:00 2024-03-26T06:09:47.015 INFO:teuthology.orchestra.run.smithi066.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 578 kB 00:00 2024-03-26T06:09:47.040 INFO:teuthology.orchestra.run.smithi066.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 46 kB 00:00 2024-03-26T06:09:47.066 INFO:teuthology.orchestra.run.smithi066.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-26T06:09:47.091 INFO:teuthology.orchestra.run.smithi066.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T06:09:47.116 INFO:teuthology.orchestra.run.smithi066.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-26T06:09:47.142 INFO:teuthology.orchestra.run.smithi066.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2024-03-26T06:09:47.237 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-03-26T06:09:47.242 INFO:teuthology.orchestra.run.smithi066.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-26T06:09:47.276 INFO:teuthology.orchestra.run.smithi066.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.7 MB/s | 1.0 MB 00:00 2024-03-26T06:09:47.301 INFO:teuthology.orchestra.run.smithi066.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-26T06:09:47.426 INFO:teuthology.orchestra.run.smithi066.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 39 MB/s | 5.8 MB 00:00 2024-03-26T06:09:47.451 INFO:teuthology.orchestra.run.smithi066.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 166 kB/s | 24 kB 00:00 2024-03-26T06:09:47.477 INFO:teuthology.orchestra.run.smithi066.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 362 kB/s | 18 kB 00:00 2024-03-26T06:09:47.502 INFO:teuthology.orchestra.run.smithi066.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-26T06:09:47.527 INFO:teuthology.orchestra.run.smithi066.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 634 kB/s | 32 kB 00:00 2024-03-26T06:09:47.549 INFO:teuthology.orchestra.run.smithi037.stdout:(8/174): ceph-common-19.0.0-1964.g7e98f5f6.el9. 7.2 MB/s | 20 MB 00:02 2024-03-26T06:09:47.661 INFO:teuthology.orchestra.run.smithi066.stdout:(64/174): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-03-26T06:09:47.683 INFO:teuthology.orchestra.run.smithi037.stdout:(9/174): ceph-radosgw-19.0.0-1964.g7e98f5f6.el9 9.8 MB/s | 9.8 MB 00:00 2024-03-26T06:09:47.699 INFO:teuthology.orchestra.run.smithi037.stdout:(10/174): ceph-selinux-19.0.0-1964.g7e98f5f6.el 168 kB/s | 25 kB 00:00 2024-03-26T06:09:47.758 INFO:teuthology.orchestra.run.smithi037.stdout:(11/174): cephfs-mirror-19.0.0-1964.g7e98f5f6.e 3.7 MB/s | 219 kB 00:00 2024-03-26T06:09:47.782 INFO:teuthology.orchestra.run.smithi066.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-03-26T06:09:47.800 INFO:teuthology.orchestra.run.smithi037.stdout:(12/174): cephfs-shell-19.0.0-1964.g7e98f5f6.el 648 kB/s | 27 kB 00:00 2024-03-26T06:09:47.820 INFO:teuthology.orchestra.run.smithi066.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 752 kB/s | 217 kB 00:00 2024-03-26T06:09:47.845 INFO:teuthology.orchestra.run.smithi066.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 954 kB/s | 174 kB 00:00 2024-03-26T06:09:47.851 INFO:teuthology.orchestra.run.smithi037.stdout:(13/174): libcephfs-devel-19.0.0-1964.g7e98f5f6 623 kB/s | 31 kB 00:00 2024-03-26T06:09:47.870 INFO:teuthology.orchestra.run.smithi066.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 69 kB 00:00 2024-03-26T06:09:47.887 INFO:teuthology.orchestra.run.smithi066.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 621 kB/s | 41 kB 00:00 2024-03-26T06:09:47.912 INFO:teuthology.orchestra.run.smithi066.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-26T06:09:47.938 INFO:teuthology.orchestra.run.smithi066.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.2 MB/s | 211 kB 00:00 2024-03-26T06:09:47.963 INFO:teuthology.orchestra.run.smithi066.stdout:(72/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-26T06:09:47.967 INFO:teuthology.orchestra.run.smithi037.stdout:(14/174): libcephfs2-19.0.0-1964.g7e98f5f6.el9. 6.2 MB/s | 745 kB 00:00 2024-03-26T06:09:48.008 INFO:teuthology.orchestra.run.smithi066.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 497 kB/s | 47 kB 00:00 2024-03-26T06:09:48.030 INFO:teuthology.orchestra.run.smithi066.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 466 kB/s | 43 kB 00:00 2024-03-26T06:09:48.034 INFO:teuthology.orchestra.run.smithi037.stdout:(15/174): libcephsqlite-19.0.0-1964.g7e98f5f6.e 2.4 MB/s | 163 kB 00:00 2024-03-26T06:09:48.047 INFO:teuthology.orchestra.run.smithi066.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 39 kB 00:00 2024-03-26T06:09:48.110 INFO:teuthology.orchestra.run.smithi037.stdout:(16/174): librados-devel-19.0.0-1964.g7e98f5f6. 1.6 MB/s | 127 kB 00:00 2024-03-26T06:09:48.147 INFO:teuthology.orchestra.run.smithi066.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 350 kB/s | 41 kB 00:00 2024-03-26T06:09:48.194 INFO:teuthology.orchestra.run.smithi037.stdout:(17/174): libradosstriper1-19.0.0-1964.g7e98f5f 5.9 MB/s | 503 kB 00:00 2024-03-26T06:09:48.348 INFO:teuthology.orchestra.run.smithi066.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-03-26T06:09:48.372 INFO:teuthology.orchestra.run.smithi066.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 930 kB/s | 300 kB 00:00 2024-03-26T06:09:48.389 INFO:teuthology.orchestra.run.smithi066.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 235 kB/s | 56 kB 00:00 2024-03-26T06:09:48.423 INFO:teuthology.orchestra.run.smithi066.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 838 kB/s | 63 kB 00:00 2024-03-26T06:09:48.449 INFO:teuthology.orchestra.run.smithi066.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-26T06:09:48.474 INFO:teuthology.orchestra.run.smithi066.stdout:(82/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 526 kB/s | 44 kB 00:00 2024-03-26T06:09:48.507 INFO:teuthology.orchestra.run.smithi066.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2024-03-26T06:09:48.541 INFO:teuthology.orchestra.run.smithi066.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-26T06:09:48.566 INFO:teuthology.orchestra.run.smithi066.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-26T06:09:48.591 INFO:teuthology.orchestra.run.smithi066.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-26T06:09:48.616 INFO:teuthology.orchestra.run.smithi066.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-26T06:09:48.742 INFO:teuthology.orchestra.run.smithi066.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-26T06:09:48.744 INFO:teuthology.orchestra.run.smithi037.stdout:(18/174): librgw2-19.0.0-1964.g7e98f5f6.el9.x86 9.3 MB/s | 5.1 MB 00:00 2024-03-26T06:09:48.776 INFO:teuthology.orchestra.run.smithi066.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-26T06:09:48.819 INFO:teuthology.orchestra.run.smithi037.stdout:(19/174): python3-ceph-argparse-19.0.0-1964.g7e 604 kB/s | 45 kB 00:00 2024-03-26T06:09:48.886 INFO:teuthology.orchestra.run.smithi037.stdout:(20/174): python3-ceph-common-19.0.0-1964.g7e98 1.9 MB/s | 131 kB 00:00 2024-03-26T06:09:48.892 INFO:teuthology.orchestra.run.smithi066.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-03-26T06:09:48.926 INFO:teuthology.orchestra.run.smithi066.stdout:(91/174): python3-requests-2.25.1-8.el9.noarch. 680 kB/s | 125 kB 00:00 2024-03-26T06:09:48.962 INFO:teuthology.orchestra.run.smithi037.stdout:(21/174): python3-cephfs-19.0.0-1964.g7e98f5f6. 2.1 MB/s | 162 kB 00:00 2024-03-26T06:09:48.976 INFO:teuthology.orchestra.run.smithi066.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 107 kB/s | 21 kB 00:00 2024-03-26T06:09:49.053 INFO:teuthology.orchestra.run.smithi037.stdout:(22/174): python3-rados-19.0.0-1964.g7e98f5f6.e 3.4 MB/s | 321 kB 00:00 2024-03-26T06:09:49.129 INFO:teuthology.orchestra.run.smithi037.stdout:(23/174): python3-rbd-19.0.0-1964.g7e98f5f6.el9 3.9 MB/s | 298 kB 00:00 2024-03-26T06:09:49.135 INFO:teuthology.orchestra.run.smithi066.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 105 kB/s | 25 kB 00:00 2024-03-26T06:09:49.196 INFO:teuthology.orchestra.run.smithi037.stdout:(24/174): python3-rgw-19.0.0-1964.g7e98f5f6.el9 1.5 MB/s | 99 kB 00:00 2024-03-26T06:09:49.210 INFO:teuthology.orchestra.run.smithi066.stdout:(94/174): postgresql-13.14-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-03-26T06:09:49.262 INFO:teuthology.orchestra.run.smithi037.stdout:(25/174): rbd-fuse-19.0.0-1964.g7e98f5f6.el9.x8 1.3 MB/s | 86 kB 00:00 2024-03-26T06:09:49.360 INFO:teuthology.orchestra.run.smithi066.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 894 kB/s | 133 kB 00:00 2024-03-26T06:09:49.394 INFO:teuthology.orchestra.run.smithi066.stdout:(96/174): postgresql-contrib-13.14-1.el9.x86_64 3.4 MB/s | 911 kB 00:00 2024-03-26T06:09:49.477 INFO:teuthology.orchestra.run.smithi066.stdout:(97/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 7.9 MB/s | 4.1 MB 00:00 2024-03-26T06:09:49.621 INFO:teuthology.orchestra.run.smithi037.stdout:(26/174): rbd-mirror-19.0.0-1964.g7e98f5f6.el9. 8.6 MB/s | 3.1 MB 00:00 2024-03-26T06:09:49.627 INFO:teuthology.orchestra.run.smithi066.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-03-26T06:09:49.651 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T06:09:49.652 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T06:09:49.653 INFO:teuthology.orchestra.run.smithi066.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-03-26T06:09:49.666 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T06:09:49.667 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T06:09:49.668 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T06:09:49.669 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T06:09:49.671 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T06:09:49.674 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T06:09:49.675 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T06:09:49.680 INFO:teuthology.orchestra.run.smithi037.stdout:(27/174): rbd-nbd-19.0.0-1964.g7e98f5f6.el9.x86 2.9 MB/s | 170 kB 00:00 2024-03-26T06:09:49.747 INFO:teuthology.orchestra.run.smithi037.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1964.g 510 kB/s | 34 kB 00:00 2024-03-26T06:09:49.756 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-26T06:09:49.771 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-26T06:09:49.771 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 6.6 k 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 5.4 M 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 889 k 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 145 k 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 1.5 M 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 139 k 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 1.8 M 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 7.4 M 2024-03-26T06:09:49.772 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 49 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 9.8 M 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 47 M 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 264 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 749 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 219 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 27 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 18 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 31 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 745 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 127 k 2024-03-26T06:09:49.773 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 162 k 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 321 k 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 298 k 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 99 k 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 86 k 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 3.1 M 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 170 k 2024-03-26T06:09:49.774 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T06:09:49.775 INFO:teuthology.orchestra.run.smithi149.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 3.4 M 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 3.1 M 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T06:09:49.776 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T06:09:49.777 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 20 M 2024-03-26T06:09:49.777 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 34 k 2024-03-26T06:09:49.777 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 2.1 M 2024-03-26T06:09:49.777 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 248 k 2024-03-26T06:09:49.777 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 4.7 M 2024-03-26T06:09:49.777 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 17 M 2024-03-26T06:09:49.777 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1964.g7e98f5f6.el9 ceph-noarch 16 k 2024-03-26T06:09:49.777 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 25 k 2024-03-26T06:09:49.777 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T06:09:49.777 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T06:09:49.777 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 163 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 503 k 2024-03-26T06:09:49.778 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 5.1 M 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T06:09:49.779 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 45 k 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.0.0-1964.g7e98f5f6.el9 ceph 131 k 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T06:09:49.780 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T06:09:49.781 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T06:09:49.782 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T06:09:49.783 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T06:09:49.784 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T06:09:49.785 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-03-26T06:09:49.785 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T06:09:49.785 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:09:49.785 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-26T06:09:49.785 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-26T06:09:49.785 INFO:teuthology.orchestra.run.smithi149.stdout:Install 150 Packages 2024-03-26T06:09:49.785 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 24 Packages 2024-03-26T06:09:49.785 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:09:49.785 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 223 M 2024-03-26T06:09:49.785 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-26T06:09:49.814 INFO:teuthology.orchestra.run.smithi037.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1964.g7e98f5f 2.0 MB/s | 139 kB 00:00 2024-03-26T06:09:49.819 INFO:teuthology.orchestra.run.smithi066.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 253 kB/s | 36 kB 00:00 2024-03-26T06:09:49.845 INFO:teuthology.orchestra.run.smithi066.stdout:(101/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 221 kB 00:00 2024-03-26T06:09:50.012 INFO:teuthology.orchestra.run.smithi066.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-03-26T06:09:50.022 INFO:teuthology.orchestra.run.smithi037.stdout:(30/174): ceph-mgr-dashboard-19.0.0-1964.g7e98f 8.6 MB/s | 1.8 MB 00:00 2024-03-26T06:09:50.054 INFO:teuthology.orchestra.run.smithi066.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 449 kB/s | 94 kB 00:00 2024-03-26T06:09:50.162 INFO:teuthology.orchestra.run.smithi066.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 149 kB/s | 22 kB 00:00 2024-03-26T06:09:50.338 INFO:teuthology.orchestra.run.smithi066.stdout:(105/174): postgresql-server-13.14-1.el9.x86_64 5.9 MB/s | 5.8 MB 00:00 2024-03-26T06:09:50.579 INFO:teuthology.orchestra.run.smithi066.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.6 MB/s | 863 kB 00:00 2024-03-26T06:09:50.721 INFO:teuthology.orchestra.run.smithi149.stdout:(1/174): ceph-19.0.0-1964.g7e98f5f6.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-03-26T06:09:50.798 INFO:teuthology.orchestra.run.smithi037.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 9.5 MB/s | 7.4 MB 00:00 2024-03-26T06:09:50.880 INFO:teuthology.orchestra.run.smithi066.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-03-26T06:09:50.905 INFO:teuthology.orchestra.run.smithi066.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 545 kB/s | 309 kB 00:00 2024-03-26T06:09:50.963 INFO:teuthology.orchestra.run.smithi066.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 684 kB/s | 548 kB 00:00 2024-03-26T06:09:51.005 INFO:teuthology.orchestra.run.smithi066.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-03-26T06:09:51.081 INFO:teuthology.orchestra.run.smithi066.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-03-26T06:09:51.148 INFO:teuthology.orchestra.run.smithi066.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 474 kB/s | 67 kB 00:00 2024-03-26T06:09:51.265 INFO:teuthology.orchestra.run.smithi037.stdout:(32/174): ceph-test-19.0.0-1964.g7e98f5f6.el9.x 13 MB/s | 47 MB 00:03 2024-03-26T06:09:51.290 INFO:teuthology.orchestra.run.smithi037.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1964.g7e 504 kB/s | 248 kB 00:00 2024-03-26T06:09:51.298 INFO:teuthology.orchestra.run.smithi066.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 696 kB/s | 151 kB 00:00 2024-03-26T06:09:51.332 INFO:teuthology.orchestra.run.smithi037.stdout:(34/174): ceph-mgr-rook-19.0.0-1964.g7e98f5f6.e 740 kB/s | 49 kB 00:00 2024-03-26T06:09:51.366 INFO:teuthology.orchestra.run.smithi037.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1964.g7 218 kB/s | 16 kB 00:00 2024-03-26T06:09:51.423 INFO:teuthology.orchestra.run.smithi066.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.0 MB/s | 838 kB 00:00 2024-03-26T06:09:51.424 INFO:teuthology.orchestra.run.smithi037.stdout:(36/174): ceph-volume-19.0.0-1964.g7e98f5f6.el9 2.8 MB/s | 264 kB 00:00 2024-03-26T06:09:51.515 INFO:teuthology.orchestra.run.smithi066.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 474 kB/s | 43 kB 00:00 2024-03-26T06:09:51.516 INFO:teuthology.orchestra.run.smithi037.stdout:(37/174): cephfs-top-19.0.0-1964.g7e98f5f6.el9. 199 kB/s | 18 kB 00:00 2024-03-26T06:09:51.579 INFO:teuthology.orchestra.run.smithi149.stdout:(2/174): ceph-fuse-19.0.0-1964.g7e98f5f6.el9.x8 1.0 MB/s | 889 kB 00:00 2024-03-26T06:09:51.607 INFO:teuthology.orchestra.run.smithi066.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-03-26T06:09:51.624 INFO:teuthology.orchestra.run.smithi066.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-03-26T06:09:51.658 INFO:teuthology.orchestra.run.smithi037.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2024-03-26T06:09:51.700 INFO:teuthology.orchestra.run.smithi066.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-03-26T06:09:51.717 INFO:teuthology.orchestra.run.smithi037.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-03-26T06:09:51.721 INFO:teuthology.orchestra.run.smithi149.stdout:(3/174): ceph-base-19.0.0-1964.g7e98f5f6.el9.x8 3.7 MB/s | 5.4 MB 00:01 2024-03-26T06:09:51.724 INFO:teuthology.orchestra.run.smithi066.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-26T06:09:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-26T06:09:51.772 INFO:teuthology.orchestra.run.smithi149.stdout:(4/174): ceph-immutable-object-cache-19.0.0-196 756 kB/s | 145 kB 00:00 2024-03-26T06:09:51.784 INFO:teuthology.orchestra.run.smithi037.stdout:(41/174): cephadm-19.0.0-1964.g7e98f5f6.el9.noa 1.7 MB/s | 749 kB 00:00 2024-03-26T06:09:51.809 INFO:teuthology.orchestra.run.smithi037.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-03-26T06:09:51.825 INFO:teuthology.orchestra.run.smithi066.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-03-26T06:09:51.826 INFO:teuthology.orchestra.run.smithi037.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 763 kB/s | 32 kB 00:00 2024-03-26T06:09:51.902 INFO:teuthology.orchestra.run.smithi037.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 32 MB/s | 3.0 MB 00:00 2024-03-26T06:09:51.908 INFO:teuthology.orchestra.run.smithi066.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 424 kB/s | 35 kB 00:00 2024-03-26T06:09:51.918 INFO:teuthology.orchestra.run.smithi037.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 189 kB/s | 17 kB 00:00 2024-03-26T06:09:51.936 INFO:teuthology.orchestra.run.smithi037.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-03-26T06:09:51.942 INFO:teuthology.orchestra.run.smithi066.stdout:(122/174): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-03-26T06:09:51.961 INFO:teuthology.orchestra.run.smithi037.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-26T06:09:51.978 INFO:teuthology.orchestra.run.smithi037.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-26T06:09:52.025 INFO:teuthology.orchestra.run.smithi066.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-03-26T06:09:52.105 INFO:teuthology.orchestra.run.smithi149.stdout:(5/174): ceph-mds-19.0.0-1964.g7e98f5f6.el9.x86 5.5 MB/s | 2.1 MB 00:00 2024-03-26T06:09:52.118 INFO:teuthology.orchestra.run.smithi066.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-03-26T06:09:52.120 INFO:teuthology.orchestra.run.smithi037.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 661 kB 00:00 2024-03-26T06:09:52.153 INFO:teuthology.orchestra.run.smithi037.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 3.2 MB/s | 578 kB 00:00 2024-03-26T06:09:52.170 INFO:teuthology.orchestra.run.smithi037.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 46 kB 00:00 2024-03-26T06:09:52.196 INFO:teuthology.orchestra.run.smithi037.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-26T06:09:52.209 INFO:teuthology.orchestra.run.smithi066.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-03-26T06:09:52.221 INFO:teuthology.orchestra.run.smithi037.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T06:09:52.246 INFO:teuthology.orchestra.run.smithi037.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-26T06:09:52.272 INFO:teuthology.orchestra.run.smithi037.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 688 kB/s | 34 kB 00:00 2024-03-26T06:09:52.318 INFO:teuthology.orchestra.run.smithi066.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-03-26T06:09:52.418 INFO:teuthology.orchestra.run.smithi066.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-03-26T06:09:52.447 INFO:teuthology.orchestra.run.smithi037.stdout:(56/174): ceph-osd-19.0.0-1964.g7e98f5f6.el9.x8 2.8 MB/s | 17 MB 00:06 2024-03-26T06:09:52.510 INFO:teuthology.orchestra.run.smithi066.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-03-26T06:09:52.522 INFO:teuthology.orchestra.run.smithi037.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-03-26T06:09:52.556 INFO:teuthology.orchestra.run.smithi037.stdout:(58/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-03-26T06:09:52.581 INFO:teuthology.orchestra.run.smithi037.stdout:(59/174): python3-attrs-20.3.0-7.el9.noarch.rpm 595 kB/s | 78 kB 00:00 2024-03-26T06:09:52.611 INFO:teuthology.orchestra.run.smithi066.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-03-26T06:09:52.614 INFO:teuthology.orchestra.run.smithi149.stdout:(6/174): ceph-mgr-19.0.0-1964.g7e98f5f6.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-03-26T06:09:52.715 INFO:teuthology.orchestra.run.smithi037.stdout:(60/174): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 5.8 MB 00:00 2024-03-26T06:09:52.719 INFO:teuthology.orchestra.run.smithi066.stdout:(130/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-03-26T06:09:52.811 INFO:teuthology.orchestra.run.smithi066.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 506 kB/s | 46 kB 00:00 2024-03-26T06:09:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 76 kB/s | 18 kB 00:00 2024-03-26T06:09:52.912 INFO:teuthology.orchestra.run.smithi066.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 790 kB/s | 79 kB 00:00 2024-03-26T06:09:52.915 INFO:teuthology.orchestra.run.smithi037.stdout:(62/174): python3-jsonpatch-1.21-16.el9.noarch. 74 kB/s | 24 kB 00:00 2024-03-26T06:09:52.941 INFO:teuthology.orchestra.run.smithi037.stdout:(63/174): python3-mako-1.1.4-6.el9.noarch.rpm 640 kB/s | 144 kB 00:00 2024-03-26T06:09:52.945 INFO:teuthology.orchestra.run.smithi066.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 3.1 MB/s | 1.0 MB 00:00 2024-03-26T06:09:52.966 INFO:teuthology.orchestra.run.smithi037.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 222 kB/s | 32 kB 00:00 2024-03-26T06:09:52.991 INFO:teuthology.orchestra.run.smithi037.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 217 kB 00:00 2024-03-26T06:09:53.012 INFO:teuthology.orchestra.run.smithi066.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-03-26T06:09:53.031 INFO:teuthology.orchestra.run.smithi149.stdout:(7/174): ceph-mon-19.0.0-1964.g7e98f5f6.el9.x86 5.1 MB/s | 4.7 MB 00:00 2024-03-26T06:09:53.046 INFO:teuthology.orchestra.run.smithi066.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-03-26T06:09:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:(66/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-26T06:09:53.104 INFO:teuthology.orchestra.run.smithi066.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-03-26T06:09:53.116 INFO:teuthology.orchestra.run.smithi037.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-03-26T06:09:53.142 INFO:teuthology.orchestra.run.smithi037.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 466 kB/s | 69 kB 00:00 2024-03-26T06:09:53.146 INFO:teuthology.orchestra.run.smithi066.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 898 kB/s | 90 kB 00:00 2024-03-26T06:09:53.158 INFO:teuthology.orchestra.run.smithi037.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 622 kB/s | 41 kB 00:00 2024-03-26T06:09:53.184 INFO:teuthology.orchestra.run.smithi037.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-03-26T06:09:53.209 INFO:teuthology.orchestra.run.smithi037.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-26T06:09:53.213 INFO:teuthology.orchestra.run.smithi066.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-03-26T06:09:53.234 INFO:teuthology.orchestra.run.smithi037.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-26T06:09:53.238 INFO:teuthology.orchestra.run.smithi066.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-03-26T06:09:53.239 INFO:teuthology.orchestra.run.smithi149.stdout:(8/174): ceph-common-19.0.0-1964.g7e98f5f6.el9. 6.7 MB/s | 20 MB 00:02 2024-03-26T06:09:53.251 INFO:teuthology.orchestra.run.smithi037.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 708 kB/s | 47 kB 00:00 2024-03-26T06:09:53.269 INFO:teuthology.orchestra.run.smithi037.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2024-03-26T06:09:53.313 INFO:teuthology.orchestra.run.smithi066.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-26T06:09:53.314 INFO:teuthology.orchestra.run.smithi149.stdout:(9/174): ceph-selinux-19.0.0-1964.g7e98f5f6.el9 336 kB/s | 25 kB 00:00 2024-03-26T06:09:53.339 INFO:teuthology.orchestra.run.smithi066.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 591 kB/s | 59 kB 00:00 2024-03-26T06:09:53.402 INFO:teuthology.orchestra.run.smithi037.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 259 kB/s | 39 kB 00:00 2024-03-26T06:09:53.422 INFO:teuthology.orchestra.run.smithi066.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-03-26T06:09:53.448 INFO:teuthology.orchestra.run.smithi066.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-03-26T06:09:53.460 INFO:teuthology.orchestra.run.smithi037.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 216 kB/s | 41 kB 00:00 2024-03-26T06:09:53.523 INFO:teuthology.orchestra.run.smithi066.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-26T06:09:53.540 INFO:teuthology.orchestra.run.smithi066.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 978 kB/s | 90 kB 00:00 2024-03-26T06:09:53.632 INFO:teuthology.orchestra.run.smithi066.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-03-26T06:09:53.690 INFO:teuthology.orchestra.run.smithi066.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-03-26T06:09:53.732 INFO:teuthology.orchestra.run.smithi066.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-26T06:09:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-03-26T06:09:53.769 INFO:teuthology.orchestra.run.smithi037.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 822 kB/s | 300 kB 00:00 2024-03-26T06:09:53.786 INFO:teuthology.orchestra.run.smithi037.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 174 kB/s | 56 kB 00:00 2024-03-26T06:09:53.803 INFO:teuthology.orchestra.run.smithi037.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-03-26T06:09:53.820 INFO:teuthology.orchestra.run.smithi037.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-26T06:09:53.845 INFO:teuthology.orchestra.run.smithi037.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-26T06:09:53.871 INFO:teuthology.orchestra.run.smithi037.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-26T06:09:53.896 INFO:teuthology.orchestra.run.smithi037.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-26T06:09:53.938 INFO:teuthology.orchestra.run.smithi037.stdout:(85/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-26T06:09:53.963 INFO:teuthology.orchestra.run.smithi037.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-26T06:09:53.997 INFO:teuthology.orchestra.run.smithi037.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 187 kB 00:00 2024-03-26T06:09:54.080 INFO:teuthology.orchestra.run.smithi037.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-03-26T06:09:54.085 INFO:teuthology.orchestra.run.smithi066.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-03-26T06:09:54.105 INFO:teuthology.orchestra.run.smithi037.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-03-26T06:09:54.172 INFO:teuthology.orchestra.run.smithi037.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-03-26T06:09:54.198 INFO:teuthology.orchestra.run.smithi037.stdout:(91/174): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-03-26T06:09:54.382 INFO:teuthology.orchestra.run.smithi037.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 78 kB/s | 21 kB 00:00 2024-03-26T06:09:54.448 INFO:teuthology.orchestra.run.smithi037.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 92 kB/s | 25 kB 00:00 2024-03-26T06:09:55.008 INFO:teuthology.orchestra.run.smithi066.stdout:(150/174): libarrow-9.0.0-10.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:04 2024-03-26T06:09:55.049 INFO:teuthology.orchestra.run.smithi037.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 1.5 MB/s | 911 kB 00:00 2024-03-26T06:09:55.050 INFO:teuthology.orchestra.run.smithi066.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-26T06:09:55.090 INFO:teuthology.orchestra.run.smithi037.stdout:(95/174): postgresql-13.14-1.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-03-26T06:09:55.131 INFO:teuthology.orchestra.run.smithi149.stdout:(10/174): ceph-radosgw-19.0.0-1964.g7e98f5f6.el 4.7 MB/s | 9.8 MB 00:02 2024-03-26T06:09:55.157 INFO:teuthology.orchestra.run.smithi037.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-03-26T06:09:55.192 INFO:teuthology.orchestra.run.smithi066.stdout:(152/174): librbd1-19.0.0-1964.g7e98f5f6.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-03-26T06:09:55.240 INFO:teuthology.orchestra.run.smithi149.stdout:(11/174): cephfs-mirror-19.0.0-1964.g7e98f5f6.e 2.0 MB/s | 219 kB 00:00 2024-03-26T06:09:55.259 INFO:teuthology.orchestra.run.smithi066.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-03-26T06:09:55.266 INFO:teuthology.orchestra.run.smithi037.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-03-26T06:09:55.315 INFO:teuthology.orchestra.run.smithi149.stdout:(12/174): cephfs-shell-19.0.0-1964.g7e98f5f6.el 361 kB/s | 27 kB 00:00 2024-03-26T06:09:55.326 INFO:teuthology.orchestra.run.smithi066.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 632 kB/s | 42 kB 00:00 2024-03-26T06:09:55.390 INFO:teuthology.orchestra.run.smithi149.stdout:(13/174): libcephfs-devel-19.0.0-1964.g7e98f5f6 416 kB/s | 31 kB 00:00 2024-03-26T06:09:55.391 INFO:teuthology.orchestra.run.smithi037.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-03-26T06:09:55.401 INFO:teuthology.orchestra.run.smithi066.stdout:(155/174): NetworkManager-libnm-1.47.2-1.el9.x8 8.8 MB/s | 1.8 MB 00:00 2024-03-26T06:09:55.443 INFO:teuthology.orchestra.run.smithi066.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.1 MB/s | 248 kB 00:00 2024-03-26T06:09:55.477 INFO:teuthology.orchestra.run.smithi066.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 950 kB/s | 71 kB 00:00 2024-03-26T06:09:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.9 MB/s | 221 kB 00:00 2024-03-26T06:09:55.527 INFO:teuthology.orchestra.run.smithi066.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 437 kB/s | 36 kB 00:00 2024-03-26T06:09:55.569 INFO:teuthology.orchestra.run.smithi066.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.1 MB/s | 192 kB 00:00 2024-03-26T06:09:55.607 INFO:teuthology.orchestra.run.smithi149.stdout:(14/174): libcephfs2-19.0.0-1964.g7e98f5f6.el9. 3.4 MB/s | 745 kB 00:00 2024-03-26T06:09:55.609 INFO:teuthology.orchestra.run.smithi037.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 359 kB/s | 36 kB 00:00 2024-03-26T06:09:55.652 INFO:teuthology.orchestra.run.smithi066.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-03-26T06:09:55.674 INFO:teuthology.orchestra.run.smithi149.stdout:(15/174): libcephsqlite-19.0.0-1964.g7e98f5f6.e 2.4 MB/s | 163 kB 00:00 2024-03-26T06:09:55.675 INFO:teuthology.orchestra.run.smithi037.stdout:(101/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.8 MB/s | 4.1 MB 00:01 2024-03-26T06:09:55.678 INFO:teuthology.orchestra.run.smithi066.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 212 kB/s | 23 kB 00:00 2024-03-26T06:09:55.717 INFO:teuthology.orchestra.run.smithi037.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-03-26T06:09:55.741 INFO:teuthology.orchestra.run.smithi149.stdout:(16/174): librados-devel-19.0.0-1964.g7e98f5f6. 1.9 MB/s | 127 kB 00:00 2024-03-26T06:09:55.776 INFO:teuthology.orchestra.run.smithi037.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 932 kB/s | 94 kB 00:00 2024-03-26T06:09:55.778 INFO:teuthology.orchestra.run.smithi066.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 866 kB/s | 108 kB 00:00 2024-03-26T06:09:55.810 INFO:teuthology.orchestra.run.smithi037.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-03-26T06:09:55.820 INFO:teuthology.orchestra.run.smithi066.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 155 kB 00:00 2024-03-26T06:09:55.884 INFO:teuthology.orchestra.run.smithi149.stdout:(17/174): libradosstriper1-19.0.0-1964.g7e98f5f 3.5 MB/s | 503 kB 00:00 2024-03-26T06:09:55.922 INFO:teuthology.orchestra.run.smithi066.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 957 kB/s | 136 kB 00:00 2024-03-26T06:09:55.937 INFO:teuthology.orchestra.run.smithi066.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 546 kB/s | 64 kB 00:00 2024-03-26T06:09:55.987 INFO:teuthology.orchestra.run.smithi066.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 429 kB/s | 29 kB 00:00 2024-03-26T06:09:56.188 INFO:teuthology.orchestra.run.smithi066.stdout:(167/174): librados2-19.0.0-1964.g7e98f5f6.el9. 1.4 MB/s | 3.4 MB 00:02 2024-03-26T06:09:56.218 INFO:teuthology.orchestra.run.smithi037.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-03-26T06:09:56.230 INFO:teuthology.orchestra.run.smithi066.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 4.1 MB/s | 1.2 MB 00:00 2024-03-26T06:09:56.372 INFO:teuthology.orchestra.run.smithi066.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.2 MB/s | 465 kB 00:00 2024-03-26T06:09:56.447 INFO:teuthology.orchestra.run.smithi066.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 414 kB/s | 31 kB 00:00 2024-03-26T06:09:56.506 INFO:teuthology.orchestra.run.smithi066.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.2 MB/s | 2.2 MB 00:00 2024-03-26T06:09:56.569 INFO:teuthology.orchestra.run.smithi037.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 722 kB/s | 548 kB 00:00 2024-03-26T06:09:56.575 INFO:teuthology.orchestra.run.smithi149.stdout:(18/174): ceph-osd-19.0.0-1964.g7e98f5f6.el9.x8 4.2 MB/s | 17 MB 00:03 2024-03-26T06:09:56.647 INFO:teuthology.orchestra.run.smithi066.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.0 MB/s | 2.3 MB 00:00 2024-03-26T06:09:56.667 INFO:teuthology.orchestra.run.smithi149.stdout:(19/174): python3-ceph-argparse-19.0.0-1964.g7e 493 kB/s | 45 kB 00:00 2024-03-26T06:09:56.677 INFO:teuthology.orchestra.run.smithi037.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-03-26T06:09:56.681 INFO:teuthology.orchestra.run.smithi066.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.6 MB/s | 1.1 MB 00:00 2024-03-26T06:09:56.731 INFO:teuthology.orchestra.run.smithi066.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 330 kB/s | 74 kB 00:00 2024-03-26T06:09:56.738 INFO:teuthology.orchestra.run.smithi037.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 597 kB/s | 309 kB 00:00 2024-03-26T06:09:56.739 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-03-26T06:09:56.739 INFO:teuthology.orchestra.run.smithi066.stdout:Total 11 MB/s | 223 MB 00:20 2024-03-26T06:09:56.776 INFO:teuthology.orchestra.run.smithi149.stdout:(20/174): python3-ceph-common-19.0.0-1964.g7e98 1.2 MB/s | 131 kB 00:00 2024-03-26T06:09:56.853 INFO:teuthology.orchestra.run.smithi037.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-03-26T06:09:56.885 INFO:teuthology.orchestra.run.smithi149.stdout:(21/174): python3-cephfs-19.0.0-1964.g7e98f5f6. 1.5 MB/s | 162 kB 00:00 2024-03-26T06:09:56.961 INFO:teuthology.orchestra.run.smithi037.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-03-26T06:09:57.104 INFO:teuthology.orchestra.run.smithi037.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 475 kB/s | 67 kB 00:00 2024-03-26T06:09:57.160 INFO:teuthology.orchestra.run.smithi149.stdout:(22/174): librgw2-19.0.0-1964.g7e98f5f6.el9.x86 4.0 MB/s | 5.1 MB 00:01 2024-03-26T06:09:57.229 INFO:teuthology.orchestra.run.smithi037.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-03-26T06:09:57.307 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-03-26T06:09:57.310 INFO:teuthology.orchestra.run.smithi149.stdout:(23/174): python3-rbd-19.0.0-1964.g7e98f5f6.el9 1.9 MB/s | 298 kB 00:00 2024-03-26T06:09:57.397 INFO:teuthology.orchestra.run.smithi149.stdout:(24/174): python3-rgw-19.0.0-1964.g7e98f5f6.el9 1.1 MB/s | 99 kB 00:00 2024-03-26T06:09:57.447 INFO:teuthology.orchestra.run.smithi149.stdout:(25/174): python3-rados-19.0.0-1964.g7e98f5f6.e 571 kB/s | 321 kB 00:00 2024-03-26T06:09:57.473 INFO:teuthology.orchestra.run.smithi149.stdout:(26/174): rbd-fuse-19.0.0-1964.g7e98f5f6.el9.x8 1.1 MB/s | 86 kB 00:00 2024-03-26T06:09:57.565 INFO:teuthology.orchestra.run.smithi149.stdout:(27/174): rbd-nbd-19.0.0-1964.g7e98f5f6.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-26T06:09:57.579 INFO:teuthology.orchestra.run.smithi037.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2024-03-26T06:09:57.609 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-03-26T06:09:57.609 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-03-26T06:09:57.682 INFO:teuthology.orchestra.run.smithi149.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1964.g 291 kB/s | 34 kB 00:00 2024-03-26T06:09:57.749 INFO:teuthology.orchestra.run.smithi149.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1964.g7e98f5f 2.0 MB/s | 139 kB 00:00 2024-03-26T06:09:57.788 INFO:teuthology.orchestra.run.smithi037.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-03-26T06:09:57.913 INFO:teuthology.orchestra.run.smithi037.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 347 kB/s | 43 kB 00:00 2024-03-26T06:09:58.097 INFO:teuthology.orchestra.run.smithi037.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 175 kB/s | 32 kB 00:00 2024-03-26T06:09:58.132 INFO:teuthology.orchestra.run.smithi149.stdout:(30/174): rbd-mirror-19.0.0-1964.g7e98f5f6.el9. 4.5 MB/s | 3.1 MB 00:00 2024-03-26T06:09:58.174 INFO:teuthology.orchestra.run.smithi149.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1964.g7e98f 4.2 MB/s | 1.8 MB 00:00 2024-03-26T06:09:58.214 INFO:teuthology.orchestra.run.smithi037.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-03-26T06:09:58.283 INFO:teuthology.orchestra.run.smithi149.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1964.g7e 2.2 MB/s | 248 kB 00:00 2024-03-26T06:09:58.397 INFO:teuthology.orchestra.run.smithi037.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 937 kB/s | 172 kB 00:00 2024-03-26T06:09:58.408 INFO:teuthology.orchestra.run.smithi149.stdout:(33/174): ceph-mgr-rook-19.0.0-1964.g7e98f5f6.e 395 kB/s | 49 kB 00:00 2024-03-26T06:09:58.517 INFO:teuthology.orchestra.run.smithi149.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1964.g7 152 kB/s | 16 kB 00:00 2024-03-26T06:09:58.581 INFO:teuthology.orchestra.run.smithi037.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-03-26T06:09:58.625 INFO:teuthology.orchestra.run.smithi149.stdout:(35/174): ceph-volume-19.0.0-1964.g7e98f5f6.el9 2.4 MB/s | 264 kB 00:00 2024-03-26T06:09:58.673 INFO:teuthology.orchestra.run.smithi037.stdout:(120/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.2 MB/s | 4.4 MB 00:01 2024-03-26T06:09:58.731 INFO:teuthology.orchestra.run.smithi037.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-03-26T06:09:58.798 INFO:teuthology.orchestra.run.smithi037.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 281 kB/s | 35 kB 00:00 2024-03-26T06:09:58.865 INFO:teuthology.orchestra.run.smithi037.stdout:(123/174): postgresql-server-13.14-1.el9.x86_64 1.5 MB/s | 5.8 MB 00:03 2024-03-26T06:09:58.890 INFO:teuthology.orchestra.run.smithi037.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.2 MB/s | 203 kB 00:00 2024-03-26T06:09:58.892 INFO:teuthology.orchestra.run.smithi149.stdout:(36/174): cephadm-19.0.0-1964.g7e98f5f6.el9.noa 2.7 MB/s | 749 kB 00:00 2024-03-26T06:09:58.993 INFO:teuthology.orchestra.run.smithi149.stdout:(37/174): cephfs-top-19.0.0-1964.g7e98f5f6.el9. 183 kB/s | 18 kB 00:00 2024-03-26T06:09:59.024 INFO:teuthology.orchestra.run.smithi037.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-03-26T06:09:59.141 INFO:teuthology.orchestra.run.smithi037.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2024-03-26T06:09:59.201 INFO:teuthology.orchestra.run.smithi149.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2024-03-26T06:09:59.266 INFO:teuthology.orchestra.run.smithi037.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-03-26T06:09:59.285 INFO:teuthology.orchestra.run.smithi149.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 920 kB 00:00 2024-03-26T06:09:59.334 INFO:teuthology.orchestra.run.smithi037.stdout:(128/174): python3-grpcio-tools-1.46.7-10.el9.x 309 kB/s | 144 kB 00:00 2024-03-26T06:09:59.335 INFO:teuthology.orchestra.run.smithi149.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-26T06:09:59.392 INFO:teuthology.orchestra.run.smithi037.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2024-03-26T06:09:59.402 INFO:teuthology.orchestra.run.smithi149.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-03-26T06:09:59.435 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-03-26T06:09:59.435 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-03-26T06:09:59.442 INFO:teuthology.orchestra.run.smithi037.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-03-26T06:09:59.453 INFO:teuthology.orchestra.run.smithi149.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 629 kB/s | 32 kB 00:00 2024-03-26T06:09:59.601 INFO:teuthology.orchestra.run.smithi037.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 292 kB/s | 46 kB 00:00 2024-03-26T06:09:59.620 INFO:teuthology.orchestra.run.smithi149.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-03-26T06:09:59.686 INFO:teuthology.orchestra.run.smithi149.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 258 kB/s | 17 kB 00:00 2024-03-26T06:09:59.745 INFO:teuthology.orchestra.run.smithi149.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 172 kB/s | 10 kB 00:00 2024-03-26T06:09:59.768 INFO:teuthology.orchestra.run.smithi037.stdout:(132/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.1 MB/s | 2.0 MB 00:00 2024-03-26T06:09:59.795 INFO:teuthology.orchestra.run.smithi149.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-26T06:09:59.802 INFO:teuthology.orchestra.run.smithi037.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 2.5 MB/s | 1.0 MB 00:00 2024-03-26T06:09:59.827 INFO:teuthology.orchestra.run.smithi037.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 349 kB/s | 79 kB 00:00 2024-03-26T06:09:59.888 INFO:teuthology.orchestra.run.smithi149.stdout:(47/174): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-03-26T06:09:59.913 INFO:teuthology.orchestra.run.smithi149.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 377 kB/s | 44 kB 00:00 2024-03-26T06:09:59.935 INFO:teuthology.orchestra.run.smithi037.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 345 kB/s | 58 kB 00:00 2024-03-26T06:10:00.010 INFO:teuthology.orchestra.run.smithi037.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 89 kB/s | 16 kB 00:00 2024-03-26T06:10:00.038 INFO:teuthology.orchestra.run.smithi149.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 661 kB 00:00 2024-03-26T06:10:00.063 INFO:teuthology.orchestra.run.smithi149.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 578 kB 00:00 2024-03-26T06:10:00.069 INFO:teuthology.orchestra.run.smithi037.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 673 kB/s | 90 kB 00:00 2024-03-26T06:10:00.097 INFO:teuthology.orchestra.run.smithi149.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 797 kB/s | 46 kB 00:00 2024-03-26T06:10:00.122 INFO:teuthology.orchestra.run.smithi149.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-03-26T06:10:00.136 INFO:teuthology.orchestra.run.smithi037.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 225 kB/s | 28 kB 00:00 2024-03-26T06:10:00.188 INFO:teuthology.orchestra.run.smithi149.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-03-26T06:10:00.195 INFO:teuthology.orchestra.run.smithi037.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2024-03-26T06:10:00.215 INFO:teuthology.orchestra.run.smithi149.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 217 kB 00:00 2024-03-26T06:10:00.248 INFO:teuthology.orchestra.run.smithi149.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 572 kB/s | 34 kB 00:00 2024-03-26T06:10:00.362 INFO:teuthology.orchestra.run.smithi037.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 354 kB/s | 59 kB 00:00 2024-03-26T06:10:00.387 INFO:teuthology.orchestra.run.smithi037.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 752 kB/s | 188 kB 00:00 2024-03-26T06:10:00.457 INFO:teuthology.orchestra.run.smithi149.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-03-26T06:10:00.462 INFO:teuthology.orchestra.run.smithi037.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 357 kB/s | 36 kB 00:00 2024-03-26T06:10:00.491 INFO:teuthology.orchestra.run.smithi149.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2024-03-26T06:10:00.504 INFO:teuthology.orchestra.run.smithi037.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 436 kB/s | 51 kB 00:00 2024-03-26T06:10:00.516 INFO:teuthology.orchestra.run.smithi149.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-26T06:10:00.663 INFO:teuthology.orchestra.run.smithi037.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 566 kB/s | 90 kB 00:00 2024-03-26T06:10:00.763 INFO:teuthology.orchestra.run.smithi037.stdout:(145/174): python3-pecan-1.4.2-3.el9.noarch.rpm 291 kB/s | 272 kB 00:00 2024-03-26T06:10:00.930 INFO:teuthology.orchestra.run.smithi037.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 120 kB/s | 20 kB 00:00 2024-03-26T06:10:01.041 INFO:teuthology.orchestra.run.smithi149.stdout:(59/174): ceph-test-19.0.0-1964.g7e98f5f6.el9.x 6.1 MB/s | 47 MB 00:07 2024-03-26T06:10:01.080 INFO:teuthology.orchestra.run.smithi037.stdout:(147/174): python3-webob-1.8.7-6.el9.noarch.rpm 372 kB/s | 230 kB 00:00 2024-03-26T06:10:01.105 INFO:teuthology.orchestra.run.smithi037.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-03-26T06:10:01.131 INFO:teuthology.orchestra.run.smithi037.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 913 kB/s | 427 kB 00:00 2024-03-26T06:10:01.166 INFO:teuthology.orchestra.run.smithi149.stdout:(60/174): python3-babel-2.9.1-2.el9.noarch.rpm 8.6 MB/s | 5.8 MB 00:00 2024-03-26T06:10:01.183 INFO:teuthology.orchestra.run.smithi149.stdout:(61/174): python3-jsonpatch-1.21-16.el9.noarch. 37 kB/s | 24 kB 00:00 2024-03-26T06:10:01.200 INFO:teuthology.orchestra.run.smithi149.stdout:(62/174): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-03-26T06:10:01.225 INFO:teuthology.orchestra.run.smithi149.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 757 kB/s | 32 kB 00:00 2024-03-26T06:10:01.251 INFO:teuthology.orchestra.run.smithi149.stdout:(64/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-03-26T06:10:01.441 INFO:teuthology.orchestra.run.smithi037.stdout:(150/174): librbd1-19.0.0-1964.g7e98f5f6.el9.x8 10 MB/s | 3.1 MB 00:00 2024-03-26T06:10:01.485 INFO:teuthology.orchestra.run.smithi149.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-03-26T06:10:01.506 INFO:teuthology.orchestra.run.smithi037.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.7 MB/s | 321 kB 00:00 2024-03-26T06:10:01.509 INFO:teuthology.orchestra.run.smithi149.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 767 kB/s | 217 kB 00:00 2024-03-26T06:10:01.535 INFO:teuthology.orchestra.run.smithi149.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 618 kB/s | 174 kB 00:00 2024-03-26T06:10:01.602 INFO:teuthology.orchestra.run.smithi149.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-26T06:10:01.627 INFO:teuthology.orchestra.run.smithi149.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 450 kB/s | 41 kB 00:00 2024-03-26T06:10:01.652 INFO:teuthology.orchestra.run.smithi149.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 238 kB 00:00 2024-03-26T06:10:01.678 INFO:teuthology.orchestra.run.smithi149.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-26T06:10:01.720 INFO:teuthology.orchestra.run.smithi149.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-26T06:10:01.745 INFO:teuthology.orchestra.run.smithi149.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 514 kB/s | 47 kB 00:00 2024-03-26T06:10:01.765 INFO:teuthology.orchestra.run.smithi037.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 8.9 MB/s | 2.3 MB 00:00 2024-03-26T06:10:01.779 INFO:teuthology.orchestra.run.smithi149.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 423 kB/s | 43 kB 00:00 2024-03-26T06:10:01.974 INFO:teuthology.orchestra.run.smithi037.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 8.8 MB/s | 1.8 MB 00:00 2024-03-26T06:10:02.032 INFO:teuthology.orchestra.run.smithi037.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 723 kB/s | 42 kB 00:00 2024-03-26T06:10:02.038 INFO:teuthology.orchestra.run.smithi149.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 150 kB/s | 39 kB 00:00 2024-03-26T06:10:02.063 INFO:teuthology.orchestra.run.smithi149.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 145 kB/s | 41 kB 00:00 2024-03-26T06:10:02.116 INFO:teuthology.orchestra.run.smithi037.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.9 MB/s | 248 kB 00:00 2024-03-26T06:10:02.174 INFO:teuthology.orchestra.run.smithi037.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-26T06:10:02.217 INFO:teuthology.orchestra.run.smithi037.stdout:(157/174): thrift-0.15.0-2.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-03-26T06:10:02.258 INFO:teuthology.orchestra.run.smithi037.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 436 kB/s | 36 kB 00:00 2024-03-26T06:10:02.300 INFO:teuthology.orchestra.run.smithi037.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.3 MB/s | 192 kB 00:00 2024-03-26T06:10:02.351 INFO:teuthology.orchestra.run.smithi037.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 462 kB/s | 23 kB 00:00 2024-03-26T06:10:02.372 INFO:teuthology.orchestra.run.smithi149.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-03-26T06:10:02.376 INFO:teuthology.orchestra.run.smithi037.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-03-26T06:10:02.405 INFO:teuthology.orchestra.run.smithi149.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 817 kB/s | 300 kB 00:00 2024-03-26T06:10:02.430 INFO:teuthology.orchestra.run.smithi149.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 155 kB/s | 56 kB 00:00 2024-03-26T06:10:02.443 INFO:teuthology.orchestra.run.smithi037.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 108 kB 00:00 2024-03-26T06:10:02.447 INFO:teuthology.orchestra.run.smithi149.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 841 kB/s | 63 kB 00:00 2024-03-26T06:10:02.472 INFO:teuthology.orchestra.run.smithi149.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.9 MB/s | 79 kB 00:00 2024-03-26T06:10:02.477 INFO:teuthology.orchestra.run.smithi037.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-26T06:10:02.489 INFO:teuthology.orchestra.run.smithi149.stdout:(82/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 753 kB/s | 44 kB 00:00 2024-03-26T06:10:02.514 INFO:teuthology.orchestra.run.smithi149.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-26T06:10:02.544 INFO:teuthology.orchestra.run.smithi037.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-03-26T06:10:02.565 INFO:teuthology.orchestra.run.smithi149.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-26T06:10:02.569 INFO:teuthology.orchestra.run.smithi037.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 695 kB/s | 64 kB 00:00 2024-03-26T06:10:02.590 INFO:teuthology.orchestra.run.smithi149.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-26T06:10:02.611 INFO:teuthology.orchestra.run.smithi037.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 429 kB/s | 29 kB 00:00 2024-03-26T06:10:02.615 INFO:teuthology.orchestra.run.smithi149.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-26T06:10:02.641 INFO:teuthology.orchestra.run.smithi149.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-26T06:10:02.741 INFO:teuthology.orchestra.run.smithi149.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-26T06:10:02.775 INFO:teuthology.orchestra.run.smithi149.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-03-26T06:10:02.814 INFO:teuthology.orchestra.run.smithi037.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 4.9 MB/s | 1.2 MB 00:00 2024-03-26T06:10:02.833 INFO:teuthology.orchestra.run.smithi149.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-03-26T06:10:02.877 INFO:teuthology.orchestra.run.smithi149.stdout:(91/174): python3-requests-2.25.1-8.el9.noarch. 934 kB/s | 125 kB 00:00 2024-03-26T06:10:02.931 INFO:teuthology.orchestra.run.smithi037.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.7 MB/s | 2.2 MB 00:00 2024-03-26T06:10:03.000 INFO:teuthology.orchestra.run.smithi149.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 95 kB/s | 21 kB 00:00 2024-03-26T06:10:03.065 INFO:teuthology.orchestra.run.smithi037.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.4 MB/s | 465 kB 00:00 2024-03-26T06:10:03.084 INFO:teuthology.orchestra.run.smithi149.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 102 kB/s | 25 kB 00:00 2024-03-26T06:10:03.157 INFO:teuthology.orchestra.run.smithi037.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 339 kB/s | 31 kB 00:00 2024-03-26T06:10:03.200 INFO:teuthology.orchestra.run.smithi037.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.9 MB/s | 2.3 MB 00:00 2024-03-26T06:10:03.358 INFO:teuthology.orchestra.run.smithi037.stdout:(172/174): librados2-19.0.0-1964.g7e98f5f6.el9. 1.5 MB/s | 3.4 MB 00:02 2024-03-26T06:10:03.425 INFO:teuthology.orchestra.run.smithi037.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.0 MB/s | 1.1 MB 00:00 2024-03-26T06:10:03.508 INFO:teuthology.orchestra.run.smithi037.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 241 kB/s | 74 kB 00:00 2024-03-26T06:10:03.513 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-26T06:10:03.513 INFO:teuthology.orchestra.run.smithi037.stdout:Total 11 MB/s | 223 MB 00:19 2024-03-26T06:10:04.116 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-26T06:10:04.410 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-26T06:10:04.410 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-26T06:10:04.501 INFO:teuthology.orchestra.run.smithi149.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 643 kB/s | 911 kB 00:01 2024-03-26T06:10:04.568 INFO:teuthology.orchestra.run.smithi149.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-03-26T06:10:05.251 INFO:teuthology.orchestra.run.smithi149.stdout:(96/174): postgresql-server-13.14-1.el9.x86_64. 8.5 MB/s | 5.8 MB 00:00 2024-03-26T06:10:06.126 INFO:teuthology.orchestra.run.smithi149.stdout:(97/174): postgresql-13.14-1.el9.x86_64.rpm 520 kB/s | 1.6 MB 00:03 2024-03-26T06:10:06.580 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-26T06:10:06.580 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-26T06:10:06.860 INFO:teuthology.orchestra.run.smithi149.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 153 kB/s | 245 kB 00:01 2024-03-26T06:10:06.900 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-03-26T06:10:07.027 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T06:10:07.053 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T06:10:07.084 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T06:10:07.093 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T06:10:07.134 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T06:10:07.188 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T06:10:07.221 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T06:10:07.238 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T06:10:07.281 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T06:10:07.624 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T06:10:07.671 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 10/198 2024-03-26T06:10:07.779 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 10/198 2024-03-26T06:10:07.806 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 11/198 2024-03-26T06:10:07.881 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 11/198 2024-03-26T06:10:07.910 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:19.0.0-1964.g7e98f5f6.el9.x86_64 12/198 2024-03-26T06:10:07.946 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T06:10:08.146 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T06:10:08.169 INFO:teuthology.orchestra.run.smithi149.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 121 kB/s | 248 kB 00:02 2024-03-26T06:10:08.200 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T06:10:08.236 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 16/198 2024-03-26T06:10:08.260 INFO:teuthology.orchestra.run.smithi149.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 392 kB/s | 36 kB 00:00 2024-03-26T06:10:08.286 INFO:teuthology.orchestra.run.smithi149.stdout:(101/174): qatlib-24.02.0-1.el9.x86_64.rpm 155 kB/s | 221 kB 00:01 2024-03-26T06:10:08.341 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 16/198 2024-03-26T06:10:08.344 INFO:teuthology.orchestra.run.smithi149.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-03-26T06:10:08.370 INFO:teuthology.orchestra.run.smithi149.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-03-26T06:10:08.370 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86 17/198 2024-03-26T06:10:08.434 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86 17/198 2024-03-26T06:10:08.445 INFO:teuthology.orchestra.run.smithi149.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-03-26T06:10:08.474 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T06:10:08.496 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T06:10:08.513 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T06:10:08.562 INFO:teuthology.orchestra.run.smithi149.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-03-26T06:10:08.604 INFO:teuthology.orchestra.run.smithi149.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-03-26T06:10:08.629 INFO:teuthology.orchestra.run.smithi149.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-03-26T06:10:08.671 INFO:teuthology.orchestra.run.smithi149.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-03-26T06:10:08.681 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T06:10:08.705 INFO:teuthology.orchestra.run.smithi149.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 780 kB/s | 26 kB 00:00 2024-03-26T06:10:08.711 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T06:10:08.738 INFO:teuthology.orchestra.run.smithi149.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-03-26T06:10:08.742 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T06:10:08.814 INFO:teuthology.orchestra.run.smithi149.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-03-26T06:10:08.839 INFO:teuthology.orchestra.run.smithi149.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-03-26T06:10:08.850 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T06:10:08.898 INFO:teuthology.orchestra.run.smithi149.stdout:(113/174): openssl-devel-3.0.7-27.el9.x86_64.rp 703 kB/s | 4.1 MB 00:06 2024-03-26T06:10:08.923 INFO:teuthology.orchestra.run.smithi149.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-03-26T06:10:08.945 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T06:10:08.981 INFO:teuthology.orchestra.run.smithi149.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 838 kB 00:00 2024-03-26T06:10:08.998 INFO:teuthology.orchestra.run.smithi149.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 579 kB/s | 43 kB 00:00 2024-03-26T06:10:09.005 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T06:10:09.015 INFO:teuthology.orchestra.run.smithi149.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 961 kB/s | 32 kB 00:00 2024-03-26T06:10:09.032 INFO:teuthology.orchestra.run.smithi149.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 446 kB/s | 15 kB 00:00 2024-03-26T06:10:09.057 INFO:teuthology.orchestra.run.smithi149.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-03-26T06:10:09.066 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T06:10:09.091 INFO:teuthology.orchestra.run.smithi149.stdout:(120/174): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-03-26T06:10:09.125 INFO:teuthology.orchestra.run.smithi149.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-03-26T06:10:09.148 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T06:10:09.150 INFO:teuthology.orchestra.run.smithi149.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 605 kB/s | 35 kB 00:00 2024-03-26T06:10:09.184 INFO:teuthology.orchestra.run.smithi149.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-03-26T06:10:09.209 INFO:teuthology.orchestra.run.smithi149.stdout:(124/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-03-26T06:10:09.234 INFO:teuthology.orchestra.run.smithi149.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-03-26T06:10:09.259 INFO:teuthology.orchestra.run.smithi149.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-03-26T06:10:09.285 INFO:teuthology.orchestra.run.smithi149.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-03-26T06:10:09.310 INFO:teuthology.orchestra.run.smithi149.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-03-26T06:10:09.335 INFO:teuthology.orchestra.run.smithi149.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 384 kB/s | 19 kB 00:00 2024-03-26T06:10:09.359 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T06:10:09.383 INFO:teuthology.orchestra.run.smithi149.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 270 kB/s | 20 kB 00:00 2024-03-26T06:10:09.436 INFO:teuthology.orchestra.run.smithi149.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.1 MB/s | 2.0 MB 00:00 2024-03-26T06:10:09.461 INFO:teuthology.orchestra.run.smithi149.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 597 kB/s | 46 kB 00:00 2024-03-26T06:10:09.472 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T06:10:09.495 INFO:teuthology.orchestra.run.smithi149.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-03-26T06:10:09.520 INFO:teuthology.orchestra.run.smithi149.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 940 kB/s | 79 kB 00:00 2024-03-26T06:10:09.545 INFO:teuthology.orchestra.run.smithi149.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 686 kB/s | 58 kB 00:00 2024-03-26T06:10:09.571 INFO:teuthology.orchestra.run.smithi149.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-03-26T06:10:09.576 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T06:10:09.596 INFO:teuthology.orchestra.run.smithi149.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-03-26T06:10:09.611 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T06:10:09.621 INFO:teuthology.orchestra.run.smithi149.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-03-26T06:10:09.637 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T06:10:09.647 INFO:teuthology.orchestra.run.smithi149.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-03-26T06:10:09.665 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T06:10:09.672 INFO:teuthology.orchestra.run.smithi149.stdout:(140/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-03-26T06:10:09.697 INFO:teuthology.orchestra.run.smithi149.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-26T06:10:09.723 INFO:teuthology.orchestra.run.smithi149.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-03-26T06:10:09.734 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T06:10:09.748 INFO:teuthology.orchestra.run.smithi149.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-03-26T06:10:09.773 INFO:teuthology.orchestra.run.smithi149.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-03-26T06:10:09.786 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T06:10:09.798 INFO:teuthology.orchestra.run.smithi149.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-03-26T06:10:09.823 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T06:10:09.823 INFO:teuthology.orchestra.run.smithi149.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-26T06:10:09.849 INFO:teuthology.orchestra.run.smithi149.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-03-26T06:10:09.874 INFO:teuthology.orchestra.run.smithi149.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-03-26T06:10:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T06:10:09.908 INFO:teuthology.orchestra.run.smithi149.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-03-26T06:10:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T06:10:09.974 INFO:teuthology.orchestra.run.smithi149.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-03-26T06:10:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T06:10:10.014 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T06:10:10.016 INFO:teuthology.orchestra.run.smithi149.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-03-26T06:10:10.042 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T06:10:10.115 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T06:10:10.155 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:19.0.0-1964.g7e98f5f6.el9. 39/198 2024-03-26T06:10:10.207 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:19.0.0-1964.g7e98f5f6.el 40/198 2024-03-26T06:10:10.237 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:19.0.0-1964.g7e98f5f6.el9.x86_6 41/198 2024-03-26T06:10:10.310 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T06:10:10.359 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T06:10:10.436 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T06:10:10.483 INFO:teuthology.orchestra.run.smithi149.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 4.9 MB/s | 2.3 MB 00:00 2024-03-26T06:10:10.534 INFO:teuthology.orchestra.run.smithi149.stdout:(153/174): librados2-19.0.0-1964.g7e98f5f6.el9. 5.1 MB/s | 3.4 MB 00:00 2024-03-26T06:10:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T06:10:10.609 INFO:teuthology.orchestra.run.smithi149.stdout:(154/174): librbd1-19.0.0-1964.g7e98f5f6.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-03-26T06:10:10.629 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T06:10:10.726 INFO:teuthology.orchestra.run.smithi149.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 219 kB/s | 42 kB 00:00 2024-03-26T06:10:10.843 INFO:teuthology.orchestra.run.smithi149.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.0 MB/s | 248 kB 00:00 2024-03-26T06:10:10.868 INFO:teuthology.orchestra.run.smithi149.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 503 kB/s | 71 kB 00:00 2024-03-26T06:10:10.952 INFO:teuthology.orchestra.run.smithi149.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 336 kB/s | 36 kB 00:00 2024-03-26T06:10:11.060 INFO:teuthology.orchestra.run.smithi149.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.0 MB/s | 192 kB 00:00 2024-03-26T06:10:11.119 INFO:teuthology.orchestra.run.smithi149.stdout:(160/174): NetworkManager-libnm-1.47.2-1.el9.x8 2.9 MB/s | 1.8 MB 00:00 2024-03-26T06:10:11.169 INFO:teuthology.orchestra.run.smithi149.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.1 MB/s | 256 kB 00:00 2024-03-26T06:10:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T06:10:11.202 INFO:teuthology.orchestra.run.smithi149.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 162 kB/s | 23 kB 00:00 2024-03-26T06:10:11.267 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T06:10:11.322 INFO:teuthology.orchestra.run.smithi149.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 535 kB/s | 108 kB 00:00 2024-03-26T06:10:11.347 INFO:teuthology.orchestra.run.smithi149.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 872 kB/s | 155 kB 00:00 2024-03-26T06:10:11.372 INFO:teuthology.orchestra.run.smithi149.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 800 kB/s | 136 kB 00:00 2024-03-26T06:10:11.381 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T06:10:11.417 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T06:10:11.427 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T06:10:11.448 INFO:teuthology.orchestra.run.smithi149.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 509 kB/s | 64 kB 00:00 2024-03-26T06:10:11.449 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T06:10:11.473 INFO:teuthology.orchestra.run.smithi149.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 228 kB/s | 29 kB 00:00 2024-03-26T06:10:11.490 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T06:10:11.528 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_6 53/198 2024-03-26T06:10:11.557 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T06:10:11.789 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T06:10:11.943 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T06:10:11.957 INFO:teuthology.orchestra.run.smithi149.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 2.1 MB/s | 1.2 MB 00:00 2024-03-26T06:10:11.992 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T06:10:12.037 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T06:10:12.183 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T06:10:12.245 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T06:10:12.296 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T06:10:12.365 INFO:teuthology.orchestra.run.smithi149.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 2.3 MB 00:00 2024-03-26T06:10:12.371 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T06:10:12.409 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T06:10:12.432 INFO:teuthology.orchestra.run.smithi149.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.2 MB/s | 2.2 MB 00:00 2024-03-26T06:10:12.446 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T06:10:12.499 INFO:teuthology.orchestra.run.smithi149.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 233 kB/s | 31 kB 00:00 2024-03-26T06:10:12.500 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T06:10:12.524 INFO:teuthology.orchestra.run.smithi149.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 820 kB/s | 465 kB 00:00 2024-03-26T06:10:12.535 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T06:10:12.578 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T06:10:12.612 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T06:10:12.754 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T06:10:12.800 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T06:10:12.831 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T06:10:12.872 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T06:10:12.908 INFO:teuthology.orchestra.run.smithi149.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 2.3 MB/s | 1.1 MB 00:00 2024-03-26T06:10:12.970 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T06:10:13.005 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T06:10:13.045 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T06:10:13.070 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T06:10:13.095 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T06:10:13.171 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T06:10:13.302 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T06:10:13.410 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T06:10:13.453 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T06:10:13.503 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T06:10:13.567 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T06:10:13.758 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T06:10:13.807 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T06:10:13.960 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T06:10:14.005 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T06:10:14.037 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T06:10:14.068 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T06:10:14.089 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-26T06:10:14.096 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T06:10:14.130 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T06:10:14.170 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T06:10:14.201 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T06:10:14.205 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T06:10:14.231 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T06:10:14.256 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T06:10:14.302 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T06:10:14.310 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T06:10:14.313 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T06:10:14.341 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T06:10:14.363 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T06:10:14.366 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T06:10:14.408 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T06:10:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T06:10:14.442 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T06:10:14.457 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T06:10:14.482 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T06:10:14.538 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T06:10:14.544 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T06:10:14.642 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T06:10:14.690 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T06:10:14.720 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T06:10:14.760 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T06:10:14.872 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T06:10:14.885 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T06:10:14.923 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T06:10:14.974 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 10/198 2024-03-26T06:10:15.091 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 10/198 2024-03-26T06:10:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 11/198 2024-03-26T06:10:15.318 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 11/198 2024-03-26T06:10:15.347 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.0.0-1964.g7e98f5f6.el9.x86_64 12/198 2024-03-26T06:10:15.384 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T06:10:15.578 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T06:10:15.637 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T06:10:15.665 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 16/198 2024-03-26T06:10:15.700 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T06:10:15.744 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T06:10:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 16/198 2024-03-26T06:10:15.778 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T06:10:15.799 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86 17/198 2024-03-26T06:10:15.827 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T06:10:15.872 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86 17/198 2024-03-26T06:10:15.910 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T06:10:15.919 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T06:10:15.942 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T06:10:15.945 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T06:10:15.958 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T06:10:15.991 INFO:teuthology.orchestra.run.smithi149.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 21 kB/s | 74 kB 00:03 2024-03-26T06:10:15.995 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-26T06:10:15.996 INFO:teuthology.orchestra.run.smithi149.stdout:Total 8.5 MB/s | 223 MB 00:26 2024-03-26T06:10:16.101 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T06:10:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T06:10:16.142 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T06:10:16.159 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T06:10:16.172 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T06:10:16.248 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T06:10:16.377 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T06:10:16.541 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-26T06:10:16.542 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T06:10:16.602 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T06:10:16.693 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T06:10:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T06:10:16.814 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-26T06:10:16.814 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-26T06:10:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T06:10:16.975 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T06:10:17.044 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T06:10:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T06:10:17.182 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T06:10:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T06:10:17.244 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T06:10:17.279 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T06:10:17.382 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T06:10:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T06:10:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T06:10:17.562 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T06:10:17.595 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T06:10:17.638 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T06:10:17.672 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T06:10:17.698 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T06:10:17.770 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T06:10:17.803 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.0.0-1964.g7e98f5f6.el9. 39/198 2024-03-26T06:10:17.847 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.0.0-1964.g7e98f5f6.el 40/198 2024-03-26T06:10:17.875 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.0.0-1964.g7e98f5f6.el9.x86_6 41/198 2024-03-26T06:10:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T06:10:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T06:10:18.092 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T06:10:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T06:10:18.286 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T06:10:18.336 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T06:10:18.409 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T06:10:18.440 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T06:10:18.546 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-26T06:10:18.547 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-26T06:10:18.560 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T06:10:18.716 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T06:10:18.841 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T06:10:18.924 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T06:10:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T06:10:19.082 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T06:10:19.093 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T06:10:19.124 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T06:10:19.138 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T06:10:19.170 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 124/198 2024-03-26T06:10:19.172 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T06:10:19.210 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_6 53/198 2024-03-26T06:10:19.240 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T06:10:19.261 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 124/198 2024-03-26T06:10:19.292 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 125/198 2024-03-26T06:10:19.328 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T06:10:19.530 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T06:10:19.807 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T06:10:19.837 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T06:10:19.839 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 128/198 2024-03-26T06:10:19.894 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 128/198 2024-03-26T06:10:19.924 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 129/198 2024-03-26T06:10:19.942 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T06:10:20.104 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T06:10:20.438 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T06:10:20.497 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T06:10:20.554 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T06:10:20.628 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T06:10:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T06:10:20.718 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T06:10:20.766 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T06:10:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T06:10:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T06:10:20.912 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T06:10:21.052 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T06:10:21.100 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T06:10:21.138 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T06:10:21.180 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T06:10:21.278 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T06:10:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T06:10:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T06:10:21.395 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T06:10:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T06:10:21.517 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T06:10:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T06:10:21.766 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T06:10:21.811 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T06:10:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T06:10:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T06:10:22.135 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T06:10:22.151 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 130/198 2024-03-26T06:10:22.191 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T06:10:22.196 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 130/198 2024-03-26T06:10:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T06:10:22.380 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T06:10:22.413 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T06:10:22.442 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T06:10:22.471 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T06:10:22.505 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T06:10:22.553 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T06:10:22.593 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T06:10:22.623 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T06:10:22.702 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T06:10:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T06:10:22.766 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 130/198 2024-03-26T06:10:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T06:10:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 131/198 2024-03-26T06:10:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T06:10:22.858 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T06:10:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 131/198 2024-03-26T06:10:22.880 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T06:10:22.881 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T06:10:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:22.900 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 132/198 2024-03-26T06:10:22.950 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T06:10:23.048 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T06:10:23.089 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T06:10:23.120 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T06:10:23.168 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T06:10:23.268 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T06:10:23.322 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T06:10:24.100 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T06:10:24.152 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T06:10:24.186 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T06:10:24.235 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T06:10:24.326 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T06:10:24.392 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T06:10:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T06:10:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T06:10:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T06:10:25.361 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T06:10:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T06:10:25.668 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-26T06:10:25.787 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T06:10:25.829 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T06:10:25.851 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T06:10:25.860 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T06:10:25.901 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T06:10:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T06:10:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T06:10:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T06:10:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T06:10:26.393 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T06:10:26.449 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 10/198 2024-03-26T06:10:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 10/198 2024-03-26T06:10:26.591 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 11/198 2024-03-26T06:10:26.683 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 11/198 2024-03-26T06:10:26.712 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.0.0-1964.g7e98f5f6.el9.x86_64 12/198 2024-03-26T06:10:26.753 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T06:10:26.792 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T06:10:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T06:10:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T06:10:26.937 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T06:10:26.955 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T06:10:26.982 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T06:10:26.993 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T06:10:27.030 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 16/198 2024-03-26T06:10:27.141 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 16/198 2024-03-26T06:10:27.173 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86 17/198 2024-03-26T06:10:27.219 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86 17/198 2024-03-26T06:10:27.260 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T06:10:27.291 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T06:10:27.306 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T06:10:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T06:10:27.345 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 124/198 2024-03-26T06:10:27.483 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T06:10:27.522 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T06:10:27.554 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T06:10:27.554 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 124/198 2024-03-26T06:10:27.583 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 125/198 2024-03-26T06:10:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T06:10:27.660 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T06:10:27.765 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T06:10:27.833 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T06:10:27.911 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T06:10:27.976 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T06:10:28.098 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T06:10:28.123 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 128/198 2024-03-26T06:10:28.186 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 128/198 2024-03-26T06:10:28.217 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 129/198 2024-03-26T06:10:28.234 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T06:10:28.355 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T06:10:28.454 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T06:10:28.490 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T06:10:28.517 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T06:10:28.561 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T06:10:28.635 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T06:10:28.689 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T06:10:28.727 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T06:10:28.792 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T06:10:28.834 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T06:10:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T06:10:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T06:10:28.939 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T06:10:29.009 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T06:10:29.044 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.0.0-1964.g7e98f5f6.el9. 39/198 2024-03-26T06:10:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.0.0-1964.g7e98f5f6.el 40/198 2024-03-26T06:10:29.116 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.0.0-1964.g7e98f5f6.el9.x86_6 41/198 2024-03-26T06:10:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T06:10:29.230 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T06:10:29.312 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T06:10:29.455 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T06:10:29.508 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T06:10:30.029 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T06:10:30.112 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T06:10:30.197 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T06:10:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T06:10:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T06:10:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T06:10:30.338 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T06:10:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_6 53/198 2024-03-26T06:10:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T06:10:30.548 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 130/198 2024-03-26T06:10:30.590 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 130/198 2024-03-26T06:10:30.636 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T06:10:30.813 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T06:10:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T06:10:31.122 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T06:10:31.160 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 130/198 2024-03-26T06:10:31.201 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 131/198 2024-03-26T06:10:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 131/198 2024-03-26T06:10:31.303 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T06:10:31.303 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T06:10:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:10:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 132/198 2024-03-26T06:10:31.357 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T06:10:31.475 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T06:10:31.538 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T06:10:31.611 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T06:10:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T06:10:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T06:10:31.725 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T06:10:31.762 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T06:10:31.805 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T06:10:31.855 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T06:10:31.999 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T06:10:32.044 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T06:10:32.074 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T06:10:32.116 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T06:10:32.210 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T06:10:32.250 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T06:10:32.290 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T06:10:32.315 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T06:10:32.340 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T06:10:32.420 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T06:10:32.549 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T06:10:32.657 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T06:10:32.698 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T06:10:32.747 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T06:10:32.802 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T06:10:32.994 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T06:10:33.045 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T06:10:33.172 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T06:10:33.209 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T06:10:33.241 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T06:10:33.281 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T06:10:33.317 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T06:10:33.352 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T06:10:33.390 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T06:10:33.422 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T06:10:33.452 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T06:10:33.528 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T06:10:33.563 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T06:10:33.596 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T06:10:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T06:10:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T06:10:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T06:10:33.859 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T06:10:33.911 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T06:10:33.943 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T06:10:33.990 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T06:10:34.088 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T06:10:34.144 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T06:10:34.833 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 132/198 2024-03-26T06:10:34.833 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2024-03-26T06:10:34.833 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2024-03-26T06:10:34.833 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2024-03-26T06:10:34.833 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2024-03-26T06:10:34.833 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2024-03-26T06:10:34.833 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /root 2024-03-26T06:10:34.833 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2024-03-26T06:10:34.834 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:34.880 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T06:10:34.925 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T06:10:34.959 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T06:10:35.002 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 133/198 2024-03-26T06:10:35.007 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T06:10:35.018 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 133/198 2024-03-26T06:10:35.018 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:35.018 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T06:10:35.018 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:35.094 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T06:10:35.135 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T06:10:35.321 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T06:10:35.366 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T06:10:35.444 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T06:10:36.090 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T06:10:36.205 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T06:10:36.563 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 134/198 2024-03-26T06:10:36.590 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 134/198 2024-03-26T06:10:36.591 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:36.591 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T06:10:36.591 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T06:10:36.591 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T06:10:36.591 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:36.814 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 135/198 2024-03-26T06:10:36.838 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 135/198 2024-03-26T06:10:36.838 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:36.838 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T06:10:36.838 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T06:10:36.839 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T06:10:36.839 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:37.325 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 136/198 2024-03-26T06:10:37.349 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 136/198 2024-03-26T06:10:37.349 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:37.349 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T06:10:37.349 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T06:10:37.349 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T06:10:37.349 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:37.409 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T06:10:37.443 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T06:10:37.492 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T06:10:37.532 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T06:10:37.563 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T06:10:37.590 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T06:10:37.903 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T06:10:37.904 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 138/198 2024-03-26T06:10:37.929 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 124/198 2024-03-26T06:10:37.942 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 138/198 2024-03-26T06:10:37.966 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1964.g7e98f5f6.e 139/198 2024-03-26T06:10:38.018 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1964.g7e98f5f6. 140/198 2024-03-26T06:10:38.019 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 124/198 2024-03-26T06:10:38.039 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noa 141/198 2024-03-26T06:10:38.050 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 125/198 2024-03-26T06:10:38.086 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T06:10:38.284 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noa 141/198 2024-03-26T06:10:38.315 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.n 142/198 2024-03-26T06:10:38.542 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T06:10:38.574 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 128/198 2024-03-26T06:10:38.627 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 128/198 2024-03-26T06:10:38.651 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 129/198 2024-03-26T06:10:39.329 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.n 142/198 2024-03-26T06:10:39.357 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e9 143/198 2024-03-26T06:10:39.464 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e9 143/198 2024-03-26T06:10:39.627 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1964.g7e98f5f6.el 144/198 2024-03-26T06:10:39.651 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 145/198 2024-03-26T06:10:39.675 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 145/198 2024-03-26T06:10:39.675 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:39.675 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T06:10:39.675 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T06:10:39.676 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T06:10:39.676 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:39.726 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 146/198 2024-03-26T06:10:39.737 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 146/198 2024-03-26T06:10:39.775 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:19.0.0-1964.g7e98f5f6.el9.x86_64 147/198 2024-03-26T06:10:40.683 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T06:10:40.712 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 149/198 2024-03-26T06:10:40.736 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 149/198 2024-03-26T06:10:40.736 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:40.736 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T06:10:40.736 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T06:10:40.736 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T06:10:40.736 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:40.779 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1964.g7e98f 150/198 2024-03-26T06:10:40.785 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 130/198 2024-03-26T06:10:40.802 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1964.g7e98f 150/198 2024-03-26T06:10:40.802 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:40.802 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T06:10:40.802 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:40.813 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 130/198 2024-03-26T06:10:40.846 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 151/198 2024-03-26T06:10:40.870 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 151/198 2024-03-26T06:10:40.870 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:40.870 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T06:10:40.870 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T06:10:40.870 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T06:10:40.870 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:41.197 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 152/198 2024-03-26T06:10:41.220 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 152/198 2024-03-26T06:10:41.221 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:41.221 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T06:10:41.221 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T06:10:41.221 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T06:10:41.221 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:41.351 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 130/198 2024-03-26T06:10:41.384 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 131/198 2024-03-26T06:10:41.465 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 131/198 2024-03-26T06:10:41.465 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T06:10:41.466 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T06:10:41.466 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:10:41.494 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 132/198 2024-03-26T06:10:43.500 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 132/198 2024-03-26T06:10:43.500 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-03-26T06:10:43.500 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-03-26T06:10:43.500 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-03-26T06:10:43.501 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-03-26T06:10:43.501 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-03-26T06:10:43.501 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-03-26T06:10:43.501 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-03-26T06:10:43.501 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:10:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 133/198 2024-03-26T06:10:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 133/198 2024-03-26T06:10:43.678 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:43.678 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T06:10:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:10:45.223 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 134/198 2024-03-26T06:10:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 134/198 2024-03-26T06:10:45.250 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:45.250 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T06:10:45.250 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T06:10:45.250 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T06:10:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:10:45.476 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 135/198 2024-03-26T06:10:45.499 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 135/198 2024-03-26T06:10:45.499 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:45.499 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T06:10:45.499 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T06:10:45.500 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T06:10:45.500 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:10:45.893 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:19.0.0-1964.g7e98f5f6.el9.x86_64 153/198 2024-03-26T06:10:45.935 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 154/198 2024-03-26T06:10:45.959 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 136/198 2024-03-26T06:10:45.974 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 155/198 2024-03-26T06:10:45.981 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 136/198 2024-03-26T06:10:45.981 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:45.982 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T06:10:45.982 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T06:10:45.982 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T06:10:45.982 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:10:46.000 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T06:10:46.086 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T06:10:46.193 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-shell-2:19.0.0-1964.g7e98f5f6.el9.x86_64 157/198 2024-03-26T06:10:46.312 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T06:10:46.357 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 159/198 2024-03-26T06:10:46.396 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T06:10:46.448 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T06:10:46.491 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T06:10:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 138/198 2024-03-26T06:10:46.524 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_ 163/198 2024-03-26T06:10:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 138/198 2024-03-26T06:10:46.565 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T06:10:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1964.g7e98f5f6.e 139/198 2024-03-26T06:10:46.594 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T06:10:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1964.g7e98f5f6. 140/198 2024-03-26T06:10:46.648 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T06:10:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noa 141/198 2024-03-26T06:10:46.700 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T06:10:46.726 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T06:10:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noa 141/198 2024-03-26T06:10:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.n 142/198 2024-03-26T06:10:47.369 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T06:10:47.407 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T06:10:47.427 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T06:10:47.963 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.n 142/198 2024-03-26T06:10:47.992 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e9 143/198 2024-03-26T06:10:48.101 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e9 143/198 2024-03-26T06:10:48.270 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1964.g7e98f5f6.el 144/198 2024-03-26T06:10:48.302 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 145/198 2024-03-26T06:10:48.311 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-top-2:19.0.0-1964.g7e98f5f6.el9.noarch 170/198 2024-03-26T06:10:48.325 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 145/198 2024-03-26T06:10:48.325 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:48.326 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T06:10:48.326 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T06:10:48.326 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T06:10:48.326 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:10:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 146/198 2024-03-26T06:10:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 146/198 2024-03-26T06:10:48.410 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T06:10:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.0.0-1964.g7e98f5f6.el9.x86_64 147/198 2024-03-26T06:10:48.451 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T06:10:48.534 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T06:10:48.571 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T06:10:48.605 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T06:10:48.630 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T06:10:48.657 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T06:10:48.657 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T06:10:48.728 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T06:10:48.757 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T06:10:48.757 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T06:10:48.842 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T06:10:48.868 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T06:10:48.902 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T06:10:48.927 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T06:10:48.952 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T06:10:48.985 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T06:10:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T06:10:49.057 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T06:10:49.082 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T06:10:49.107 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T06:10:49.132 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T06:10:49.158 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T06:10:49.164 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T06:10:49.164 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T06:10:49.352 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T06:10:49.352 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T06:10:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T06:10:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 149/198 2024-03-26T06:10:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 149/198 2024-03-26T06:10:49.404 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:49.404 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T06:10:49.404 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T06:10:49.404 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T06:10:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:10:49.435 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T06:10:49.456 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1964.g7e98f 150/198 2024-03-26T06:10:49.459 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T06:10:49.478 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1964.g7e98f 150/198 2024-03-26T06:10:49.478 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:49.478 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T06:10:49.478 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:10:49.492 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T06:10:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T06:10:49.523 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 151/198 2024-03-26T06:10:49.546 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 151/198 2024-03-26T06:10:49.546 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:49.546 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T06:10:49.546 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T06:10:49.546 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T06:10:49.547 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:10:49.552 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T06:10:49.620 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T06:10:49.874 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 152/198 2024-03-26T06:10:49.897 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 152/198 2024-03-26T06:10:49.897 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:49.897 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T06:10:49.897 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T06:10:49.897 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T06:10:49.897 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:10:53.240 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 132/198 2024-03-26T06:10:53.240 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-03-26T06:10:53.240 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-03-26T06:10:53.240 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-03-26T06:10:53.240 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-03-26T06:10:53.240 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-03-26T06:10:53.240 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-03-26T06:10:53.241 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-03-26T06:10:53.241 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:10:53.408 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 133/198 2024-03-26T06:10:53.423 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 133/198 2024-03-26T06:10:53.423 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:53.423 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T06:10:53.423 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:10:54.235 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T06:10:54.235 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:19.0.0-1964.g7e98f5f6.el9.x86_64 1/198 2024-03-26T06:10:54.235 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 3/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 4/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1964.g7e98f 5/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 6/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 7/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 8/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 9/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 10/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 11/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:19.0.0-1964.g7e98f5f6.el9.x86_64 12/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 13/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-shell-2:19.0.0-1964.g7e98f5f6.el9.x86_64 14/198 2024-03-26T06:10:54.236 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_ 15/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 16/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 17/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_6 18/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86 19/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 20/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1964.g7e98f5f6.el 21/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:19.0.0-1964.g7e98f5f6.el9. 22/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:19.0.0-1964.g7e98f5f6.el9.x86_6 23/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:19.0.0-1964.g7e98f5f6.el9.x86_64 24/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 25/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 26/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 27/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 28/198 2024-03-26T06:10:54.237 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 29/198 2024-03-26T06:10:54.255 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1964.g7e98f5f6. 30/198 2024-03-26T06:10:54.255 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noa 31/198 2024-03-26T06:10:54.255 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.n 32/198 2024-03-26T06:10:54.255 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e9 33/198 2024-03-26T06:10:54.255 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1964.g7e98f5f6.el 34/198 2024-03-26T06:10:54.255 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 35/198 2024-03-26T06:10:54.255 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1964.g7e98f5f6.e 36/198 2024-03-26T06:10:54.255 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 37/198 2024-03-26T06:10:54.255 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 38/198 2024-03-26T06:10:54.255 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-top-2:19.0.0-1964.g7e98f5f6.el9.noarch 39/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T06:10:54.256 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T06:10:54.257 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T06:10:54.258 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T06:10:54.259 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T06:10:54.260 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T06:10:54.260 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T06:10:54.260 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T06:10:54.260 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T06:10:54.260 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T06:10:54.260 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T06:10:54.260 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T06:10:54.260 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T06:10:54.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T06:10:54.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T06:10:54.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T06:10:54.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T06:10:54.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T06:10:54.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T06:10:54.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T06:10:54.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T06:10:54.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T06:10:54.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T06:10:54.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T06:10:54.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 151/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 153/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T06:10:54.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T06:10:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T06:10:54.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T06:10:54.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T06:10:54.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T06:10:54.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T06:10:54.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T06:10:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.0.0-1964.g7e98f5f6.el9.x86_64 153/198 2024-03-26T06:10:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 154/198 2024-03-26T06:10:54.675 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 155/198 2024-03-26T06:10:54.711 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T06:10:54.902 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 134/198 2024-03-26T06:10:54.906 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.0.0-1964.g7e98f5f6.el9.x86_64 157/198 2024-03-26T06:10:54.928 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 134/198 2024-03-26T06:10:54.928 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:54.928 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T06:10:54.928 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T06:10:54.928 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T06:10:54.928 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:10:55.031 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T06:10:55.068 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 159/198 2024-03-26T06:10:55.129 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 135/198 2024-03-26T06:10:55.150 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 135/198 2024-03-26T06:10:55.150 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:55.151 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T06:10:55.151 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T06:10:55.151 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T06:10:55.151 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:10:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T06:10:55.217 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T06:10:55.260 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T06:10:55.293 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_ 163/198 2024-03-26T06:10:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T06:10:55.371 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T06:10:55.437 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T06:10:55.502 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T06:10:55.547 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T06:10:55.587 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 136/198 2024-03-26T06:10:55.609 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 136/198 2024-03-26T06:10:55.610 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:55.610 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T06:10:55.610 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T06:10:55.610 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T06:10:55.610 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:10:55.705 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T06:10:56.156 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 138/198 2024-03-26T06:10:56.180 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T06:10:56.188 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 138/198 2024-03-26T06:10:56.219 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T06:10:56.220 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1964.g7e98f5f6.e 139/198 2024-03-26T06:10:56.239 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T06:10:56.279 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1964.g7e98f5f6. 140/198 2024-03-26T06:10:56.302 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noa 141/198 2024-03-26T06:10:56.533 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noa 141/198 2024-03-26T06:10:56.570 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.n 142/198 2024-03-26T06:10:57.128 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.0.0-1964.g7e98f5f6.el9.noarch 170/198 2024-03-26T06:10:57.215 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T06:10:57.265 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T06:10:57.338 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T06:10:57.338 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:57.338 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2024-03-26T06:10:57.338 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T06:10:57.338 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T06:10:57.338 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T06:10:57.338 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T06:10:57.338 INFO:teuthology.orchestra.run.smithi066.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T06:10:57.338 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T06:10:57.338 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T06:10:57.339 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.340 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:10:57.341 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.342 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.342 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.342 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:10:57.342 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:10:57.342 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.342 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.342 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:10:57.342 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T06:10:57.342 INFO:teuthology.orchestra.run.smithi066.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T06:10:57.342 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T06:10:57.342 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: jq-1.6-16.el9.x86_64 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.343 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T06:10:57.344 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T06:10:57.345 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T06:10:57.346 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T06:10:57.347 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T06:10:57.348 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.349 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T06:10:57.350 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T06:10:57.351 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T06:10:57.352 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T06:10:57.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-03-26T06:10:57.352 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-03-26T06:10:57.357 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T06:10:57.393 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T06:10:57.418 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T06:10:57.443 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T06:10:57.469 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T06:10:57.470 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T06:10:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.n 142/198 2024-03-26T06:10:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T06:10:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e9 143/198 2024-03-26T06:10:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T06:10:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T06:10:57.655 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T06:10:57.660 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e9 143/198 2024-03-26T06:10:57.681 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T06:10:57.714 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T06:10:57.740 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T06:10:57.765 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T06:10:57.793 DEBUG:teuthology.parallel:result is None 2024-03-26T06:10:57.799 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T06:10:57.823 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1964.g7e98f5f6.el 144/198 2024-03-26T06:10:57.852 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T06:10:57.856 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 145/198 2024-03-26T06:10:57.879 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 145/198 2024-03-26T06:10:57.879 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:57.879 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T06:10:57.879 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T06:10:57.879 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T06:10:57.879 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:10:57.886 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T06:10:57.911 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T06:10:57.925 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 146/198 2024-03-26T06:10:57.936 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 146/198 2024-03-26T06:10:57.936 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T06:10:57.962 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T06:10:57.982 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.0.0-1964.g7e98f5f6.el9.x86_64 147/198 2024-03-26T06:10:57.987 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T06:10:57.995 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T06:10:57.995 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T06:10:58.190 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T06:10:58.190 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T06:10:58.290 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T06:10:58.315 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T06:10:58.347 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T06:10:58.374 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T06:10:58.399 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T06:10:58.468 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T06:10:58.863 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T06:10:58.894 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 149/198 2024-03-26T06:10:58.917 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 149/198 2024-03-26T06:10:58.917 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:58.917 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T06:10:58.917 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T06:10:58.917 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T06:10:58.917 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:10:58.961 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1964.g7e98f 150/198 2024-03-26T06:10:58.983 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1964.g7e98f 150/198 2024-03-26T06:10:58.983 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:58.983 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T06:10:58.983 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:10:59.029 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 151/198 2024-03-26T06:10:59.051 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 151/198 2024-03-26T06:10:59.051 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:59.051 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T06:10:59.051 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T06:10:59.051 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T06:10:59.051 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:10:59.363 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 152/198 2024-03-26T06:10:59.385 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 152/198 2024-03-26T06:10:59.385 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T06:10:59.385 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T06:10:59.386 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T06:10:59.386 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T06:10:59.386 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:11:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T06:11:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.0.0-1964.g7e98f5f6.el9.x86_64 1/198 2024-03-26T06:11:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2/198 2024-03-26T06:11:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 3/198 2024-03-26T06:11:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 4/198 2024-03-26T06:11:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1964.g7e98f 5/198 2024-03-26T06:11:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 6/198 2024-03-26T06:11:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 7/198 2024-03-26T06:11:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 8/198 2024-03-26T06:11:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 9/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 10/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 11/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.0.0-1964.g7e98f5f6.el9.x86_64 12/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 13/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.0.0-1964.g7e98f5f6.el9.x86_64 14/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_ 15/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 16/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 17/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_6 18/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86 19/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 20/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1964.g7e98f5f6.el 21/198 2024-03-26T06:11:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.0.0-1964.g7e98f5f6.el9. 22/198 2024-03-26T06:11:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.0.0-1964.g7e98f5f6.el9.x86_6 23/198 2024-03-26T06:11:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.0.0-1964.g7e98f5f6.el9.x86_64 24/198 2024-03-26T06:11:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 25/198 2024-03-26T06:11:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 26/198 2024-03-26T06:11:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 27/198 2024-03-26T06:11:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 28/198 2024-03-26T06:11:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 29/198 2024-03-26T06:11:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1964.g7e98f5f6. 30/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noa 31/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.n 32/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e9 33/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1964.g7e98f5f6.el 34/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 35/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1964.g7e98f5f6.e 36/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 37/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 38/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.0.0-1964.g7e98f5f6.el9.noarch 39/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T06:11:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T06:11:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T06:11:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T06:11:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T06:11:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T06:11:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T06:11:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T06:11:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T06:11:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 151/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 153/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T06:11:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T06:11:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T06:11:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T06:11:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T06:11:03.826 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.0.0-1964.g7e98f5f6.el9.x86_64 153/198 2024-03-26T06:11:03.867 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 154/198 2024-03-26T06:11:03.898 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 155/198 2024-03-26T06:11:03.933 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T06:11:04.122 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-shell-2:19.0.0-1964.g7e98f5f6.el9.x86_64 157/198 2024-03-26T06:11:04.258 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T06:11:04.307 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 159/198 2024-03-26T06:11:04.346 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T06:11:04.398 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T06:11:04.442 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T06:11:04.474 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_ 163/198 2024-03-26T06:11:04.506 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T06:11:04.536 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T06:11:04.597 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T06:11:04.664 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T06:11:04.719 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T06:11:05.294 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T06:11:05.333 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T06:11:05.352 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T06:11:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:19.0.0-1964.g7e98f5f6.el9.noarch 170/198 2024-03-26T06:11:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T06:11:06.302 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T06:11:06.383 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T06:11:06.432 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T06:11:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T06:11:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T06:11:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T06:11:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T06:11:06.598 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T06:11:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T06:11:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:11:06.614 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-03-26T06:11:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T06:11:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T06:11:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T06:11:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T06:11:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T06:11:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T06:11:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T06:11:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T06:11:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T06:11:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T06:11:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T06:11:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T06:11:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-26T06:11:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: jq-1.6-16.el9.x86_64 2024-03-26T06:11:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T06:11:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T06:11:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T06:11:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T06:11:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T06:11:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T06:11:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T06:11:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T06:11:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T06:11:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T06:11:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T06:11:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T06:11:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T06:11:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T06:11:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T06:11:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T06:11:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:11:06.629 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-26T06:11:06.629 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T06:11:06.630 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T06:11:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T06:11:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T06:11:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T06:11:06.780 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T06:11:06.805 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T06:11:06.847 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T06:11:06.886 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T06:11:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T06:11:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T06:11:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T06:11:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T06:11:07.028 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T06:11:07.034 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T06:11:07.034 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T06:11:07.080 DEBUG:teuthology.parallel:result is None 2024-03-26T06:11:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T06:11:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T06:11:07.289 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T06:11:07.313 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T06:11:07.346 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T06:11:07.373 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T06:11:07.398 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T06:11:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T06:11:12.387 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T06:11:12.387 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.0.0-1964.g7e98f5f6.el9.x86_64 1/198 2024-03-26T06:11:12.387 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2/198 2024-03-26T06:11:12.387 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 3/198 2024-03-26T06:11:12.387 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 4/198 2024-03-26T06:11:12.387 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1964.g7e98f 5/198 2024-03-26T06:11:12.387 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 6/198 2024-03-26T06:11:12.387 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 7/198 2024-03-26T06:11:12.387 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 8/198 2024-03-26T06:11:12.387 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 9/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 10/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 11/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.0.0-1964.g7e98f5f6.el9.x86_64 12/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 13/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-shell-2:19.0.0-1964.g7e98f5f6.el9.x86_64 14/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_ 15/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 16/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 17/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_6 18/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86 19/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 20/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1964.g7e98f5f6.el 21/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.0.0-1964.g7e98f5f6.el9. 22/198 2024-03-26T06:11:12.388 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.0.0-1964.g7e98f5f6.el9.x86_6 23/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.0.0-1964.g7e98f5f6.el9.x86_64 24/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 25/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 26/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 27/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 28/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 29/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1964.g7e98f5f6. 30/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noa 31/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.n 32/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e9 33/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1964.g7e98f5f6.el 34/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 35/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1964.g7e98f5f6.e 36/198 2024-03-26T06:11:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 37/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 38/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:19.0.0-1964.g7e98f5f6.el9.noarch 39/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T06:11:12.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T06:11:12.391 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T06:11:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T06:11:12.393 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T06:11:12.394 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T06:11:12.395 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T06:11:12.396 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T06:11:12.397 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 151/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T06:11:12.398 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 153/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T06:11:12.399 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T06:11:12.400 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T06:11:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T06:11:12.402 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T06:11:12.402 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T06:11:12.402 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T06:11:15.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T06:11:15.393 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:11:15.393 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-03-26T06:11:15.393 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T06:11:15.393 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T06:11:15.393 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T06:11:15.393 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T06:11:15.393 INFO:teuthology.orchestra.run.smithi149.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T06:11:15.393 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T06:11:15.393 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T06:11:15.393 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T06:11:15.394 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.395 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:15.396 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.397 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.397 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.397 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:15.397 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:15.397 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.397 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.397 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:19.0.0-1964.g7e98f5f6.el9.noarch 2024-03-26T06:11:15.397 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T06:11:15.397 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: jq-1.6-16.el9.x86_64 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T06:11:15.399 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T06:11:15.400 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T06:11:15.401 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T06:11:15.402 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T06:11:15.403 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T06:11:15.404 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T06:11:15.405 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.0.0-1964.g7e98f5f6.el9.x86_64 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T06:11:15.406 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T06:11:15.407 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T06:11:15.407 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T06:11:15.407 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T06:11:15.407 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T06:11:15.407 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:11:15.407 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-26T06:11:15.881 DEBUG:teuthology.parallel:result is None 2024-03-26T06:11:15.882 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:11:16.100 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T06:11:16.125 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-1964.g7e98f5f6.el9 2024-03-26T06:11:16.125 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1964.g7e98f5f6.el9 2024-03-26T06:11:16.126 INFO:teuthology.task.install:The correct ceph version 19.0.0-1964.g7e98f5f6 is installed. 2024-03-26T06:11:16.128 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:11:16.264 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T06:11:16.290 INFO:teuthology.orchestra.run.smithi066.stdout:19.0.0-1964.g7e98f5f6.el9 2024-03-26T06:11:16.291 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1964.g7e98f5f6.el9 2024-03-26T06:11:16.291 INFO:teuthology.task.install:The correct ceph version 19.0.0-1964.g7e98f5f6 is installed. 2024-03-26T06:11:16.293 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:11:16.511 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T06:11:16.535 INFO:teuthology.orchestra.run.smithi149.stdout:19.0.0-1964.g7e98f5f6.el9 2024-03-26T06:11:16.536 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1964.g7e98f5f6.el9 2024-03-26T06:11:16.536 INFO:teuthology.task.install:The correct ceph version 19.0.0-1964.g7e98f5f6 is installed. 2024-03-26T06:11:16.538 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-26T06:11:16.538 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:11:16.538 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T06:11:16.573 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-26T06:11:16.573 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T06:11:16.612 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-26T06:11:16.613 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T06:11:16.645 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-26T06:11:16.645 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:11:16.646 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T06:11:16.676 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T06:11:16.746 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-26T06:11:16.746 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T06:11:16.783 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T06:11:16.857 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-26T06:11:16.857 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T06:11:16.888 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T06:11:16.956 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-26T06:11:16.957 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:11:16.957 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T06:11:16.987 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T06:11:17.056 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-26T06:11:17.056 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T06:11:17.089 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T06:11:17.160 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-26T06:11:17.160 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T06:11:17.188 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T06:11:17.256 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-26T06:11:17.256 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:11:17.256 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T06:11:17.287 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T06:11:17.359 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-26T06:11:17.359 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T06:11:17.390 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T06:11:17.464 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-26T06:11:17.464 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T06:11:17.492 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T06:11:17.561 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-26T06:11:17.667 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '7e98f5f6b535defc7fa715b456904ffc0500aa9f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-26T06:11:17.668 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:11:17.668 INFO:tasks.cephadm:Cluster fsid is a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:11:17.668 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-26T06:11:17.668 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.37', 'mon.b': '172.21.15.66', 'mon.c': '172.21.15.149'} 2024-03-26T06:11:17.669 INFO:tasks.cephadm:First mon is mon.a on smithi037 2024-03-26T06:11:17.669 INFO:tasks.cephadm:First mgr is x 2024-03-26T06:11:17.669 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-26T06:11:17.669 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname $(hostname -s) 2024-03-26T06:11:17.701 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname $(hostname -s) 2024-03-26T06:11:17.732 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-03-26T06:11:17.760 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-26T06:11:17.760 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:11:17.906 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '7e98f5f6b535defc7fa715b456904ffc0500aa9f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77904/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1964-g7e98f5f6', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1964.g7e98f5f6'}, 'url': 'https://4.chacra.ceph.com/r/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-23 11:54:09.441197', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-2024mar8-2', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-26T06:11:18.048 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref rishabh-2024mar8-2, sha1 7e98f5f6b535defc7fa715b456904ffc0500aa9f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:11:18.050 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/x86_64/flavors/default/cephadm 2024-03-26T06:11:18.050 DEBUG:teuthology.orchestra.run.smithi037:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T06:11:18.529 INFO:teuthology.orchestra.run.smithi037.stdout:-rw-r--r--. 1 ubuntu ubuntu 767477 Mar 26 06:11 /home/ubuntu/cephtest/cephadm 2024-03-26T06:11:18.529 DEBUG:teuthology.orchestra.run.smithi066:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T06:11:19.132 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r--. 1 ubuntu ubuntu 767477 Mar 26 06:11 /home/ubuntu/cephtest/cephadm 2024-03-26T06:11:19.133 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-2024mar8-2/7e98f5f6b535defc7fa715b456904ffc0500aa9f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T06:11:19.718 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 767477 Mar 26 06:11 /home/ubuntu/cephtest/cephadm 2024-03-26T06:11:19.718 DEBUG:teuthology.orchestra.run.smithi037:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T06:11:19.737 DEBUG:teuthology.orchestra.run.smithi066:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T06:11:19.757 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T06:11:19.791 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f on all hosts... 2024-03-26T06:11:19.792 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f pull 2024-03-26T06:11:19.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f pull 2024-03-26T06:11:19.800 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f pull 2024-03-26T06:11:20.648 INFO:teuthology.orchestra.run.smithi037.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f... 2024-03-26T06:11:20.648 INFO:teuthology.orchestra.run.smithi066.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f... 2024-03-26T06:11:20.654 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f... 2024-03-26T06:12:24.914 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-26T06:12:24.915 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.0.0-1964-g7e98f5f6 (7e98f5f6b535defc7fa715b456904ffc0500aa9f) squid (dev)", 2024-03-26T06:12:24.915 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c", 2024-03-26T06:12:24.915 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-03-26T06:12:24.915 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265" 2024-03-26T06:12:24.915 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-03-26T06:12:24.915 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-26T06:12:33.235 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:12:33.235 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1964-g7e98f5f6 (7e98f5f6b535defc7fa715b456904ffc0500aa9f) squid (dev)", 2024-03-26T06:12:33.235 INFO:teuthology.orchestra.run.smithi037.stdout: "image_id": "10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c", 2024-03-26T06:12:33.235 INFO:teuthology.orchestra.run.smithi037.stdout: "repo_digests": [ 2024-03-26T06:12:33.236 INFO:teuthology.orchestra.run.smithi037.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265" 2024-03-26T06:12:33.236 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:12:33.236 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:12:33.576 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-26T06:12:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.0.0-1964-g7e98f5f6 (7e98f5f6b535defc7fa715b456904ffc0500aa9f) squid (dev)", 2024-03-26T06:12:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "image_id": "10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c", 2024-03-26T06:12:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "repo_digests": [ 2024-03-26T06:12:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265" 2024-03-26T06:12:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-03-26T06:12:33.577 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-26T06:12:33.601 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph 2024-03-26T06:12:33.631 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph 2024-03-26T06:12:33.665 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-03-26T06:12:33.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /etc/ceph 2024-03-26T06:12:33.727 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /etc/ceph 2024-03-26T06:12:33.758 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-03-26T06:12:33.787 INFO:tasks.cephadm:Writing seed config... 2024-03-26T06:12:33.789 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-26T06:12:33.789 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-26T06:12:33.789 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-26T06:12:33.789 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-26T06:12:33.789 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-26T06:12:33.789 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-26T06:12:33.789 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-26T06:12:33.789 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-26T06:12:33.789 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-26T06:12:33.789 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-26T06:12:33.790 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-26T06:12:33.791 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-26T06:12:33.792 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-26T06:12:33.792 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-26T06:12:33.792 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-26T06:12:33.792 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-26T06:12:33.792 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-26T06:12:33.792 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-26T06:12:33.792 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-26T06:12:33.792 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-26T06:12:33.792 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-26T06:12:33.792 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-26T06:12:33.793 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:12:33.793 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-26T06:12:33.808 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a86fb218-eb37-11ee-95d0-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-26T06:12:33.808 DEBUG:teuthology.orchestra.run.smithi037:mon.a> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.a.service 2024-03-26T06:12:33.850 DEBUG:teuthology.orchestra.run.smithi037:mgr.x> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.x.service 2024-03-26T06:12:33.893 INFO:tasks.cephadm:Bootstrapping... 2024-03-26T06:12:33.893 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f -v bootstrap --fsid a86fb218-eb37-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.37 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-26T06:12:34.088 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-26T06:12:34.088 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f', '-v', 'bootstrap', '--fsid', 'a86fb218-eb37-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.37', '--skip-admin-label'] 2024-03-26T06:12:34.113 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T06:12:34.113 INFO:teuthology.orchestra.run.smithi037.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-26T06:12:34.113 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying podman|docker is present... 2024-03-26T06:12:34.136 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T06:12:34.136 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying lvm2 is present... 2024-03-26T06:12:34.137 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying time synchronization is in place... 2024-03-26T06:12:34.144 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T06:12:34.144 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T06:12:34.150 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T06:12:34.150 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-26T06:12:34.156 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-03-26T06:12:34.162 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-03-26T06:12:34.162 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-03-26T06:12:34.162 INFO:teuthology.orchestra.run.smithi037.stdout:Repeating the final host check... 2024-03-26T06:12:34.185 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T06:12:34.185 INFO:teuthology.orchestra.run.smithi037.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-26T06:12:34.185 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl is present 2024-03-26T06:12:34.185 INFO:teuthology.orchestra.run.smithi037.stdout:lvcreate is present 2024-03-26T06:12:34.191 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T06:12:34.191 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T06:12:34.197 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T06:12:34.197 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-26T06:12:34.203 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-03-26T06:12:34.209 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-03-26T06:12:34.209 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-03-26T06:12:34.210 INFO:teuthology.orchestra.run.smithi037.stdout:Host looks OK 2024-03-26T06:12:34.210 INFO:teuthology.orchestra.run.smithi037.stdout:Cluster fsid: a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:12:34.210 INFO:teuthology.orchestra.run.smithi037.stdout:Acquiring lock 139778606468544 on /run/cephadm/a86fb218-eb37-11ee-95d0-87774f69a715.lock 2024-03-26T06:12:34.210 INFO:teuthology.orchestra.run.smithi037.stdout:Lock 139778606468544 acquired on /run/cephadm/a86fb218-eb37-11ee-95d0-87774f69a715.lock 2024-03-26T06:12:34.210 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 3300 ... 2024-03-26T06:12:34.211 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 6789 ... 2024-03-26T06:12:34.211 INFO:teuthology.orchestra.run.smithi037.stdout:Base mon IP(s) is [172.21.15.37:3300, 172.21.15.37:6789], mon addrv is [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-03-26T06:12:34.214 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.37 metric 100 2024-03-26T06:12:34.214 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.37 metric 100 2024-03-26T06:12:34.216 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-26T06:12:34.217 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-26T06:12:34.217 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-26T06:12:34.219 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-26T06:12:34.219 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-26T06:12:34.219 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T06:12:34.219 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-26T06:12:34.219 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-03-26T06:12:34.219 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T06:12:34.220 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-03-26T06:12:34.220 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-03-26T06:12:34.220 INFO:teuthology.orchestra.run.smithi037.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-26T06:12:34.220 INFO:teuthology.orchestra.run.smithi037.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-26T06:12:34.221 INFO:teuthology.orchestra.run.smithi037.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f... 2024-03-26T06:12:34.689 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c 2024-03-26T06:12:34.689 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f... 2024-03-26T06:12:34.689 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Getting image source signatures 2024-03-26T06:12:34.689 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:cefe3f08b7d2324bed4b4b268e386dce7ad863cb02a7a7641e64ed611288b7c8 2024-03-26T06:12:34.689 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-26T06:12:34.689 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying config sha256:10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c 2024-03-26T06:12:34.689 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-26T06:12:35.693 INFO:teuthology.orchestra.run.smithi037.stdout:ceph: stdout ceph version 19.0.0-1964-g7e98f5f6 (7e98f5f6b535defc7fa715b456904ffc0500aa9f) squid (dev) 2024-03-26T06:12:35.693 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph version: ceph version 19.0.0-1964-g7e98f5f6 (7e98f5f6b535defc7fa715b456904ffc0500aa9f) squid (dev) 2024-03-26T06:12:35.693 INFO:teuthology.orchestra.run.smithi037.stdout:Extracting ceph user uid/gid from container image... 2024-03-26T06:12:36.676 INFO:teuthology.orchestra.run.smithi037.stdout:stat: stdout 167 167 2024-03-26T06:12:36.676 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial keys... 2024-03-26T06:12:37.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQBUZwJmmCDYOhAAlo9i0tj0/FGWT9Z/ioknMA== 2024-03-26T06:12:38.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQBVZwJmwHkdNRAAW/F/9X/VeAxDuRJBTi0h9g== 2024-03-26T06:12:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQBWZwJmdwyFMRAA0jlpsccbcwaHDkfd6u44NQ== 2024-03-26T06:12:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial monmap... 2024-03-26T06:12:40.490 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T06:12:40.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-26T06:12:40.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:12:40.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T06:12:40.491 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool for a [v2:172.21.15.37:3300,v1:172.21.15.37:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T06:12:40.491 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-03-26T06:12:40.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: set fsid to a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:12:40.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T06:12:40.491 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:12:40.491 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mon... 2024-03-26T06:12:41.370 INFO:teuthology.orchestra.run.smithi037.stdout:create mon.a on 2024-03-26T06:12:41.582 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-26T06:12:41.720 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-26T06:12:41.996 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a86fb218-eb37-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-a86fb218-eb37-11ee-95d0-87774f69a715.target. 2024-03-26T06:12:41.996 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a86fb218-eb37-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-a86fb218-eb37-11ee-95d0-87774f69a715.target. 2024-03-26T06:12:42.238 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.a 2024-03-26T06:12:42.238 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-26T06:12:42.402 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a86fb218-eb37-11ee-95d0-87774f69a715.target.wants/ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-a86fb218-eb37-11ee-95d0-87774f69a715@.service. 2024-03-26T06:12:42.919 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T06:12:42.919 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-26T06:12:42.925 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T06:12:42.925 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-26T06:12:42.925 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-26T06:12:42.925 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T06:12:42.926 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon to start... 2024-03-26T06:12:42.926 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon... 2024-03-26T06:12:43.978 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:43 smithi037 ceph-mon[31597]: from='client.? 172.21.15.37:0/1733886938' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-26T06:12:43.988 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout cluster: 2024-03-26T06:12:43.988 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout id: a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:12:43.988 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-26T06:12:43.988 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:12:43.988 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout services: 2024-03-26T06:12:43.988 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.541742s) 2024-03-26T06:12:43.988 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout data: 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pgs: 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.347+0000 7f91a8a68700 1 Processor -- start 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.347+0000 7f91a8a68700 1 -- start start 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.347+0000 7f91a8a68700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.347+0000 7f91a8a68700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a407b8b0 con 0x7f91a407ce30 2024-03-26T06:12:43.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.348+0000 7f91a259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.348+0000 7f91a259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a407b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54358/0 (socket says 172.21.15.37:54358) 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.348+0000 7f91a259c700 1 -- 172.21.15.37:0/242306167 learned_addr learned my addr 172.21.15.37:0/242306167 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.348+0000 7f91a259c700 1 -- 172.21.15.37:0/242306167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a407ba30 con 0x7f91a407ce30 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.348+0000 7f91a259c700 1 --2- 172.21.15.37:0/242306167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a407b2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f91940098a0 tx=0x7f9194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b228270a8b986c78 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.348+0000 7f91a159a700 1 -- 172.21.15.37:0/242306167 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9194004160 con 0x7f91a407ce30 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.348+0000 7f91a159a700 1 -- 172.21.15.37:0/242306167 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9194004300 con 0x7f91a407ce30 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.348+0000 7f91a159a700 1 -- 172.21.15.37:0/242306167 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91940036a0 con 0x7f91a407ce30 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.349+0000 7f91a8a68700 1 -- 172.21.15.37:0/242306167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 msgr2=0x7f91a407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.349+0000 7f91a8a68700 1 --2- 172.21.15.37:0/242306167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a407b2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f91940098a0 tx=0x7f9194009bb0 comp rx=0 tx=0).stop 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.349+0000 7f91a8a68700 1 -- 172.21.15.37:0/242306167 shutdown_connections 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.349+0000 7f91a8a68700 1 --2- 172.21.15.37:0/242306167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a407b2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:12:43.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.349+0000 7f91a8a68700 1 -- 172.21.15.37:0/242306167 >> 172.21.15.37:0/242306167 conn(0x7f91a4102f10 msgr2=0x7f91a4105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.349+0000 7f91a8a68700 1 -- 172.21.15.37:0/242306167 shutdown_connections 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.349+0000 7f91a8a68700 1 -- 172.21.15.37:0/242306167 wait complete. 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.350+0000 7f91a8a68700 1 Processor -- start 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.350+0000 7f91a8a68700 1 -- start start 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.350+0000 7f91a8a68700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a419d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.350+0000 7f91a8a68700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a407bf60 con 0x7f91a407ce30 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.350+0000 7f91a259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a419d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.350+0000 7f91a259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a419d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54374/0 (socket says 172.21.15.37:54374) 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.350+0000 7f91a259c700 1 -- 172.21.15.37:0/1733886938 learned_addr learned my addr 172.21.15.37:0/1733886938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.350+0000 7f91a259c700 1 -- 172.21.15.37:0/1733886938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a419d6c0 con 0x7f91a407ce30 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.350+0000 7f91a259c700 1 --2- 172.21.15.37:0/1733886938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a419d180 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9194000c00 tx=0x7f9194004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:12:43.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.350+0000 7f91937fe700 1 -- 172.21.15.37:0/1733886938 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9194003db0 con 0x7f91a407ce30 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.350+0000 7f91937fe700 1 -- 172.21.15.37:0/1733886938 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9194011630 con 0x7f91a407ce30 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.351+0000 7f91a8a68700 1 -- 172.21.15.37:0/1733886938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a419d950 con 0x7f91a407ce30 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.351+0000 7f91937fe700 1 -- 172.21.15.37:0/1733886938 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9194011950 con 0x7f91a407ce30 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.351+0000 7f91a8a68700 1 -- 172.21.15.37:0/1733886938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a41a0640 con 0x7f91a407ce30 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.351+0000 7f91937fe700 1 -- 172.21.15.37:0/1733886938 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9194011af0 con 0x7f91a407ce30 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.351+0000 7f91937fe700 1 -- 172.21.15.37:0/1733886938 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f919401c450 con 0x7f91a407ce30 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.352+0000 7f91a8a68700 1 -- 172.21.15.37:0/1733886938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a4068ec0 con 0x7f91a407ce30 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.354+0000 7f91937fe700 1 -- 172.21.15.37:0/1733886938 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f9194032020 con 0x7f91a407ce30 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.427+0000 7f91a8a68700 1 -- 172.21.15.37:0/1733886938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f91a41a0aa0 con 0x7f91a407ce30 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.428+0000 7f91937fe700 1 -- 172.21.15.37:0/1733886938 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f9194004260 con 0x7f91a407ce30 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.428+0000 7f91a8a68700 1 -- 172.21.15.37:0/1733886938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 msgr2=0x7f91a419d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:12:43.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.428+0000 7f91a8a68700 1 --2- 172.21.15.37:0/1733886938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a419d180 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9194000c00 tx=0x7f9194004750 comp rx=0 tx=0).stop 2024-03-26T06:12:43.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.428+0000 7f91a8a68700 1 -- 172.21.15.37:0/1733886938 shutdown_connections 2024-03-26T06:12:43.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.428+0000 7f91a8a68700 1 --2- 172.21.15.37:0/1733886938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a407ce30 0x7f91a419d180 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:12:43.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.428+0000 7f91a8a68700 1 -- 172.21.15.37:0/1733886938 >> 172.21.15.37:0/1733886938 conn(0x7f91a4102f10 msgr2=0x7f91a4103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:12:43.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.429+0000 7f91a8a68700 1 -- 172.21.15.37:0/1733886938 shutdown_connections 2024-03-26T06:12:43.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:43.429+0000 7f91a8a68700 1 -- 172.21.15.37:0/1733886938 wait complete. 2024-03-26T06:12:43.993 INFO:teuthology.orchestra.run.smithi037.stdout:mon is available 2024-03-26T06:12:43.993 INFO:teuthology.orchestra.run.smithi037.stdout:Assimilating anything we can from ceph.conf... 2024-03-26T06:12:45.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:12:45.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T06:12:45.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.389+0000 7fdf84a9e700 1 Processor -- start 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.389+0000 7fdf84a9e700 1 -- start start 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.389+0000 7fdf84a9e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8010a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.390+0000 7fdf84a9e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8007ae50 con 0x7fdf80108360 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.390+0000 7fdf7e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8010a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.390+0000 7fdf7e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8010a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54388/0 (socket says 172.21.15.37:54388) 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.390+0000 7fdf7e59c700 1 -- 172.21.15.37:0/2142759408 learned_addr learned my addr 172.21.15.37:0/2142759408 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.390+0000 7fdf7e59c700 1 -- 172.21.15.37:0/2142759408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8010ac90 con 0x7fdf80108360 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.390+0000 7fdf7e59c700 1 --2- 172.21.15.37:0/2142759408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8010a750 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdf700098a0 tx=0x7fdf70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a865777d197c3b0e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:12:45.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.390+0000 7fdf7d59a700 1 -- 172.21.15.37:0/2142759408 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf70004160 con 0x7fdf80108360 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.390+0000 7fdf7d59a700 1 -- 172.21.15.37:0/2142759408 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf70004300 con 0x7fdf80108360 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.390+0000 7fdf7d59a700 1 -- 172.21.15.37:0/2142759408 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf700036a0 con 0x7fdf80108360 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.391+0000 7fdf84a9e700 1 -- 172.21.15.37:0/2142759408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 msgr2=0x7fdf8010a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.391+0000 7fdf84a9e700 1 --2- 172.21.15.37:0/2142759408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8010a750 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdf700098a0 tx=0x7fdf70009bb0 comp rx=0 tx=0).stop 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.391+0000 7fdf84a9e700 1 -- 172.21.15.37:0/2142759408 shutdown_connections 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.391+0000 7fdf84a9e700 1 --2- 172.21.15.37:0/2142759408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8010a750 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.391+0000 7fdf84a9e700 1 -- 172.21.15.37:0/2142759408 >> 172.21.15.37:0/2142759408 conn(0x7fdf80101e10 msgr2=0x7fdf80104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.391+0000 7fdf84a9e700 1 -- 172.21.15.37:0/2142759408 shutdown_connections 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.391+0000 7fdf84a9e700 1 -- 172.21.15.37:0/2142759408 wait complete. 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf84a9e700 1 Processor -- start 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf84a9e700 1 -- start start 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf84a9e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf84a9e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8010b100 con 0x7fdf80108360 2024-03-26T06:12:45.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf7e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:12:45.064 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf7e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8019afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54394/0 (socket says 172.21.15.37:54394) 2024-03-26T06:12:45.064 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf7e59c700 1 -- 172.21.15.37:0/622349682 learned_addr learned my addr 172.21.15.37:0/622349682 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:12:45.064 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf7e59c700 1 -- 172.21.15.37:0/622349682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8019b4f0 con 0x7fdf80108360 2024-03-26T06:12:45.064 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf7e59c700 1 --2- 172.21.15.37:0/622349682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8019afb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdf70000c00 tx=0x7fdf70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:12:45.064 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf6f7fe700 1 -- 172.21.15.37:0/622349682 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf70003db0 con 0x7fdf80108360 2024-03-26T06:12:45.064 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf6f7fe700 1 -- 172.21.15.37:0/622349682 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf70011630 con 0x7fdf80108360 2024-03-26T06:12:45.064 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf6f7fe700 1 -- 172.21.15.37:0/622349682 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf70011950 con 0x7fdf80108360 2024-03-26T06:12:45.064 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf84a9e700 1 -- 172.21.15.37:0/622349682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8019b780 con 0x7fdf80108360 2024-03-26T06:12:45.064 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.392+0000 7fdf84a9e700 1 -- 172.21.15.37:0/622349682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8019e470 con 0x7fdf80108360 2024-03-26T06:12:45.064 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.393+0000 7fdf6f7fe700 1 -- 172.21.15.37:0/622349682 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdf7001a460 con 0x7fdf80108360 2024-03-26T06:12:45.064 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.393+0000 7fdf6f7fe700 1 -- 172.21.15.37:0/622349682 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdf7001b780 con 0x7fdf80108360 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.393+0000 7fdf84a9e700 1 -- 172.21.15.37:0/622349682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf80068ec0 con 0x7fdf80108360 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.396+0000 7fdf6f7fe700 1 -- 172.21.15.37:0/622349682 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fdf7001ba20 con 0x7fdf80108360 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.467+0000 7fdf84a9e700 1 -- 172.21.15.37:0/622349682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fdf8002d9c0 con 0x7fdf80108360 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.470+0000 7fdf6f7fe700 1 -- 172.21.15.37:0/622349682 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fdf7001bc00 con 0x7fdf80108360 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.470+0000 7fdf6f7fe700 1 -- 172.21.15.37:0/622349682 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf7001bde0 con 0x7fdf80108360 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.471+0000 7fdf84a9e700 1 -- 172.21.15.37:0/622349682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 msgr2=0x7fdf8019afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.471+0000 7fdf84a9e700 1 --2- 172.21.15.37:0/622349682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8019afb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdf70000c00 tx=0x7fdf70004750 comp rx=0 tx=0).stop 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.471+0000 7fdf84a9e700 1 -- 172.21.15.37:0/622349682 shutdown_connections 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.471+0000 7fdf84a9e700 1 --2- 172.21.15.37:0/622349682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf80108360 0x7fdf8019afb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.471+0000 7fdf84a9e700 1 -- 172.21.15.37:0/622349682 >> 172.21.15.37:0/622349682 conn(0x7fdf80101e10 msgr2=0x7fdf801040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.471+0000 7fdf84a9e700 1 -- 172.21.15.37:0/622349682 shutdown_connections 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:44.471+0000 7fdf84a9e700 1 -- 172.21.15.37:0/622349682 wait complete. 2024-03-26T06:12:45.065 INFO:teuthology.orchestra.run.smithi037.stdout:Generating new minimal ceph.conf... 2024-03-26T06:12:45.177 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:44 smithi037 ceph-mon[31597]: from='client.? 172.21.15.37:0/622349682' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T06:12:45.177 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:44 smithi037 ceph-mon[31597]: from='client.? 172.21.15.37:0/622349682' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-26T06:12:46.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:45 smithi037 ceph-mon[31597]: from='client.? 172.21.15.37:0/4247024130' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:12:46.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.507+0000 7f3854635700 1 Processor -- start 2024-03-26T06:12:46.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.507+0000 7f3854635700 1 -- start start 2024-03-26T06:12:46.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.507+0000 7f3854635700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c10a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.507+0000 7f3854635700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c07ae10 con 0x7f384c108330 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.508+0000 7f38523d1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c10a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.508+0000 7f38523d1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c10a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54398/0 (socket says 172.21.15.37:54398) 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.508+0000 7f38523d1700 1 -- 172.21.15.37:0/4065354761 learned_addr learned my addr 172.21.15.37:0/4065354761 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.508+0000 7f38523d1700 1 -- 172.21.15.37:0/4065354761 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c10ac60 con 0x7f384c108330 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.508+0000 7f38523d1700 1 --2- 172.21.15.37:0/4065354761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c10a720 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f383c0098a0 tx=0x7f383c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b644ffa4027a6d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.508+0000 7f38513cf700 1 -- 172.21.15.37:0/4065354761 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f383c004160 con 0x7f384c108330 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.508+0000 7f38513cf700 1 -- 172.21.15.37:0/4065354761 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f383c004300 con 0x7f384c108330 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.508+0000 7f38513cf700 1 -- 172.21.15.37:0/4065354761 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f383c011b50 con 0x7f384c108330 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.509+0000 7f3854635700 1 -- 172.21.15.37:0/4065354761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 msgr2=0x7f384c10a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.509+0000 7f3854635700 1 --2- 172.21.15.37:0/4065354761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c10a720 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f383c0098a0 tx=0x7f383c009bb0 comp rx=0 tx=0).stop 2024-03-26T06:12:46.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.509+0000 7f3854635700 1 -- 172.21.15.37:0/4065354761 shutdown_connections 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.509+0000 7f3854635700 1 --2- 172.21.15.37:0/4065354761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c10a720 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.509+0000 7f3854635700 1 -- 172.21.15.37:0/4065354761 >> 172.21.15.37:0/4065354761 conn(0x7f384c101da0 msgr2=0x7f384c1041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.509+0000 7f3854635700 1 -- 172.21.15.37:0/4065354761 shutdown_connections 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.509+0000 7f3854635700 1 -- 172.21.15.37:0/4065354761 wait complete. 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.510+0000 7f3854635700 1 Processor -- start 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.510+0000 7f3854635700 1 -- start start 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.510+0000 7f3854635700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c107bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.510+0000 7f3854635700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c10bc20 con 0x7f384c108330 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.510+0000 7f38523d1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c107bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.510+0000 7f38523d1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c107bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54412/0 (socket says 172.21.15.37:54412) 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.510+0000 7f38523d1700 1 -- 172.21.15.37:0/4247024130 learned_addr learned my addr 172.21.15.37:0/4247024130 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.510+0000 7f38523d1700 1 -- 172.21.15.37:0/4247024130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c1080f0 con 0x7f384c108330 2024-03-26T06:12:46.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.510+0000 7f38523d1700 1 --2- 172.21.15.37:0/4247024130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c107bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f383c011b60 tx=0x7f383c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.510+0000 7f38437fe700 1 -- 172.21.15.37:0/4247024130 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f383c003970 con 0x7f384c108330 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.511+0000 7f38437fe700 1 -- 172.21.15.37:0/4247024130 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f383c003b10 con 0x7f384c108330 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.511+0000 7f3854635700 1 -- 172.21.15.37:0/4247024130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384c1062e0 con 0x7f384c108330 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.511+0000 7f3854635700 1 -- 172.21.15.37:0/4247024130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c106820 con 0x7f384c108330 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.511+0000 7f38437fe700 1 -- 172.21.15.37:0/4247024130 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f383c024610 con 0x7f384c108330 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.511+0000 7f38437fe700 1 -- 172.21.15.37:0/4247024130 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f383c024cc0 con 0x7f384c108330 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.511+0000 7f38437fe700 1 -- 172.21.15.37:0/4247024130 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f383c011e00 con 0x7f384c108330 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.512+0000 7f3854635700 1 -- 172.21.15.37:0/4247024130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384c068ec0 con 0x7f384c108330 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.514+0000 7f38437fe700 1 -- 172.21.15.37:0/4247024130 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f383c043e50 con 0x7f384c108330 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.584+0000 7f3854635700 1 -- 172.21.15.37:0/4247024130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f384c106d50 con 0x7f384c108330 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.585+0000 7f38437fe700 1 -- 172.21.15.37:0/4247024130 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f383c032030 con 0x7f384c108330 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.585+0000 7f3854635700 1 -- 172.21.15.37:0/4247024130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 msgr2=0x7f384c107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.585+0000 7f3854635700 1 --2- 172.21.15.37:0/4247024130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c107bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f383c011b60 tx=0x7f383c011c40 comp rx=0 tx=0).stop 2024-03-26T06:12:46.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.585+0000 7f3854635700 1 -- 172.21.15.37:0/4247024130 shutdown_connections 2024-03-26T06:12:46.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.585+0000 7f3854635700 1 --2- 172.21.15.37:0/4247024130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f384c108330 0x7f384c107bb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:12:46.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.585+0000 7f3854635700 1 -- 172.21.15.37:0/4247024130 >> 172.21.15.37:0/4247024130 conn(0x7f384c101da0 msgr2=0x7f384c102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:12:46.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.586+0000 7f3854635700 1 -- 172.21.15.37:0/4247024130 shutdown_connections 2024-03-26T06:12:46.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:45.586+0000 7f3854635700 1 -- 172.21.15.37:0/4247024130 wait complete. 2024-03-26T06:12:46.077 INFO:teuthology.orchestra.run.smithi037.stdout:Restarting the monitor... 2024-03-26T06:12:46.337 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:46 smithi037 systemd[1]: Stopping Ceph mon.a for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:12:46.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:46 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a[31593]: 2024-03-26T06:12:46.336+0000 7f352eb15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:12:46.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:46 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a[31593]: 2024-03-26T06:12:46.336+0000 7f352eb15700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-26T06:12:46.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:46 smithi037 podman[31842]: 2024-03-26 06:12:46.368264776 +0000 UTC m=+0.133052015 container died 7785af8099e899dd74b4742f707fb18da3a8109fde35e84cc419393e1a83d56a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-26T06:12:46.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:46 smithi037 podman[31842]: 2024-03-26 06:12:46.695510589 +0000 UTC m=+0.460297824 container cleanup 7785af8099e899dd74b4742f707fb18da3a8109fde35e84cc419393e1a83d56a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:12:46.879 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:46 smithi037 bash[31842]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a 2024-03-26T06:12:47.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:46 smithi037 podman[31856]: 2024-03-26 06:12:46.879817818 +0000 UTC m=+0.514385675 container remove 7785af8099e899dd74b4742f707fb18da3a8109fde35e84cc419393e1a83d56a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:12:47.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:47 smithi037 systemd[1]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-26T06:12:47.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:47 smithi037 systemd[1]: Stopped Ceph mon.a for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:12:47.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:47 smithi037 systemd[1]: Starting Ceph mon.a for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:12:47.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:47 smithi037 podman[31955]: 2024-03-26 06:12:47.234574045 +0000 UTC m=+0.017458237 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:12:47.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:47 smithi037 podman[31955]: 2024-03-26 06:12:47.385084029 +0000 UTC m=+0.167968263 container create 7e242629978a455360fddaf2f813bf3ffc4352b4f07663664f9fc9e9c728e0db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5) 2024-03-26T06:12:49.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 podman[31955]: 2024-03-26 06:12:49.442869236 +0000 UTC m=+2.225753442 container init 7e242629978a455360fddaf2f813bf3ffc4352b4f07663664f9fc9e9c728e0db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 podman[31955]: 2024-03-26 06:12:49.446976918 +0000 UTC m=+2.229861111 container start 7e242629978a455360fddaf2f813bf3ffc4352b4f07663664f9fc9e9c728e0db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: ceph version 19.0.0-1964-g7e98f5f6 (7e98f5f6b535defc7fa715b456904ffc0500aa9f) squid (dev), process ceph-mon, pid 2 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: pidfile_write: ignore empty --pid-file 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: load: jerasure load: lrc 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: RocksDB version: 7.9.2 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Git sha 0 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Compile date 2024-03-23 10:35:25 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: DB SUMMARY 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: DB Session ID: 8E66SLL89RIQT5LNP075 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: CURRENT file: CURRENT 2024-03-26T06:12:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100512 ; 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.error_if_exists: 0 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.create_if_missing: 0 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.paranoid_checks: 1 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.env: 0x562e40b83b20 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T06:12:49.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.info_log: 0x562e4181d8e0 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.statistics: (nil) 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.use_fsync: 0 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_log_file_size: 0 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.allow_fallocate: 1 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.use_direct_reads: 0 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T06:12:49.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.db_log_dir: 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.wal_dir: 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.write_buffer_manager: 0x562e417f1400 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T06:12:49.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.unordered_write: 0 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.row_cache: None 2024-03-26T06:12:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.wal_filter: None 2024-03-26T06:12:49.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T06:12:49.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T06:12:49.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.two_write_queues: 0 2024-03-26T06:12:49.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T06:12:49.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.wal_compression: 0 2024-03-26T06:12:49.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.atomic_flush: 0 2024-03-26T06:12:49.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T06:12:49.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T06:12:49.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T06:12:49.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.log_readahead_size: 0 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_background_jobs: 2 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_background_compactions: -1 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_subcompactions: 1 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T06:12:49.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_open_files: -1 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_background_flushes: -1 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Compression algorithms supported: 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T06:12:49.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: kZSTD supported: 0 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: kXpressCompression supported: 0 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: kZlibCompression supported: 1 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: kSnappyCompression supported: 1 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: kLZ4Compression supported: 1 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: kBZip2Compression supported: 0 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.merge_operator: 2024-03-26T06:12:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_filter: None 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562e4181d9e0) 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_type: 0 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_index_type: 0 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_shortening: 1 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T06:12:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout: checksum: 4 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: no_block_cache: 0 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache: 0x562e41823090 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_options: 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: capacity : 536870912 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_shard_bits : 4 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: strict_capacity_limit : 0 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_compressed: (nil) 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: persistent_cache: (nil) 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size: 4096 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size_deviation: 10 2024-03-26T06:12:49.709 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_restart_interval: 16 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_block_restart_interval: 1 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: metadata_block_size: 4096 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: partition_filters: 0 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: use_delta_encoding: 1 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: filter_policy: bloomfilter 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: whole_key_filtering: 1 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: verify_compression: 0 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: format_version: 5 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_index_compression: 1 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_align: 0 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: max_auto_readahead_size: 262144 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: prepopulate_block_cache: 0 2024-03-26T06:12:49.710 INFO:journalctl@ceph.mon.a.smithi037.stdout: initial_auto_readahead_size: 8192 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compression: NoCompression 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.num_levels: 7 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T06:12:49.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T06:12:49.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T06:12:49.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T06:12:49.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.table_properties_collectors: 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.inplace_update_support: 0 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T06:12:49.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.bloom_locality: 0 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.max_successive_merges: 0 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.ttl: 2592000 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.enable_blob_files: false 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.min_blob_size: 0 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T06:12:49.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2786f55d-0e68-4efd-a497-14650cf882a6 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711433569480573, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711433569481731, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95573, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 93563, "index_size": 292, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13989, "raw_average_key_size": 50, "raw_value_size": 86004, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711433569, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2786f55d-0e68-4efd-a497-14650cf882a6", "db_session_id": "8E66SLL89RIQT5LNP075", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-26T06:12:49.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711433569481835, "job": 1, "event": "recovery_finished"} 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562e41916000 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: DB pointer 0x562e41900000 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** DB Stats ** 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T06:12:49.718 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: L0 2/0 95.17 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: Sum 2/0 95.17 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 118.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T06:12:49.719 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative compaction: 0.00 GB write, 25.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval compaction: 0.00 GB write, 25.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache BinnedLRUCache@0x562e41823090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-26T06:12:49.720 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: starting mon.a rank 0 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: mon.a@-1(???) e1 preinit fsid a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: mon.a@-1(???).mds e1 new map 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: mon.a@-1(???).mds e1 print_map 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout: e1 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout: legacy client fscid: -1 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout: No filesystems configured 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T06:12:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T06:12:49.722 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T06:12:49.722 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T06:12:49.722 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-26T06:12:49.722 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T06:12:49.722 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: monmap e1: 1 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T06:12:49.722 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: fsmap 2024-03-26T06:12:49.722 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: osdmap e1: 0 total, 0 up, 0 in 2024-03-26T06:12:49.722 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:49 smithi037 ceph-mon[31974]: mgrmap e1: no daemons active 2024-03-26T06:12:50.335 INFO:teuthology.orchestra.run.smithi037.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-26T06:12:50.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:50 smithi037 bash[31955]: 7e242629978a455360fddaf2f813bf3ffc4352b4f07663664f9fc9e9c728e0db 2024-03-26T06:12:50.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:50 smithi037 systemd[1]: Started Ceph mon.a for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:12:54.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:54 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3682229288' entity='client.admin' 2024-03-26T06:12:56.540 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.384+0000 7f83f9e49700 1 Processor -- start 2024-03-26T06:12:56.540 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.384+0000 7f83f9e49700 1 -- start start 2024-03-26T06:12:56.540 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.384+0000 7f83f9e49700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f4106bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:12:56.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.384+0000 7f83f9e49700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41071a0 con 0x7f83f41067b0 2024-03-26T06:12:56.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.384+0000 7f83f37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f4106bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:12:56.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.384+0000 7f83f37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f4106bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34944/0 (socket says 172.21.15.37:34944) 2024-03-26T06:12:56.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.384+0000 7f83f37fe700 1 -- 172.21.15.37:0/2264637172 learned_addr learned my addr 172.21.15.37:0/2264637172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:12:56.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.385+0000 7f83f37fe700 1 -- 172.21.15.37:0/2264637172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f41079b0 con 0x7f83f41067b0 2024-03-26T06:12:56.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.385+0000 7f83f37fe700 1 --2- 172.21.15.37:0/2264637172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f4106bd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f83e40098a0 tx=0x7f83e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87e6cc26635b1bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:12:56.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.385+0000 7f83f27fc700 1 -- 172.21.15.37:0/2264637172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83e4004160 con 0x7f83f41067b0 2024-03-26T06:12:56.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.385+0000 7f83f27fc700 1 -- 172.21.15.37:0/2264637172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83e4004300 con 0x7f83f41067b0 2024-03-26T06:12:56.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.385+0000 7f83f27fc700 1 -- 172.21.15.37:0/2264637172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83e4011b50 con 0x7f83f41067b0 2024-03-26T06:12:56.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.386+0000 7f83f9e49700 1 -- 172.21.15.37:0/2264637172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 msgr2=0x7f83f4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:12:56.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.386+0000 7f83f9e49700 1 --2- 172.21.15.37:0/2264637172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f4106bd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f83e40098a0 tx=0x7f83e4009bb0 comp rx=0 tx=0).stop 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.386+0000 7f83f9e49700 1 -- 172.21.15.37:0/2264637172 shutdown_connections 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.386+0000 7f83f9e49700 1 --2- 172.21.15.37:0/2264637172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f4106bd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.386+0000 7f83f9e49700 1 -- 172.21.15.37:0/2264637172 >> 172.21.15.37:0/2264637172 conn(0x7f83f4101df0 msgr2=0x7f83f4104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.386+0000 7f83f9e49700 1 -- 172.21.15.37:0/2264637172 shutdown_connections 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.386+0000 7f83f9e49700 1 -- 172.21.15.37:0/2264637172 wait complete. 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.387+0000 7f83f9e49700 1 Processor -- start 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.387+0000 7f83f9e49700 1 -- start start 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.387+0000 7f83f9e49700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f407aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.387+0000 7f83f9e49700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f410ad60 con 0x7f83f41067b0 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.387+0000 7f83f37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f407aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.387+0000 7f83f37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f407aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34948/0 (socket says 172.21.15.37:34948) 2024-03-26T06:12:56.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.387+0000 7f83f37fe700 1 -- 172.21.15.37:0/3682229288 learned_addr learned my addr 172.21.15.37:0/3682229288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.387+0000 7f83f37fe700 1 -- 172.21.15.37:0/3682229288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f419e620 con 0x7f83f41067b0 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.388+0000 7f83f37fe700 1 --2- 172.21.15.37:0/3682229288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f407aa30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009fb0 tx=0x7f83e4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.388+0000 7f83f0ff9700 1 -- 172.21.15.37:0/3682229288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83e40038b0 con 0x7f83f41067b0 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.388+0000 7f83f0ff9700 1 -- 172.21.15.37:0/3682229288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83e4011e50 con 0x7f83f41067b0 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.388+0000 7f83f9e49700 1 -- 172.21.15.37:0/3682229288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f419e7f0 con 0x7f83f41067b0 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.388+0000 7f83f0ff9700 1 -- 172.21.15.37:0/3682229288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83e4019550 con 0x7f83f41067b0 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.388+0000 7f83f9e49700 1 -- 172.21.15.37:0/3682229288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f419eb90 con 0x7f83f41067b0 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.388+0000 7f83f0ff9700 1 -- 172.21.15.37:0/3682229288 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f83e4019b90 con 0x7f83f41067b0 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.388+0000 7f83f0ff9700 1 -- 172.21.15.37:0/3682229288 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f83e4024730 con 0x7f83f41067b0 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.389+0000 7f83f9e49700 1 -- 172.21.15.37:0/3682229288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f4063410 con 0x7f83f41067b0 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.391+0000 7f83f0ff9700 1 -- 172.21.15.37:0/3682229288 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f83e4003a50 con 0x7f83f41067b0 2024-03-26T06:12:56.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.462+0000 7f83f9e49700 1 -- 172.21.15.37:0/3682229288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f83f419eff0 con 0x7f83f41067b0 2024-03-26T06:12:56.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.464+0000 7f83f0ff9700 1 -- 172.21.15.37:0/3682229288 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f83e4024930 con 0x7f83f41067b0 2024-03-26T06:12:56.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.464+0000 7f83f9e49700 1 -- 172.21.15.37:0/3682229288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 msgr2=0x7f83f407aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:12:56.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.464+0000 7f83f9e49700 1 --2- 172.21.15.37:0/3682229288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f407aa30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009fb0 tx=0x7f83e4011bf0 comp rx=0 tx=0).stop 2024-03-26T06:12:56.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.464+0000 7f83f9e49700 1 -- 172.21.15.37:0/3682229288 shutdown_connections 2024-03-26T06:12:56.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.464+0000 7f83f9e49700 1 --2- 172.21.15.37:0/3682229288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83f41067b0 0x7f83f407aa30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:12:56.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.464+0000 7f83f9e49700 1 -- 172.21.15.37:0/3682229288 >> 172.21.15.37:0/3682229288 conn(0x7f83f4101df0 msgr2=0x7f83f4102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:12:56.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.465+0000 7f83f9e49700 1 -- 172.21.15.37:0/3682229288 shutdown_connections 2024-03-26T06:12:56.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:53.465+0000 7f83f9e49700 1 -- 172.21.15.37:0/3682229288 wait complete. 2024-03-26T06:12:56.544 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-26T06:12:56.557 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-26T06:12:56.557 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mgr... 2024-03-26T06:12:56.557 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-26T06:12:56.558 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-26T06:12:56.802 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.x 2024-03-26T06:12:56.802 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.x.service: Unit ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.x.service not loaded. 2024-03-26T06:12:56.959 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a86fb218-eb37-11ee-95d0-87774f69a715.target.wants/ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-a86fb218-eb37-11ee-95d0-87774f69a715@.service. 2024-03-26T06:12:57.467 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T06:12:57.467 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-26T06:12:57.474 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T06:12:57.475 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-26T06:12:57.475 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-26T06:12:57.475 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T06:12:57.483 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T06:12:57.484 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-26T06:12:57.491 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T06:12:57.491 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-26T06:12:57.491 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-26T06:12:57.491 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-26T06:12:57.492 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr to start... 2024-03-26T06:12:57.492 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr... 2024-03-26T06:12:57.991 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:57 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:57.947+0000 7f470ec60200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T06:12:58.301 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:12:58 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1721913025' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T06:12:58.301 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:58.047+0000 7f470ec60200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T06:12:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:12:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-26T06:12:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:12:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T06:12:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T06:12:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T06:12:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T06:12:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:12:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T06:12:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:12:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T06:12:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T06:12:58.567 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T06:12:42.888717+0000", 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.907+0000 7ffbb0a4b700 1 Processor -- start 2024-03-26T06:12:58.568 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.907+0000 7ffbb0a4b700 1 -- start start 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.907+0000 7ffbb0a4b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac10a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.908+0000 7ffbb0a4b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac07ae50 con 0x7ffbac108360 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.908+0000 7ffbaa59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac10a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.908+0000 7ffbaa59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac10a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60012/0 (socket says 172.21.15.37:60012) 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.908+0000 7ffbaa59c700 1 -- 172.21.15.37:0/396472266 learned_addr learned my addr 172.21.15.37:0/396472266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.908+0000 7ffbaa59c700 1 -- 172.21.15.37:0/396472266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac10ac90 con 0x7ffbac108360 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.908+0000 7ffbaa59c700 1 --2- 172.21.15.37:0/396472266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac10a750 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0098a0 tx=0x7ffb9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830d54d834476068 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.908+0000 7ffba959a700 1 -- 172.21.15.37:0/396472266 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb9c004160 con 0x7ffbac108360 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.909+0000 7ffba959a700 1 -- 172.21.15.37:0/396472266 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb9c004300 con 0x7ffbac108360 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.909+0000 7ffba959a700 1 -- 172.21.15.37:0/396472266 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb9c011b50 con 0x7ffbac108360 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.909+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/396472266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 msgr2=0x7ffbac10a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.909+0000 7ffbb0a4b700 1 --2- 172.21.15.37:0/396472266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac10a750 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0098a0 tx=0x7ffb9c009bb0 comp rx=0 tx=0).stop 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.909+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/396472266 shutdown_connections 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.909+0000 7ffbb0a4b700 1 --2- 172.21.15.37:0/396472266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac10a750 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:12:58.569 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.909+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/396472266 >> 172.21.15.37:0/396472266 conn(0x7ffbac101e10 msgr2=0x7ffbac104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.909+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/396472266 shutdown_connections 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.909+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/396472266 wait complete. 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.910+0000 7ffbb0a4b700 1 Processor -- start 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.910+0000 7ffbb0a4b700 1 -- start start 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.910+0000 7ffbb0a4b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.910+0000 7ffbb0a4b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac10bcb0 con 0x7ffbac108360 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.910+0000 7ffbaa59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.910+0000 7ffbaa59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac19b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60018/0 (socket says 172.21.15.37:60018) 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.910+0000 7ffbaa59c700 1 -- 172.21.15.37:0/1721913025 learned_addr learned my addr 172.21.15.37:0/1721913025 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.911+0000 7ffbaa59c700 1 -- 172.21.15.37:0/1721913025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac19bda0 con 0x7ffbac108360 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.911+0000 7ffbaa59c700 1 --2- 172.21.15.37:0/1721913025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac19b860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009fb0 tx=0x7ffb9c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.911+0000 7ffb9b7fe700 1 -- 172.21.15.37:0/1721913025 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb9c0038b0 con 0x7ffbac108360 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.911+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/1721913025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac19c030 con 0x7ffbac108360 2024-03-26T06:12:58.570 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.911+0000 7ffb9b7fe700 1 -- 172.21.15.37:0/1721913025 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb9c011e50 con 0x7ffbac108360 2024-03-26T06:12:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.911+0000 7ffb9b7fe700 1 -- 172.21.15.37:0/1721913025 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb9c019420 con 0x7ffbac108360 2024-03-26T06:12:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.911+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/1721913025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac19ed20 con 0x7ffbac108360 2024-03-26T06:12:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.911+0000 7ffb9b7fe700 1 -- 172.21.15.37:0/1721913025 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ffb9c019a80 con 0x7ffbac108360 2024-03-26T06:12:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.911+0000 7ffb9b7fe700 1 -- 172.21.15.37:0/1721913025 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffb9c024730 con 0x7ffbac108360 2024-03-26T06:12:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.912+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/1721913025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac068ec0 con 0x7ffbac108360 2024-03-26T06:12:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.916+0000 7ffb9b7fe700 1 -- 172.21.15.37:0/1721913025 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7ffb9c043e50 con 0x7ffbac108360 2024-03-26T06:12:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.990+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/1721913025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ffbac19f1c0 con 0x7ffbac108360 2024-03-26T06:12:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.991+0000 7ffb9b7fe700 1 -- 172.21.15.37:0/1721913025 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ffb9c032030 con 0x7ffbac108360 2024-03-26T06:12:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.992+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/1721913025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 msgr2=0x7ffbac19b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:12:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.992+0000 7ffbb0a4b700 1 --2- 172.21.15.37:0/1721913025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac19b860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009fb0 tx=0x7ffb9c011bf0 comp rx=0 tx=0).stop 2024-03-26T06:12:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.992+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/1721913025 shutdown_connections 2024-03-26T06:12:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.992+0000 7ffbb0a4b700 1 --2- 172.21.15.37:0/1721913025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbac108360 0x7ffbac19b860 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:12:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.992+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/1721913025 >> 172.21.15.37:0/1721913025 conn(0x7ffbac101e10 msgr2=0x7ffbac102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:12:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.992+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/1721913025 shutdown_connections 2024-03-26T06:12:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:12:57.992+0000 7ffbb0a4b700 1 -- 172.21.15.37:0/1721913025 wait complete. 2024-03-26T06:12:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (1/15)... 2024-03-26T06:12:58.812 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:58.717+0000 7f470ec60200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T06:12:58.812 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:58.812+0000 7f470ec60200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T06:12:59.137 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:58.955+0000 7f470ec60200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T06:12:59.137 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:59 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:59.053+0000 7f470ec60200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T06:12:59.137 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:59 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:59.137+0000 7f470ec60200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T06:12:59.697 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:59 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:59.405+0000 7f470ec60200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T06:12:59.697 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:59 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:59.516+0000 7f470ec60200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T06:12:59.697 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:59 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:59.604+0000 7f470ec60200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T06:13:00.197 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:59 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:59.700+0000 7f470ec60200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T06:13:00.197 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:12:59 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:12:59.878+0000 7f470ec60200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T06:13:00.947 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:00.879+0000 7f470ec60200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T06:13:01.364 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:01 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3779287737' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T06:13:01.364 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:01 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:01.018+0000 7f470ec60200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T06:13:01.657 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:01 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:01.360+0000 7f470ec60200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T06:13:01.657 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:01 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:01.452+0000 7f470ec60200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T06:13:01.657 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:01 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:01.657+0000 7f470ec60200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:13:01.947 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:01 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:01.847+0000 7f470ec60200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T06:13:02.403 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:02 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:02.140+0000 7f470ec60200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:13:02.697 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:02 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:02.403+0000 7f470ec60200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T06:13:02.697 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:02 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:02.648+0000 7f470ec60200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T06:13:02.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:13:02.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-26T06:13:02.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:13:02.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T06:13:02.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T06:13:02.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:02.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T06:13:02.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T06:13:02.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T06:12:42.888717+0000", 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T06:13:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.994+0000 7f30cec43700 1 Processor -- start 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.994+0000 7f30cec43700 1 -- start start 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.994+0000 7f30cec43700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c810a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.994+0000 7f30cec43700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c807add0 con 0x7f30c8108370 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.995+0000 7f30cc9df700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c810a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.995+0000 7f30cc9df700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c810a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60034/0 (socket says 172.21.15.37:60034) 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.995+0000 7f30cc9df700 1 -- 172.21.15.37:0/2449795136 learned_addr learned my addr 172.21.15.37:0/2449795136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.995+0000 7f30cc9df700 1 -- 172.21.15.37:0/2449795136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c810aca0 con 0x7f30c8108370 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.995+0000 7f30cc9df700 1 --2- 172.21.15.37:0/2449795136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c810a760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0098a0 tx=0x7f30bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684353e81767ac41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.995+0000 7f30c77fe700 1 -- 172.21.15.37:0/2449795136 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30bc004160 con 0x7f30c8108370 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.995+0000 7f30c77fe700 1 -- 172.21.15.37:0/2449795136 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30bc004300 con 0x7f30c8108370 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.995+0000 7f30c77fe700 1 -- 172.21.15.37:0/2449795136 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30bc011b50 con 0x7f30c8108370 2024-03-26T06:13:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.996+0000 7f30cec43700 1 -- 172.21.15.37:0/2449795136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 msgr2=0x7f30c810a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.996+0000 7f30cec43700 1 --2- 172.21.15.37:0/2449795136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c810a760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0098a0 tx=0x7f30bc009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.996+0000 7f30cec43700 1 -- 172.21.15.37:0/2449795136 shutdown_connections 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.996+0000 7f30cec43700 1 --2- 172.21.15.37:0/2449795136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c810a760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.996+0000 7f30cec43700 1 -- 172.21.15.37:0/2449795136 >> 172.21.15.37:0/2449795136 conn(0x7f30c8101da0 msgr2=0x7f30c81041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.996+0000 7f30cec43700 1 -- 172.21.15.37:0/2449795136 shutdown_connections 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.996+0000 7f30cec43700 1 -- 172.21.15.37:0/2449795136 wait complete. 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.997+0000 7f30cec43700 1 Processor -- start 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.997+0000 7f30cec43700 1 -- start start 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.997+0000 7f30cec43700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.997+0000 7f30cec43700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c810bc10 con 0x7f30c8108370 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.997+0000 7f30cc9df700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.997+0000 7f30cc9df700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60050/0 (socket says 172.21.15.37:60050) 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.997+0000 7f30cc9df700 1 -- 172.21.15.37:0/3779287737 learned_addr learned my addr 172.21.15.37:0/3779287737 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:02.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.997+0000 7f30cc9df700 1 -- 172.21.15.37:0/3779287737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c819bcf0 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.997+0000 7f30cc9df700 1 --2- 172.21.15.37:0/3779287737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c819b7b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f30bc011b60 tx=0x7f30bc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.998+0000 7f30c5ffb700 1 -- 172.21.15.37:0/3779287737 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30bc003970 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.998+0000 7f30c5ffb700 1 -- 172.21.15.37:0/3779287737 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30bc003b10 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.998+0000 7f30cec43700 1 -- 172.21.15.37:0/3779287737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c819bf80 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.998+0000 7f30c5ffb700 1 -- 172.21.15.37:0/3779287737 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30bc024610 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.998+0000 7f30cec43700 1 -- 172.21.15.37:0/3779287737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c819ec70 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.998+0000 7f30c5ffb700 1 -- 172.21.15.37:0/3779287737 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f30bc024ca0 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.998+0000 7f30c5ffb700 1 -- 172.21.15.37:0/3779287737 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f30bc011e00 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:00.999+0000 7f30cec43700 1 -- 172.21.15.37:0/3779287737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c8068ec0 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:01.001+0000 7f30c5ffb700 1 -- 172.21.15.37:0/3779287737 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f30bc01a9f0 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:01.075+0000 7f30cec43700 1 -- 172.21.15.37:0/3779287737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f30c819ee40 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:01.075+0000 7f30c5ffb700 1 -- 172.21.15.37:0/3779287737 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f30bc01a430 con 0x7f30c8108370 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:01.076+0000 7f30cec43700 1 -- 172.21.15.37:0/3779287737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 msgr2=0x7f30c819b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:02.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:01.076+0000 7f30cec43700 1 --2- 172.21.15.37:0/3779287737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c819b7b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f30bc011b60 tx=0x7f30bc011c40 comp rx=0 tx=0).stop 2024-03-26T06:13:02.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:01.076+0000 7f30cec43700 1 -- 172.21.15.37:0/3779287737 shutdown_connections 2024-03-26T06:13:02.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:01.076+0000 7f30cec43700 1 --2- 172.21.15.37:0/3779287737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8108370 0x7f30c819b7b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:02.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:01.076+0000 7f30cec43700 1 -- 172.21.15.37:0/3779287737 >> 172.21.15.37:0/3779287737 conn(0x7f30c8101da0 msgr2=0x7f30c8102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:02.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:01.076+0000 7f30cec43700 1 -- 172.21.15.37:0/3779287737 shutdown_connections 2024-03-26T06:13:02.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:01.076+0000 7f30cec43700 1 -- 172.21.15.37:0/3779287737 wait complete. 2024-03-26T06:13:02.885 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (2/15)... 2024-03-26T06:13:03.681 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:03 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:03.488+0000 7f470ec60200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T06:13:04.405 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:04.082+0000 7f470ec60200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T06:13:04.406 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:04.183+0000 7f470ec60200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T06:13:04.406 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:04.405+0000 7f470ec60200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T06:13:04.663 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: Activating manager daemon x 2024-03-26T06:13:04.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: mgrmap e2: x(active, starting, since 0.00360632s) 2024-03-26T06:13:04.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: from='mgr.14100 172.21.15.37:0/1105225062' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T06:13:04.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: from='mgr.14100 172.21.15.37:0/1105225062' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T06:13:04.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: from='mgr.14100 172.21.15.37:0/1105225062' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T06:13:04.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: from='mgr.14100 172.21.15.37:0/1105225062' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:13:04.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: from='mgr.14100 172.21.15.37:0/1105225062' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T06:13:04.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: Manager daemon x is now available 2024-03-26T06:13:04.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: from='mgr.14100 172.21.15.37:0/1105225062' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T06:13:04.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: from='mgr.14100 172.21.15.37:0/1105225062' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T06:13:04.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: from='mgr.14100 172.21.15.37:0/1105225062' entity='mgr.x' 2024-03-26T06:13:04.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: from='mgr.14100 172.21.15.37:0/1105225062' entity='mgr.x' 2024-03-26T06:13:04.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:04 smithi037 ceph-mon[31974]: from='mgr.14100 172.21.15.37:0/1105225062' entity='mgr.x' 2024-03-26T06:13:05.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:05 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3886166414' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T06:13:05.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:05 smithi037 ceph-mon[31974]: mgrmap e3: x(active, since 1.00678s) 2024-03-26T06:13:06.043 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:13:06.043 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-26T06:13:06.043 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:13:06.043 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T06:13:06.043 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:06.044 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T06:13:06.045 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T06:13:06.045 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:06.045 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T06:13:06.045 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:06.045 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T06:13:06.045 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T06:13:06.045 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T06:13:06.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T06:13:06.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T06:13:06.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T06:13:06.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:06.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T06:13:06.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T06:13:06.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T06:13:06.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T06:13:06.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T06:13:06.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T06:13:06.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T06:13:06.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T06:12:42.888717+0000", 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.339+0000 7f7d49731700 1 Processor -- start 2024-03-26T06:13:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.339+0000 7f7d49731700 1 -- start start 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.340+0000 7f7d49731700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d44107d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.340+0000 7f7d49731700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d441082e0 con 0x7f7d441078f0 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.340+0000 7f7d42ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d44107d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.340+0000 7f7d42ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d44107d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60130/0 (socket says 172.21.15.37:60130) 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.340+0000 7f7d42ffd700 1 -- 172.21.15.37:0/1291961607 learned_addr learned my addr 172.21.15.37:0/1291961607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.340+0000 7f7d42ffd700 1 -- 172.21.15.37:0/1291961607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d44108af0 con 0x7f7d441078f0 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.340+0000 7f7d42ffd700 1 --2- 172.21.15.37:0/1291961607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d44107d10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d340098a0 tx=0x7f7d34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff7c75ae8a489ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.341+0000 7f7d41ffb700 1 -- 172.21.15.37:0/1291961607 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d34004160 con 0x7f7d441078f0 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.341+0000 7f7d41ffb700 1 -- 172.21.15.37:0/1291961607 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d34004300 con 0x7f7d441078f0 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.341+0000 7f7d41ffb700 1 -- 172.21.15.37:0/1291961607 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d34011b50 con 0x7f7d441078f0 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.341+0000 7f7d49731700 1 -- 172.21.15.37:0/1291961607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 msgr2=0x7f7d44107d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.341+0000 7f7d49731700 1 --2- 172.21.15.37:0/1291961607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d44107d10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d340098a0 tx=0x7f7d34009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.341+0000 7f7d49731700 1 -- 172.21.15.37:0/1291961607 shutdown_connections 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.341+0000 7f7d49731700 1 --2- 172.21.15.37:0/1291961607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d44107d10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.341+0000 7f7d49731700 1 -- 172.21.15.37:0/1291961607 >> 172.21.15.37:0/1291961607 conn(0x7f7d44102f70 msgr2=0x7f7d44105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.341+0000 7f7d49731700 1 -- 172.21.15.37:0/1291961607 shutdown_connections 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.341+0000 7f7d49731700 1 -- 172.21.15.37:0/1291961607 wait complete. 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.342+0000 7f7d49731700 1 Processor -- start 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.342+0000 7f7d49731700 1 -- start start 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.342+0000 7f7d49731700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d4407cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.342+0000 7f7d49731700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4410bea0 con 0x7f7d441078f0 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.342+0000 7f7d42ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d4407cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.342+0000 7f7d42ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d4407cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60138/0 (socket says 172.21.15.37:60138) 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.342+0000 7f7d42ffd700 1 -- 172.21.15.37:0/3886166414 learned_addr learned my addr 172.21.15.37:0/3886166414 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.343+0000 7f7d42ffd700 1 -- 172.21.15.37:0/3886166414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4407d090 con 0x7f7d441078f0 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.343+0000 7f7d42ffd700 1 --2- 172.21.15.37:0/3886166414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d4407cb50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009fb0 tx=0x7f7d34011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.343+0000 7f7d3bfff700 1 -- 172.21.15.37:0/3886166414 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d340038b0 con 0x7f7d441078f0 2024-03-26T06:13:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.343+0000 7f7d3bfff700 1 -- 172.21.15.37:0/3886166414 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d34011e50 con 0x7f7d441078f0 2024-03-26T06:13:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.343+0000 7f7d3bfff700 1 -- 172.21.15.37:0/3886166414 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d34019550 con 0x7f7d441078f0 2024-03-26T06:13:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.343+0000 7f7d49731700 1 -- 172.21.15.37:0/3886166414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4407b280 con 0x7f7d441078f0 2024-03-26T06:13:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.343+0000 7f7d49731700 1 -- 172.21.15.37:0/3886166414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4407b7c0 con 0x7f7d441078f0 2024-03-26T06:13:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.344+0000 7f7d3bfff700 1 -- 172.21.15.37:0/3886166414 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 2) v1 ==== 45862+0+0 (secure 0 0 0) 0x7f7d34019850 con 0x7f7d441078f0 2024-03-26T06:13:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.344+0000 7f7d3bfff700 1 -- 172.21.15.37:0/3886166414 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7d3404cdd0 con 0x7f7d441078f0 2024-03-26T06:13:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.344+0000 7f7d49731700 1 -- 172.21.15.37:0/3886166414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d44068ec0 con 0x7f7d441078f0 2024-03-26T06:13:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.346+0000 7f7d3bfff700 1 -- 172.21.15.37:0/3886166414 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f7d34043e50 con 0x7f7d441078f0 2024-03-26T06:13:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.419+0000 7f7d49731700 1 -- 172.21.15.37:0/3886166414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7d4407be80 con 0x7f7d441078f0 2024-03-26T06:13:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.420+0000 7f7d3bfff700 1 -- 172.21.15.37:0/3886166414 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f7d34019b40 con 0x7f7d441078f0 2024-03-26T06:13:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.420+0000 7f7d49731700 1 -- 172.21.15.37:0/3886166414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 msgr2=0x7f7d4407cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:06.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.420+0000 7f7d49731700 1 --2- 172.21.15.37:0/3886166414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d4407cb50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009fb0 tx=0x7f7d34011bf0 comp rx=0 tx=0).stop 2024-03-26T06:13:06.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.420+0000 7f7d49731700 1 -- 172.21.15.37:0/3886166414 shutdown_connections 2024-03-26T06:13:06.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.420+0000 7f7d49731700 1 --2- 172.21.15.37:0/3886166414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d441078f0 0x7f7d4407cb50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:06.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.420+0000 7f7d49731700 1 -- 172.21.15.37:0/3886166414 >> 172.21.15.37:0/3886166414 conn(0x7f7d44102f70 msgr2=0x7f7d44103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:06.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.421+0000 7f7d49731700 1 -- 172.21.15.37:0/3886166414 shutdown_connections 2024-03-26T06:13:06.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:05.421+0000 7f7d49731700 1 -- 172.21.15.37:0/3886166414 wait complete. 2024-03-26T06:13:06.059 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (3/15)... 2024-03-26T06:13:07.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:07 smithi037 ceph-mon[31974]: mgrmap e4: x(active, since 2s) 2024-03-26T06:13:09.183 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:08 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3915585639' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T06:13:09.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:13:09.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-26T06:13:09.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:13:09.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T06:13:09.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T06:13:09.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T06:13:09.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 19, 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:09.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T06:13:09.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:09.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T06:13:09.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T06:13:09.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T06:13:09.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T06:13:09.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T06:13:09.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-26T06:13:09.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T06:12:42.888717+0000", 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.474+0000 7f7c32c3f700 1 Processor -- start 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.474+0000 7f7c32c3f700 1 -- start start 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.474+0000 7f7c32c3f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c10a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.474+0000 7f7c32c3f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c07ae70 con 0x7f7c2c108360 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.474+0000 7f7c309db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c10a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:09.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.474+0000 7f7c309db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c10a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42352/0 (socket says 172.21.15.37:42352) 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.474+0000 7f7c309db700 1 -- 172.21.15.37:0/3816688879 learned_addr learned my addr 172.21.15.37:0/3816688879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.475+0000 7f7c309db700 1 -- 172.21.15.37:0/3816688879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c10ac90 con 0x7f7c2c108360 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.475+0000 7f7c309db700 1 --2- 172.21.15.37:0/3816688879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c10a750 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c200098a0 tx=0x7f7c20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686735ebfe6f0d8e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.475+0000 7f7c2b7fe700 1 -- 172.21.15.37:0/3816688879 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c20004160 con 0x7f7c2c108360 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.475+0000 7f7c2b7fe700 1 -- 172.21.15.37:0/3816688879 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c20004300 con 0x7f7c2c108360 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.475+0000 7f7c2b7fe700 1 -- 172.21.15.37:0/3816688879 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c20011b50 con 0x7f7c2c108360 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.476+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3816688879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 msgr2=0x7f7c2c10a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.476+0000 7f7c32c3f700 1 --2- 172.21.15.37:0/3816688879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c10a750 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c200098a0 tx=0x7f7c20009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.476+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3816688879 shutdown_connections 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.476+0000 7f7c32c3f700 1 --2- 172.21.15.37:0/3816688879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c10a750 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.476+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3816688879 >> 172.21.15.37:0/3816688879 conn(0x7f7c2c101e70 msgr2=0x7f7c2c104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.476+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3816688879 shutdown_connections 2024-03-26T06:13:09.413 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.476+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3816688879 wait complete. 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.476+0000 7f7c32c3f700 1 Processor -- start 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.476+0000 7f7c32c3f700 1 -- start start 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.477+0000 7f7c32c3f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c107be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.477+0000 7f7c32c3f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c10bc40 con 0x7f7c2c108360 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.477+0000 7f7c309db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c107be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.477+0000 7f7c309db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c107be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42354/0 (socket says 172.21.15.37:42354) 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.477+0000 7f7c309db700 1 -- 172.21.15.37:0/3915585639 learned_addr learned my addr 172.21.15.37:0/3915585639 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.477+0000 7f7c309db700 1 -- 172.21.15.37:0/3915585639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c108120 con 0x7f7c2c108360 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.477+0000 7f7c309db700 1 --2- 172.21.15.37:0/3915585639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c107be0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7c20011b60 tx=0x7f7c20011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.477+0000 7f7c29ffb700 1 -- 172.21.15.37:0/3915585639 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c20003970 con 0x7f7c2c108360 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.477+0000 7f7c29ffb700 1 -- 172.21.15.37:0/3915585639 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c20003b10 con 0x7f7c2c108360 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.477+0000 7f7c29ffb700 1 -- 172.21.15.37:0/3915585639 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c20024610 con 0x7f7c2c108360 2024-03-26T06:13:09.414 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.477+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3915585639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2c106310 con 0x7f7c2c108360 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.478+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3915585639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c106850 con 0x7f7c2c108360 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.478+0000 7f7c29ffb700 1 -- 172.21.15.37:0/3915585639 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f7c2001a9f0 con 0x7f7c2c108360 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.478+0000 7f7c29ffb700 1 --2- 172.21.15.37:0/3915585639 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f7c1c038ed0 0x7f7c1c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.478+0000 7f7c29ffb700 1 -- 172.21.15.37:0/3915585639 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7c20011e00 con 0x7f7c2c108360 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.478+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3915585639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c068ec0 con 0x7f7c2c108360 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.479+0000 7f7c2bfff700 1 --2- 172.21.15.37:0/3915585639 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f7c1c038ed0 0x7f7c1c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.479+0000 7f7c2bfff700 1 --2- 172.21.15.37:0/3915585639 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f7c1c038ed0 0x7f7c1c03b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006fd0 tx=0x7f7c18006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.484+0000 7f7c29ffb700 1 -- 172.21.15.37:0/3915585639 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7c2000b3e0 con 0x7f7c2c108360 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.768+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3915585639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7c2c106fe0 con 0x7f7c2c108360 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.769+0000 7f7c29ffb700 1 -- 172.21.15.37:0/3915585639 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f7c200248c0 con 0x7f7c2c108360 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.771+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3915585639 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f7c1c038ed0 msgr2=0x7f7c1c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.771+0000 7f7c32c3f700 1 --2- 172.21.15.37:0/3915585639 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f7c1c038ed0 0x7f7c1c03b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006fd0 tx=0x7f7c18006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:09.415 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.771+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3915585639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 msgr2=0x7f7c2c107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:09.416 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.771+0000 7f7c32c3f700 1 --2- 172.21.15.37:0/3915585639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c107be0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7c20011b60 tx=0x7f7c20011c40 comp rx=0 tx=0).stop 2024-03-26T06:13:09.416 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.771+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3915585639 shutdown_connections 2024-03-26T06:13:09.416 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.771+0000 7f7c32c3f700 1 --2- 172.21.15.37:0/3915585639 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f7c1c038ed0 0x7f7c1c03b390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:09.416 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.771+0000 7f7c32c3f700 1 --2- 172.21.15.37:0/3915585639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c2c108360 0x7f7c2c107be0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:09.416 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.771+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3915585639 >> 172.21.15.37:0/3915585639 conn(0x7f7c2c101e70 msgr2=0x7f7c2c1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:09.416 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.771+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3915585639 shutdown_connections 2024-03-26T06:13:09.416 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:08.771+0000 7f7c32c3f700 1 -- 172.21.15.37:0/3915585639 wait complete. 2024-03-26T06:13:09.416 INFO:teuthology.orchestra.run.smithi037.stdout:mgr is available 2024-03-26T06:13:10.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:10 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/458708656' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T06:13:10.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:13:10.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T06:13:10.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:13:10.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T06:13:10.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.802+0000 7fd16781b700 1 Processor -- start 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.803+0000 7fd16781b700 1 -- start start 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.803+0000 7fd16781b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16010a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.803+0000 7fd16781b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16007ae50 con 0x7fd160108360 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.803+0000 7fd1655b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16010a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.803+0000 7fd1655b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16010a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42370/0 (socket says 172.21.15.37:42370) 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.803+0000 7fd1655b7700 1 -- 172.21.15.37:0/2189220949 learned_addr learned my addr 172.21.15.37:0/2189220949 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.803+0000 7fd1655b7700 1 -- 172.21.15.37:0/2189220949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16010ac90 con 0x7fd160108360 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.803+0000 7fd1655b7700 1 --2- 172.21.15.37:0/2189220949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16010a750 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd1500098a0 tx=0x7fd150009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82257ee6cefd438 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.804+0000 7fd157fff700 1 -- 172.21.15.37:0/2189220949 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd150004160 con 0x7fd160108360 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.804+0000 7fd157fff700 1 -- 172.21.15.37:0/2189220949 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd150004300 con 0x7fd160108360 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.804+0000 7fd157fff700 1 -- 172.21.15.37:0/2189220949 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd150011b50 con 0x7fd160108360 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.804+0000 7fd16781b700 1 -- 172.21.15.37:0/2189220949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 msgr2=0x7fd16010a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:10.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.804+0000 7fd16781b700 1 --2- 172.21.15.37:0/2189220949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16010a750 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd1500098a0 tx=0x7fd150009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.804+0000 7fd16781b700 1 -- 172.21.15.37:0/2189220949 shutdown_connections 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.804+0000 7fd16781b700 1 --2- 172.21.15.37:0/2189220949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16010a750 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.804+0000 7fd16781b700 1 -- 172.21.15.37:0/2189220949 >> 172.21.15.37:0/2189220949 conn(0x7fd160101e10 msgr2=0x7fd160104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.804+0000 7fd16781b700 1 -- 172.21.15.37:0/2189220949 shutdown_connections 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.804+0000 7fd16781b700 1 -- 172.21.15.37:0/2189220949 wait complete. 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.805+0000 7fd16781b700 1 Processor -- start 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.805+0000 7fd16781b700 1 -- start start 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.805+0000 7fd16781b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.805+0000 7fd16781b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16010bcb0 con 0x7fd160108360 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.805+0000 7fd1655b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.805+0000 7fd1655b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16019b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42382/0 (socket says 172.21.15.37:42382) 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.805+0000 7fd1655b7700 1 -- 172.21.15.37:0/458708656 learned_addr learned my addr 172.21.15.37:0/458708656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:10.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.806+0000 7fd1655b7700 1 -- 172.21.15.37:0/458708656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16019bda0 con 0x7fd160108360 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.806+0000 7fd1655b7700 1 --2- 172.21.15.37:0/458708656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16019b860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd150009fb0 tx=0x7fd150011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.806+0000 7fd1567fc700 1 -- 172.21.15.37:0/458708656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1500038b0 con 0x7fd160108360 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.806+0000 7fd1567fc700 1 -- 172.21.15.37:0/458708656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd150011e50 con 0x7fd160108360 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.806+0000 7fd1567fc700 1 -- 172.21.15.37:0/458708656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd150019420 con 0x7fd160108360 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.806+0000 7fd16781b700 1 -- 172.21.15.37:0/458708656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16019c030 con 0x7fd160108360 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.806+0000 7fd16781b700 1 -- 172.21.15.37:0/458708656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16019ed20 con 0x7fd160108360 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.807+0000 7fd1567fc700 1 -- 172.21.15.37:0/458708656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7fd1500196d0 con 0x7fd160108360 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.807+0000 7fd1567fc700 1 --2- 172.21.15.37:0/458708656 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fd14c038ed0 0x7fd14c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.807+0000 7fd1567fc700 1 -- 172.21.15.37:0/458708656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd15004cfa0 con 0x7fd160108360 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.807+0000 7fd164db6700 1 --2- 172.21.15.37:0/458708656 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fd14c038ed0 0x7fd14c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.807+0000 7fd16781b700 1 -- 172.21.15.37:0/458708656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd160068ec0 con 0x7fd160108360 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.807+0000 7fd164db6700 1 --2- 172.21.15.37:0/458708656 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fd14c038ed0 0x7fd14c03b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd15c006fd0 tx=0x7fd15c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:09.812+0000 7fd1567fc700 1 -- 172.21.15.37:0/458708656 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd150003a50 con 0x7fd160108360 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.017+0000 7fd16781b700 1 -- 172.21.15.37:0/458708656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd16019eef0 con 0x7fd160108360 2024-03-26T06:13:10.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.018+0000 7fd1567fc700 1 -- 172.21.15.37:0/458708656 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd150032020 con 0x7fd160108360 2024-03-26T06:13:10.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.020+0000 7fd16781b700 1 -- 172.21.15.37:0/458708656 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fd14c038ed0 msgr2=0x7fd14c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:10.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.020+0000 7fd16781b700 1 --2- 172.21.15.37:0/458708656 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fd14c038ed0 0x7fd14c03b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd15c006fd0 tx=0x7fd15c006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:10.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.020+0000 7fd16781b700 1 -- 172.21.15.37:0/458708656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 msgr2=0x7fd16019b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:10.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.020+0000 7fd16781b700 1 --2- 172.21.15.37:0/458708656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16019b860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd150009fb0 tx=0x7fd150011bf0 comp rx=0 tx=0).stop 2024-03-26T06:13:10.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.020+0000 7fd16781b700 1 -- 172.21.15.37:0/458708656 shutdown_connections 2024-03-26T06:13:10.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.020+0000 7fd16781b700 1 --2- 172.21.15.37:0/458708656 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fd14c038ed0 0x7fd14c03b390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:10.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.020+0000 7fd16781b700 1 --2- 172.21.15.37:0/458708656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd160108360 0x7fd16019b860 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:10.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.020+0000 7fd16781b700 1 -- 172.21.15.37:0/458708656 >> 172.21.15.37:0/458708656 conn(0x7fd160101e10 msgr2=0x7fd160102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:10.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.021+0000 7fd16781b700 1 -- 172.21.15.37:0/458708656 shutdown_connections 2024-03-26T06:13:10.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:10.021+0000 7fd16781b700 1 -- 172.21.15.37:0/458708656 wait complete. 2024-03-26T06:13:10.832 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling cephadm module... 2024-03-26T06:13:11.832 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:11 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2531479212' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-26T06:13:11.832 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:11 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: ignoring --setuser ceph since I am not root 2024-03-26T06:13:11.832 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:11 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: ignoring --setgroup ceph since I am not root 2024-03-26T06:13:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.205+0000 7f1f6e251700 1 Processor -- start 2024-03-26T06:13:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.205+0000 7f1f6e251700 1 -- start start 2024-03-26T06:13:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.206+0000 7f1f6e251700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6810a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.206+0000 7f1f6e251700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6807ae70 con 0x7f1f681082d0 2024-03-26T06:13:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.206+0000 7f1f677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6810a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.206+0000 7f1f677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6810a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42394/0 (socket says 172.21.15.37:42394) 2024-03-26T06:13:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.206+0000 7f1f677fe700 1 -- 172.21.15.37:0/2169925750 learned_addr learned my addr 172.21.15.37:0/2169925750 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.206+0000 7f1f677fe700 1 -- 172.21.15.37:0/2169925750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6810ac00 con 0x7f1f681082d0 2024-03-26T06:13:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.206+0000 7f1f677fe700 1 --2- 172.21.15.37:0/2169925750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6810a6c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f580098a0 tx=0x7f1f58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ad556240cc6ee3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.207+0000 7f1f667fc700 1 -- 172.21.15.37:0/2169925750 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f58004160 con 0x7f1f681082d0 2024-03-26T06:13:12.056 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.207+0000 7f1f667fc700 1 -- 172.21.15.37:0/2169925750 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f58004300 con 0x7f1f681082d0 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.207+0000 7f1f667fc700 1 -- 172.21.15.37:0/2169925750 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f58011b50 con 0x7f1f681082d0 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.207+0000 7f1f6e251700 1 -- 172.21.15.37:0/2169925750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 msgr2=0x7f1f6810a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.207+0000 7f1f6e251700 1 --2- 172.21.15.37:0/2169925750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6810a6c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f580098a0 tx=0x7f1f58009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.207+0000 7f1f6e251700 1 -- 172.21.15.37:0/2169925750 shutdown_connections 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.207+0000 7f1f6e251700 1 --2- 172.21.15.37:0/2169925750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6810a6c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.207+0000 7f1f6e251700 1 -- 172.21.15.37:0/2169925750 >> 172.21.15.37:0/2169925750 conn(0x7f1f68101de0 msgr2=0x7f1f68104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.207+0000 7f1f6e251700 1 -- 172.21.15.37:0/2169925750 shutdown_connections 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.207+0000 7f1f6e251700 1 -- 172.21.15.37:0/2169925750 wait complete. 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.208+0000 7f1f6e251700 1 Processor -- start 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.208+0000 7f1f6e251700 1 -- start start 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.208+0000 7f1f6e251700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.208+0000 7f1f6e251700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6810bbb0 con 0x7f1f681082d0 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.208+0000 7f1f677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:12.057 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.208+0000 7f1f677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42396/0 (socket says 172.21.15.37:42396) 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.208+0000 7f1f677fe700 1 -- 172.21.15.37:0/2531479212 learned_addr learned my addr 172.21.15.37:0/2531479212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.209+0000 7f1f677fe700 1 -- 172.21.15.37:0/2531479212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6819bcb0 con 0x7f1f681082d0 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.209+0000 7f1f677fe700 1 --2- 172.21.15.37:0/2531479212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6819b770 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009fb0 tx=0x7f1f58011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.209+0000 7f1f64ff9700 1 -- 172.21.15.37:0/2531479212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f580038b0 con 0x7f1f681082d0 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.209+0000 7f1f64ff9700 1 -- 172.21.15.37:0/2531479212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f58011e50 con 0x7f1f681082d0 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.209+0000 7f1f6e251700 1 -- 172.21.15.37:0/2531479212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6819bfa0 con 0x7f1f681082d0 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.209+0000 7f1f64ff9700 1 -- 172.21.15.37:0/2531479212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f580193c0 con 0x7f1f681082d0 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.209+0000 7f1f6e251700 1 -- 172.21.15.37:0/2531479212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6819ec30 con 0x7f1f681082d0 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.210+0000 7f1f64ff9700 1 -- 172.21.15.37:0/2531479212 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f1f58019670 con 0x7f1f681082d0 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.210+0000 7f1f64ff9700 1 --2- 172.21.15.37:0/2531479212 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f1f50038ed0 0x7f1f5003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.210+0000 7f1f6e251700 1 -- 172.21.15.37:0/2531479212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f68063410 con 0x7f1f681082d0 2024-03-26T06:13:12.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.210+0000 7f1f64ff9700 1 -- 172.21.15.37:0/2531479212 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1f5804d1b0 con 0x7f1f681082d0 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.210+0000 7f1f66ffd700 1 --2- 172.21.15.37:0/2531479212 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f1f50038ed0 0x7f1f5003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.211+0000 7f1f66ffd700 1 --2- 172.21.15.37:0/2531479212 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f1f50038ed0 0x7f1f5003b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.215+0000 7f1f64ff9700 1 -- 172.21.15.37:0/2531479212 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1f58003ac0 con 0x7f1f681082d0 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.476+0000 7f1f6e251700 1 -- 172.21.15.37:0/2531479212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f1f6819efe0 con 0x7f1f681082d0 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.528+0000 7f1f64ff9700 1 -- 172.21.15.37:0/2531479212 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f1f5802d7c0 con 0x7f1f681082d0 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.529+0000 7f1f64ff9700 1 -- 172.21.15.37:0/2531479212 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f1f58018350 con 0x7f1f681082d0 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.530+0000 7f1f6e251700 1 -- 172.21.15.37:0/2531479212 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f1f50038ed0 msgr2=0x7f1f5003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.530+0000 7f1f6e251700 1 --2- 172.21.15.37:0/2531479212 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f1f50038ed0 0x7f1f5003b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.530+0000 7f1f6e251700 1 -- 172.21.15.37:0/2531479212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 msgr2=0x7f1f6819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.530+0000 7f1f6e251700 1 --2- 172.21.15.37:0/2531479212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6819b770 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009fb0 tx=0x7f1f58011bf0 comp rx=0 tx=0).stop 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.530+0000 7f1f6e251700 1 -- 172.21.15.37:0/2531479212 shutdown_connections 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.530+0000 7f1f6e251700 1 --2- 172.21.15.37:0/2531479212 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7f1f50038ed0 0x7f1f5003b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:12.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.530+0000 7f1f6e251700 1 --2- 172.21.15.37:0/2531479212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f681082d0 0x7f1f6819b770 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:12.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.530+0000 7f1f6e251700 1 -- 172.21.15.37:0/2531479212 >> 172.21.15.37:0/2531479212 conn(0x7f1f68101de0 msgr2=0x7f1f68102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:12.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.531+0000 7f1f6e251700 1 -- 172.21.15.37:0/2531479212 shutdown_connections 2024-03-26T06:13:12.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:11.531+0000 7f1f6e251700 1 -- 172.21.15.37:0/2531479212 wait complete. 2024-03-26T06:13:12.401 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:12 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:12.107+0000 7fe675ac7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T06:13:12.402 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:12 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:12.202+0000 7fe675ac7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T06:13:12.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:12 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2531479212' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-26T06:13:12.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:12 smithi037 ceph-mon[31974]: mgrmap e5: x(active, since 7s) 2024-03-26T06:13:12.973 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:12 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:12.878+0000 7fe675ac7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T06:13:12.973 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:12 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:12.973+0000 7fe675ac7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T06:13:13.302 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:13.118+0000 7fe675ac7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T06:13:13.302 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:13.217+0000 7fe675ac7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T06:13:13.303 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:13.302+0000 7fe675ac7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T06:13:13.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-26T06:13:13.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-26T06:13:13.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T06:13:13.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T06:13:13.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T06:13:13.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-26T06:13:13.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.502+0000 7fb33f179700 1 Processor -- start 2024-03-26T06:13:13.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.502+0000 7fb33f179700 1 -- start start 2024-03-26T06:13:13.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.502+0000 7fb33f179700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33810a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:13.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.502+0000 7fb33f179700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33807add0 con 0x7fb3381083e0 2024-03-26T06:13:13.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.502+0000 7fb33cf15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33810a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.502+0000 7fb33cf15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33810a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42426/0 (socket says 172.21.15.37:42426) 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.502+0000 7fb33cf15700 1 -- 172.21.15.37:0/1581278182 learned_addr learned my addr 172.21.15.37:0/1581278182 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.502+0000 7fb33cf15700 1 -- 172.21.15.37:0/1581278182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33810ad10 con 0x7fb3381083e0 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.503+0000 7fb33cf15700 1 --2- 172.21.15.37:0/1581278182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33810a7d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0098a0 tx=0x7fb32c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3622e9f1862ffb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.503+0000 7fb3377fe700 1 -- 172.21.15.37:0/1581278182 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb32c004160 con 0x7fb3381083e0 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.503+0000 7fb3377fe700 1 -- 172.21.15.37:0/1581278182 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb32c004300 con 0x7fb3381083e0 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.503+0000 7fb3377fe700 1 -- 172.21.15.37:0/1581278182 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb32c011b50 con 0x7fb3381083e0 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.503+0000 7fb33f179700 1 -- 172.21.15.37:0/1581278182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 msgr2=0x7fb33810a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.503+0000 7fb33f179700 1 --2- 172.21.15.37:0/1581278182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33810a7d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0098a0 tx=0x7fb32c009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.503+0000 7fb33f179700 1 -- 172.21.15.37:0/1581278182 shutdown_connections 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.503+0000 7fb33f179700 1 --2- 172.21.15.37:0/1581278182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33810a7d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.503+0000 7fb33f179700 1 -- 172.21.15.37:0/1581278182 >> 172.21.15.37:0/1581278182 conn(0x7fb338101e10 msgr2=0x7fb338104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.503+0000 7fb33f179700 1 -- 172.21.15.37:0/1581278182 shutdown_connections 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.503+0000 7fb33f179700 1 -- 172.21.15.37:0/1581278182 wait complete. 2024-03-26T06:13:13.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.504+0000 7fb33f179700 1 Processor -- start 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.504+0000 7fb33f179700 1 -- start start 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.504+0000 7fb33f179700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.504+0000 7fb33f179700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33810bc80 con 0x7fb3381083e0 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.504+0000 7fb33cf15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.504+0000 7fb33cf15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33819b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42432/0 (socket says 172.21.15.37:42432) 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.504+0000 7fb33cf15700 1 -- 172.21.15.37:0/1313426522 learned_addr learned my addr 172.21.15.37:0/1313426522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.505+0000 7fb33cf15700 1 -- 172.21.15.37:0/1313426522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33819bda0 con 0x7fb3381083e0 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.505+0000 7fb33cf15700 1 --2- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33819b860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009fb0 tx=0x7fb32c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.505+0000 7fb335ffb700 1 -- 172.21.15.37:0/1313426522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb32c0038b0 con 0x7fb3381083e0 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.505+0000 7fb33f179700 1 -- 172.21.15.37:0/1313426522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33819c030 con 0x7fb3381083e0 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.505+0000 7fb335ffb700 1 -- 172.21.15.37:0/1313426522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb32c011e50 con 0x7fb3381083e0 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.505+0000 7fb335ffb700 1 -- 172.21.15.37:0/1313426522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb32c019420 con 0x7fb3381083e0 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.505+0000 7fb33f179700 1 -- 172.21.15.37:0/1313426522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33819ed20 con 0x7fb3381083e0 2024-03-26T06:13:13.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.506+0000 7fb335ffb700 1 -- 172.21.15.37:0/1313426522 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fb32c019720 con 0x7fb3381083e0 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.506+0000 7fb33f179700 1 -- 172.21.15.37:0/1313426522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb338068ec0 con 0x7fb3381083e0 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.506+0000 7fb335ffb700 1 --2- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fb328038ed0 0x7fb32803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.506+0000 7fb335ffb700 1 -- 172.21.15.37:0/1313426522 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb32c04d1c0 con 0x7fb3381083e0 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.506+0000 7fb337fff700 1 -- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fb328038ed0 msgr2=0x7fb32803b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/718373544 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.506+0000 7fb337fff700 1 --2- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fb328038ed0 0x7fb32803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.511+0000 7fb335ffb700 1 -- 172.21.15.37:0/1313426522 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb32c00ba30 con 0x7fb3381083e0 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.707+0000 7fb337fff700 1 -- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fb328038ed0 msgr2=0x7fb32803b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/718373544 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.707+0000 7fb337fff700 1 --2- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fb328038ed0 0x7fb32803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.774+0000 7fb33f179700 1 -- 172.21.15.37:0/1313426522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb33819f2e0 con 0x7fb3381083e0 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.774+0000 7fb335ffb700 1 -- 172.21.15.37:0/1313426522 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fb32c00bc90 con 0x7fb3381083e0 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.777+0000 7fb33f179700 1 -- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fb328038ed0 msgr2=0x7fb32803b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.777+0000 7fb33f179700 1 --2- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fb328038ed0 0x7fb32803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.777+0000 7fb33f179700 1 -- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 msgr2=0x7fb33819b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:13.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.777+0000 7fb33f179700 1 --2- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33819b860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009fb0 tx=0x7fb32c011bf0 comp rx=0 tx=0).stop 2024-03-26T06:13:13.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.777+0000 7fb33f179700 1 -- 172.21.15.37:0/1313426522 shutdown_connections 2024-03-26T06:13:13.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.777+0000 7fb33f179700 1 --2- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fb328038ed0 0x7fb32803b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:13.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.777+0000 7fb33f179700 1 --2- 172.21.15.37:0/1313426522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3381083e0 0x7fb33819b860 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:13.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.777+0000 7fb33f179700 1 -- 172.21.15.37:0/1313426522 >> 172.21.15.37:0/1313426522 conn(0x7fb338101e10 msgr2=0x7fb3381029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:13.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.777+0000 7fb33f179700 1 -- 172.21.15.37:0/1313426522 shutdown_connections 2024-03-26T06:13:13.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:12.777+0000 7fb33f179700 1 -- 172.21.15.37:0/1313426522 wait complete. 2024-03-26T06:13:13.434 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-03-26T06:13:13.434 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 5... 2024-03-26T06:13:13.555 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:13 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1313426522' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T06:13:13.883 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:13.577+0000 7fe675ac7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T06:13:13.883 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:13.693+0000 7fe675ac7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T06:13:13.883 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:13.782+0000 7fe675ac7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T06:13:14.197 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:13.883+0000 7fe675ac7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T06:13:14.197 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:14 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:14.063+0000 7fe675ac7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T06:13:15.447 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:15 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:15.061+0000 7fe675ac7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T06:13:15.447 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:15 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:15.196+0000 7fe675ac7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T06:13:15.827 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:15 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:15.534+0000 7fe675ac7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T06:13:15.828 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:15 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:15.627+0000 7fe675ac7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T06:13:15.828 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:15 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:15.828+0000 7fe675ac7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:13:16.197 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:16 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:16.019+0000 7fe675ac7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T06:13:16.572 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:16 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:16.311+0000 7fe675ac7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:13:16.572 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:16 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:16.572+0000 7fe675ac7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T06:13:16.910 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:16 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:16.815+0000 7fe675ac7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T06:13:17.947 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:17 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:17.649+0000 7fe675ac7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T06:13:18.556 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:18.233+0000 7fe675ac7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T06:13:18.556 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:18.335+0000 7fe675ac7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T06:13:18.557 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:18.556+0000 7fe675ac7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T06:13:18.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: Active manager daemon x restarted 2024-03-26T06:13:18.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: Activating manager daemon x 2024-03-26T06:13:18.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: osdmap e2: 0 total, 0 up, 0 in 2024-03-26T06:13:18.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: mgrmap e6: x(active, starting, since 0.00338933s) 2024-03-26T06:13:18.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:13:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T06:13:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T06:13:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T06:13:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T06:13:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: Manager daemon x is now available 2024-03-26T06:13:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:13:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:18 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:13:19.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:19 smithi037 ceph-mon[31974]: Found migration_current of "None". Setting to last migration. 2024-03-26T06:13:19.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:19 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:13:19.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:19 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T06:13:19.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:19 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T06:13:19.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:19 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:19.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:19 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:19.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:19 smithi037 ceph-mon[31974]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T06:13:19.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:19 smithi037 ceph-mon[31974]: mgrmap e7: x(active, since 1.00596s) 2024-03-26T06:13:20.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-26T06:13:20.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-26T06:13:20.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T06:13:20.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-26T06:13:20.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.901+0000 7fe2f94be700 1 Processor -- start 2024-03-26T06:13:20.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.901+0000 7fe2f94be700 1 -- start start 2024-03-26T06:13:20.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.901+0000 7fe2f94be700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f410a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:20.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.901+0000 7fe2f94be700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f407ae90 con 0x7fe2f4108320 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.901+0000 7fe2f2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f410a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.901+0000 7fe2f2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f410a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42448/0 (socket says 172.21.15.37:42448) 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.901+0000 7fe2f2ffd700 1 -- 172.21.15.37:0/349550482 learned_addr learned my addr 172.21.15.37:0/349550482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.902+0000 7fe2f2ffd700 1 -- 172.21.15.37:0/349550482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f410ac50 con 0x7fe2f4108320 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.902+0000 7fe2f2ffd700 1 --2- 172.21.15.37:0/349550482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f410a710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0098a0 tx=0x7fe2dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5661ab7ef2feeeea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.902+0000 7fe2f1ffb700 1 -- 172.21.15.37:0/349550482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2dc004160 con 0x7fe2f4108320 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.902+0000 7fe2f1ffb700 1 -- 172.21.15.37:0/349550482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2dc004300 con 0x7fe2f4108320 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.902+0000 7fe2f1ffb700 1 -- 172.21.15.37:0/349550482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2dc011b50 con 0x7fe2f4108320 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.903+0000 7fe2f94be700 1 -- 172.21.15.37:0/349550482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 msgr2=0x7fe2f410a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.903+0000 7fe2f94be700 1 --2- 172.21.15.37:0/349550482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f410a710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0098a0 tx=0x7fe2dc009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.903+0000 7fe2f94be700 1 -- 172.21.15.37:0/349550482 shutdown_connections 2024-03-26T06:13:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.903+0000 7fe2f94be700 1 --2- 172.21.15.37:0/349550482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f410a710 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.903+0000 7fe2f94be700 1 -- 172.21.15.37:0/349550482 >> 172.21.15.37:0/349550482 conn(0x7fe2f4101e10 msgr2=0x7fe2f4104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.903+0000 7fe2f94be700 1 -- 172.21.15.37:0/349550482 shutdown_connections 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.903+0000 7fe2f94be700 1 -- 172.21.15.37:0/349550482 wait complete. 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.903+0000 7fe2f94be700 1 Processor -- start 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.904+0000 7fe2f94be700 1 -- start start 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.904+0000 7fe2f94be700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f4107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.904+0000 7fe2f94be700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f410bc80 con 0x7fe2f4108320 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.904+0000 7fe2f2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f4107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.904+0000 7fe2f2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f4107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42456/0 (socket says 172.21.15.37:42456) 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.904+0000 7fe2f2ffd700 1 -- 172.21.15.37:0/3810905727 learned_addr learned my addr 172.21.15.37:0/3810905727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.904+0000 7fe2f2ffd700 1 -- 172.21.15.37:0/3810905727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f41080e0 con 0x7fe2f4108320 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.904+0000 7fe2f2ffd700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f4107ba0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009fb0 tx=0x7fe2dc011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:20.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.904+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2dc0038b0 con 0x7fe2f4108320 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.905+0000 7fe2f94be700 1 -- 172.21.15.37:0/3810905727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f41062d0 con 0x7fe2f4108320 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.905+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2dc011e50 con 0x7fe2f4108320 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.905+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2dc019420 con 0x7fe2f4108320 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.905+0000 7fe2f94be700 1 -- 172.21.15.37:0/3810905727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f4106810 con 0x7fe2f4108320 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.906+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fe2dc019720 con 0x7fe2f4108320 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.906+0000 7fe2ebfff700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 0x7fe2e003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.906+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 --> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2e003baa0 con 0x7fe2e0038ed0 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.906+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe2dc04d180 con 0x7fe2f4108320 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.906+0000 7fe2f27fc700 1 -- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 msgr2=0x7fe2e003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/718373544 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:13.906+0000 7fe2f27fc700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 0x7fe2e003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:14.107+0000 7fe2f27fc700 1 -- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 msgr2=0x7fe2e003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/718373544 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:14.107+0000 7fe2f27fc700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 0x7fe2e003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T06:13:20.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:14.508+0000 7fe2f27fc700 1 -- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 msgr2=0x7fe2e003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/718373544 2024-03-26T06:13:20.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:14.508+0000 7fe2f27fc700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 0x7fe2e003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T06:13:20.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:15.309+0000 7fe2f27fc700 1 -- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 msgr2=0x7fe2e003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/718373544 2024-03-26T06:13:20.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:15.309+0000 7fe2f27fc700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 0x7fe2e003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T06:13:20.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:16.910+0000 7fe2f27fc700 1 -- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 msgr2=0x7fe2e003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/718373544 2024-03-26T06:13:20.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:16.910+0000 7fe2f27fc700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 0x7fe2e003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T06:13:20.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:18.572+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 6) v1 ==== 45873+0+0 (secure 0 0 0) 0x7fe2dc019a60 con 0x7fe2f4108320 2024-03-26T06:13:20.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:18.572+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 msgr2=0x7fe2e003b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T06:13:20.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:18.572+0000 7fe2ebfff700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 0x7fe2e003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:20.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.573+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7fe2dc04ef20 con 0x7fe2f4108320 2024-03-26T06:13:20.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.573+0000 7fe2ebfff700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fe2e003c7c0 0x7fe2e003ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:20.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.573+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 --> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2e003baa0 con 0x7fe2e003c7c0 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.574+0000 7fe2f27fc700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fe2e003c7c0 0x7fe2e003ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.575+0000 7fe2f27fc700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fe2e003c7c0 0x7fe2e003ebb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4003a10 tx=0x7fe2e4009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.575+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 <== mgr.14120 v2:172.21.15.37:6800/401419890 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fe2e003baa0 con 0x7fe2e003c7c0 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 -- 172.21.15.37:0/3810905727 --> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe2f4068ec0 con 0x7fe2e003c7c0 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2ebfff700 1 -- 172.21.15.37:0/3810905727 <== mgr.14120 v2:172.21.15.37:6800/401419890 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe2f4068ec0 con 0x7fe2e003c7c0 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 -- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fe2e003c7c0 msgr2=0x7fe2e003ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fe2e003c7c0 0x7fe2e003ebb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4003a10 tx=0x7fe2e4009290 comp rx=0 tx=0).stop 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 -- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 msgr2=0x7fe2f4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f4107ba0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009fb0 tx=0x7fe2dc011bf0 comp rx=0 tx=0).stop 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 -- 172.21.15.37:0/3810905727 shutdown_connections 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fe2e003c7c0 0x7fe2e003ebb0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:20.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2f4108320 0x7fe2f4107ba0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:20.204 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 --2- 172.21.15.37:0/3810905727 >> [v2:172.21.15.37:6800/718373544,v1:172.21.15.37:6801/718373544] conn(0x7fe2e0038ed0 0x7fe2e003b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:20.204 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 -- 172.21.15.37:0/3810905727 >> 172.21.15.37:0/3810905727 conn(0x7fe2f4101e10 msgr2=0x7fe2f4102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:20.204 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 -- 172.21.15.37:0/3810905727 shutdown_connections 2024-03-26T06:13:20.204 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:19.582+0000 7fe2f94be700 1 -- 172.21.15.37:0/3810905727 wait complete. 2024-03-26T06:13:20.204 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 5 is available 2024-03-26T06:13:20.204 INFO:teuthology.orchestra.run.smithi037.stdout:Setting orchestrator backend to cephadm... 2024-03-26T06:13:20.876 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:20 smithi037 ceph-mon[31974]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T06:13:20.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:20 smithi037 ceph-mon[31974]: [26/Mar/2024:06:13:19] ENGINE Bus STARTING 2024-03-26T06:13:20.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:20 smithi037 ceph-mon[31974]: [26/Mar/2024:06:13:19] ENGINE Serving on http://172.21.15.37:8765 2024-03-26T06:13:20.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:20 smithi037 ceph-mon[31974]: [26/Mar/2024:06:13:20] ENGINE Serving on https://172.21.15.37:7150 2024-03-26T06:13:20.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:20 smithi037 ceph-mon[31974]: [26/Mar/2024:06:13:20] ENGINE Bus STARTED 2024-03-26T06:13:20.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:20 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:13:21.485 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.645+0000 7f77e07a0700 1 Processor -- start 2024-03-26T06:13:21.485 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.645+0000 7f77e07a0700 1 -- start start 2024-03-26T06:13:21.485 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.645+0000 7f77e07a0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d8107d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:21.485 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.645+0000 7f77e07a0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8108330 con 0x7f77d8107940 2024-03-26T06:13:21.485 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.645+0000 7f77de53c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d8107d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.645+0000 7f77de53c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d8107d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59346/0 (socket says 172.21.15.37:59346) 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.645+0000 7f77de53c700 1 -- 172.21.15.37:0/2996007600 learned_addr learned my addr 172.21.15.37:0/2996007600 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.645+0000 7f77de53c700 1 -- 172.21.15.37:0/2996007600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d8108af0 con 0x7f77d8107940 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.646+0000 7f77de53c700 1 --2- 172.21.15.37:0/2996007600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d8107d60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f77d40098a0 tx=0x7f77d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb2261e9304ba61 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.646+0000 7f77dd53a700 1 -- 172.21.15.37:0/2996007600 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d4004160 con 0x7f77d8107940 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.646+0000 7f77dd53a700 1 -- 172.21.15.37:0/2996007600 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77d4004300 con 0x7f77d8107940 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.646+0000 7f77dd53a700 1 -- 172.21.15.37:0/2996007600 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d4011b50 con 0x7f77d8107940 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.646+0000 7f77e07a0700 1 -- 172.21.15.37:0/2996007600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 msgr2=0x7f77d8107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.646+0000 7f77e07a0700 1 --2- 172.21.15.37:0/2996007600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d8107d60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f77d40098a0 tx=0x7f77d4009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.646+0000 7f77e07a0700 1 -- 172.21.15.37:0/2996007600 shutdown_connections 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.646+0000 7f77e07a0700 1 --2- 172.21.15.37:0/2996007600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d8107d60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.646+0000 7f77e07a0700 1 -- 172.21.15.37:0/2996007600 >> 172.21.15.37:0/2996007600 conn(0x7f77d8102f40 msgr2=0x7f77d8105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:21.486 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.646+0000 7f77e07a0700 1 -- 172.21.15.37:0/2996007600 shutdown_connections 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.646+0000 7f77e07a0700 1 -- 172.21.15.37:0/2996007600 wait complete. 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.647+0000 7f77e07a0700 1 Processor -- start 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.647+0000 7f77e07a0700 1 -- start start 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.647+0000 7f77e07a0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.647+0000 7f77e07a0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d810bea0 con 0x7f77d8107940 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.647+0000 7f77de53c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.647+0000 7f77de53c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d819b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59362/0 (socket says 172.21.15.37:59362) 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.647+0000 7f77de53c700 1 -- 172.21.15.37:0/3355847782 learned_addr learned my addr 172.21.15.37:0/3355847782 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.648+0000 7f77de53c700 1 -- 172.21.15.37:0/3355847782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d819be50 con 0x7f77d8107940 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.648+0000 7f77de53c700 1 --2- 172.21.15.37:0/3355847782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d819b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f77d4011850 tx=0x7f77d4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.648+0000 7f77cf7fe700 1 -- 172.21.15.37:0/3355847782 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d4003af0 con 0x7f77d8107940 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.648+0000 7f77cf7fe700 1 -- 172.21.15.37:0/3355847782 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77d4020070 con 0x7f77d8107940 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.648+0000 7f77e07a0700 1 -- 172.21.15.37:0/3355847782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d819c0e0 con 0x7f77d8107940 2024-03-26T06:13:21.487 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.648+0000 7f77cf7fe700 1 -- 172.21.15.37:0/3355847782 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d40195a0 con 0x7f77d8107940 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.648+0000 7f77e07a0700 1 -- 172.21.15.37:0/3355847782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d819edd0 con 0x7f77d8107940 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.649+0000 7f77cf7fe700 1 -- 172.21.15.37:0/3355847782 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f77d40198a0 con 0x7f77d8107940 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.649+0000 7f77e07a0700 1 -- 172.21.15.37:0/3355847782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d8063410 con 0x7f77d8107940 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.649+0000 7f77cf7fe700 1 --2- 172.21.15.37:0/3355847782 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f77c4038f20 0x7f77c403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.649+0000 7f77cf7fe700 1 -- 172.21.15.37:0/3355847782 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f77d4011e00 con 0x7f77d8107940 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.649+0000 7f77ddd3b700 1 --2- 172.21.15.37:0/3355847782 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f77c4038f20 0x7f77c403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.649+0000 7f77ddd3b700 1 --2- 172.21.15.37:0/3355847782 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f77c4038f20 0x7f77c403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77c8006fd0 tx=0x7f77c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.654+0000 7f77cf7fe700 1 -- 172.21.15.37:0/3355847782 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f77d4003c90 con 0x7f77d8107940 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.877+0000 7f77e07a0700 1 -- 172.21.15.37:0/3355847782 --> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f77d81041c0 con 0x7f77c4038f20 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.882+0000 7f77cf7fe700 1 -- 172.21.15.37:0/3355847782 <== mgr.14120 v2:172.21.15.37:6800/401419890 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f77d81041c0 con 0x7f77c4038f20 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.884+0000 7f77e07a0700 1 -- 172.21.15.37:0/3355847782 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f77c4038f20 msgr2=0x7f77c403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.884+0000 7f77e07a0700 1 --2- 172.21.15.37:0/3355847782 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f77c4038f20 0x7f77c403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77c8006fd0 tx=0x7f77c8006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:21.488 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.884+0000 7f77e07a0700 1 -- 172.21.15.37:0/3355847782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 msgr2=0x7f77d819b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:21.489 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.884+0000 7f77e07a0700 1 --2- 172.21.15.37:0/3355847782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d819b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f77d4011850 tx=0x7f77d4011c10 comp rx=0 tx=0).stop 2024-03-26T06:13:21.489 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.884+0000 7f77e07a0700 1 -- 172.21.15.37:0/3355847782 shutdown_connections 2024-03-26T06:13:21.489 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.884+0000 7f77e07a0700 1 --2- 172.21.15.37:0/3355847782 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f77c4038f20 0x7f77c403b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:21.489 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.884+0000 7f77e07a0700 1 --2- 172.21.15.37:0/3355847782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d8107940 0x7f77d819b910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:21.489 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.884+0000 7f77e07a0700 1 -- 172.21.15.37:0/3355847782 >> 172.21.15.37:0/3355847782 conn(0x7f77d8102f40 msgr2=0x7f77d8103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:21.489 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.884+0000 7f77e07a0700 1 -- 172.21.15.37:0/3355847782 shutdown_connections 2024-03-26T06:13:21.489 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:20.884+0000 7f77e07a0700 1 -- 172.21.15.37:0/3355847782 wait complete. 2024-03-26T06:13:21.863 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:21 smithi037 ceph-mon[31974]: mgrmap e8: x(active, since 2s) 2024-03-26T06:13:21.863 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:21 smithi037 ceph-mon[31974]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:13:21.863 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:21 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:21.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:21 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:13:22.692 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:22 smithi037 ceph-mon[31974]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:13:22.699 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.895+0000 7fd718621700 1 Processor -- start 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.895+0000 7fd718621700 1 -- start start 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.896+0000 7fd718621700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd710107d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.896+0000 7fd718621700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd710108330 con 0x7fd710107940 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.896+0000 7fd7163bd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd710107d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.896+0000 7fd7163bd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd710107d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59368/0 (socket says 172.21.15.37:59368) 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.896+0000 7fd7163bd700 1 -- 172.21.15.37:0/2756476418 learned_addr learned my addr 172.21.15.37:0/2756476418 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.896+0000 7fd7163bd700 1 -- 172.21.15.37:0/2756476418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710108af0 con 0x7fd710107940 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.896+0000 7fd7163bd700 1 --2- 172.21.15.37:0/2756476418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd710107d60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0098a0 tx=0x7fd70c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367fd7a62c4f44d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.896+0000 7fd7153bb700 1 -- 172.21.15.37:0/2756476418 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd70c004160 con 0x7fd710107940 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.897+0000 7fd7153bb700 1 -- 172.21.15.37:0/2756476418 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd70c004300 con 0x7fd710107940 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.897+0000 7fd7153bb700 1 -- 172.21.15.37:0/2756476418 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd70c011b50 con 0x7fd710107940 2024-03-26T06:13:22.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.897+0000 7fd718621700 1 -- 172.21.15.37:0/2756476418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 msgr2=0x7fd710107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:22.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.897+0000 7fd718621700 1 --2- 172.21.15.37:0/2756476418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd710107d60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0098a0 tx=0x7fd70c009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:22.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.897+0000 7fd718621700 1 -- 172.21.15.37:0/2756476418 shutdown_connections 2024-03-26T06:13:22.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.897+0000 7fd718621700 1 --2- 172.21.15.37:0/2756476418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd710107d60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:22.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.897+0000 7fd718621700 1 -- 172.21.15.37:0/2756476418 >> 172.21.15.37:0/2756476418 conn(0x7fd710102f40 msgr2=0x7fd710105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:22.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.897+0000 7fd718621700 1 -- 172.21.15.37:0/2756476418 shutdown_connections 2024-03-26T06:13:22.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.897+0000 7fd718621700 1 -- 172.21.15.37:0/2756476418 wait complete. 2024-03-26T06:13:22.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.898+0000 7fd718621700 1 Processor -- start 2024-03-26T06:13:22.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.898+0000 7fd718621700 1 -- start start 2024-03-26T06:13:22.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.898+0000 7fd718621700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd71007cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:22.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.898+0000 7fd718621700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71010bea0 con 0x7fd710107940 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.898+0000 7fd7163bd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd71007cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.898+0000 7fd7163bd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd71007cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59380/0 (socket says 172.21.15.37:59380) 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.898+0000 7fd7163bd700 1 -- 172.21.15.37:0/3461359794 learned_addr learned my addr 172.21.15.37:0/3461359794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.898+0000 7fd7163bd700 1 -- 172.21.15.37:0/3461359794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71007d090 con 0x7fd710107940 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.899+0000 7fd7163bd700 1 --2- 172.21.15.37:0/3461359794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd71007cb50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd70c011850 tx=0x7fd70c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.899+0000 7fd7037fe700 1 -- 172.21.15.37:0/3461359794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd70c003af0 con 0x7fd710107940 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.899+0000 7fd7037fe700 1 -- 172.21.15.37:0/3461359794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd70c020070 con 0x7fd710107940 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.899+0000 7fd7037fe700 1 -- 172.21.15.37:0/3461359794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd70c0195a0 con 0x7fd710107940 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.899+0000 7fd718621700 1 -- 172.21.15.37:0/3461359794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71007b280 con 0x7fd710107940 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.899+0000 7fd718621700 1 -- 172.21.15.37:0/3461359794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71007b7c0 con 0x7fd710107940 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.900+0000 7fd7037fe700 1 -- 172.21.15.37:0/3461359794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fd70c0198a0 con 0x7fd710107940 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.900+0000 7fd7037fe700 1 --2- 172.21.15.37:0/3461359794 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd6fc038f20 0x7fd6fc03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.900+0000 7fd7037fe700 1 -- 172.21.15.37:0/3461359794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd70c011e00 con 0x7fd710107940 2024-03-26T06:13:22.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.900+0000 7fd718621700 1 -- 172.21.15.37:0/3461359794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd710068ec0 con 0x7fd710107940 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.900+0000 7fd715bbc700 1 --2- 172.21.15.37:0/3461359794 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd6fc038f20 0x7fd6fc03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.900+0000 7fd715bbc700 1 --2- 172.21.15.37:0/3461359794 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd6fc038f20 0x7fd6fc03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd704006fd0 tx=0x7fd704006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:21.905+0000 7fd7037fe700 1 -- 172.21.15.37:0/3461359794 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd70c003c90 con 0x7fd710107940 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.111+0000 7fd718621700 1 -- 172.21.15.37:0/3461359794 --> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fd71007c0f0 con 0x7fd6fc038f20 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.112+0000 7fd7037fe700 1 -- 172.21.15.37:0/3461359794 <== mgr.14120 v2:172.21.15.37:6800/401419890 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fd71007c0f0 con 0x7fd6fc038f20 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.114+0000 7fd718621700 1 -- 172.21.15.37:0/3461359794 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd6fc038f20 msgr2=0x7fd6fc03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.114+0000 7fd718621700 1 --2- 172.21.15.37:0/3461359794 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd6fc038f20 0x7fd6fc03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd704006fd0 tx=0x7fd704006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.114+0000 7fd718621700 1 -- 172.21.15.37:0/3461359794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 msgr2=0x7fd71007cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.114+0000 7fd718621700 1 --2- 172.21.15.37:0/3461359794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd71007cb50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd70c011850 tx=0x7fd70c011c10 comp rx=0 tx=0).stop 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.114+0000 7fd718621700 1 -- 172.21.15.37:0/3461359794 shutdown_connections 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.114+0000 7fd718621700 1 --2- 172.21.15.37:0/3461359794 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd6fc038f20 0x7fd6fc03b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.114+0000 7fd718621700 1 --2- 172.21.15.37:0/3461359794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710107940 0x7fd71007cb50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.114+0000 7fd718621700 1 -- 172.21.15.37:0/3461359794 >> 172.21.15.37:0/3461359794 conn(0x7fd710102f40 msgr2=0x7fd710103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.114+0000 7fd718621700 1 -- 172.21.15.37:0/3461359794 shutdown_connections 2024-03-26T06:13:22.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:22.114+0000 7fd718621700 1 -- 172.21.15.37:0/3461359794 wait complete. 2024-03-26T06:13:22.704 INFO:teuthology.orchestra.run.smithi037.stdout:Generating ssh key... 2024-03-26T06:13:23.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: Generating public/private rsa key pair. 2024-03-26T06:13:23.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: Your identification has been saved in /tmp/tmphtxoiiks/key. 2024-03-26T06:13:23.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: Your public key has been saved in /tmp/tmphtxoiiks/key.pub. 2024-03-26T06:13:23.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: The key fingerprint is: 2024-03-26T06:13:23.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: SHA256:j5Rzl1mMATw9ItQskTsQQ7+F7nnITd0VH1FGyRLEDjM ceph-a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:13:23.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: The key's randomart image is: 2024-03-26T06:13:23.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: +---[RSA 3072]----+ 2024-03-26T06:13:23.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: | .+oo*.o+oo=B| 2024-03-26T06:13:23.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: | .o+.*Eo* ++| 2024-03-26T06:13:23.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: | .o+.o*.+ o| 2024-03-26T06:13:23.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: | .o+ . * . | 2024-03-26T06:13:23.699 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: | S.o = . | 2024-03-26T06:13:23.699 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: | + X . | 2024-03-26T06:13:23.699 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: | * + | 2024-03-26T06:13:23.699 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: | . | 2024-03-26T06:13:23.699 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: | | 2024-03-26T06:13:23.699 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:23 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: +----[SHA256]-----+ 2024-03-26T06:13:24.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.128+0000 7fba171d7700 1 Processor -- start 2024-03-26T06:13:24.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.128+0000 7fba171d7700 1 -- start start 2024-03-26T06:13:24.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.128+0000 7fba171d7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba10107d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.128+0000 7fba171d7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba10108330 con 0x7fba10107940 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.128+0000 7fba14f73700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba10107d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.128+0000 7fba14f73700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba10107d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59388/0 (socket says 172.21.15.37:59388) 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.128+0000 7fba14f73700 1 -- 172.21.15.37:0/2864217534 learned_addr learned my addr 172.21.15.37:0/2864217534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.129+0000 7fba14f73700 1 -- 172.21.15.37:0/2864217534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10108af0 con 0x7fba10107940 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.129+0000 7fba14f73700 1 --2- 172.21.15.37:0/2864217534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba10107d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fba040098a0 tx=0x7fba04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d80af2d60d85229 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.129+0000 7fba0f7fe700 1 -- 172.21.15.37:0/2864217534 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba04004160 con 0x7fba10107940 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.129+0000 7fba0f7fe700 1 -- 172.21.15.37:0/2864217534 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba04004300 con 0x7fba10107940 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.129+0000 7fba0f7fe700 1 -- 172.21.15.37:0/2864217534 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba04011b50 con 0x7fba10107940 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.129+0000 7fba171d7700 1 -- 172.21.15.37:0/2864217534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 msgr2=0x7fba10107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.129+0000 7fba171d7700 1 --2- 172.21.15.37:0/2864217534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba10107d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fba040098a0 tx=0x7fba04009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.130+0000 7fba171d7700 1 -- 172.21.15.37:0/2864217534 shutdown_connections 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.130+0000 7fba171d7700 1 --2- 172.21.15.37:0/2864217534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba10107d60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:24.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.130+0000 7fba171d7700 1 -- 172.21.15.37:0/2864217534 >> 172.21.15.37:0/2864217534 conn(0x7fba10102f40 msgr2=0x7fba10105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.130+0000 7fba171d7700 1 -- 172.21.15.37:0/2864217534 shutdown_connections 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.130+0000 7fba171d7700 1 -- 172.21.15.37:0/2864217534 wait complete. 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.130+0000 7fba171d7700 1 Processor -- start 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.130+0000 7fba171d7700 1 -- start start 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.130+0000 7fba171d7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba1007cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.130+0000 7fba171d7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1010bea0 con 0x7fba10107940 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.130+0000 7fba14f73700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba1007cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.131+0000 7fba14f73700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba1007cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59398/0 (socket says 172.21.15.37:59398) 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.131+0000 7fba14f73700 1 -- 172.21.15.37:0/698997956 learned_addr learned my addr 172.21.15.37:0/698997956 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.131+0000 7fba14f73700 1 -- 172.21.15.37:0/698997956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1007d090 con 0x7fba10107940 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.131+0000 7fba14f73700 1 --2- 172.21.15.37:0/698997956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba1007cb50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fba04011850 tx=0x7fba04011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.131+0000 7fba0dffb700 1 -- 172.21.15.37:0/698997956 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba04003af0 con 0x7fba10107940 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.131+0000 7fba0dffb700 1 -- 172.21.15.37:0/698997956 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba04020070 con 0x7fba10107940 2024-03-26T06:13:24.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.131+0000 7fba171d7700 1 -- 172.21.15.37:0/698997956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1007b280 con 0x7fba10107940 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.131+0000 7fba0dffb700 1 -- 172.21.15.37:0/698997956 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba040195a0 con 0x7fba10107940 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.131+0000 7fba171d7700 1 -- 172.21.15.37:0/698997956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1007b7c0 con 0x7fba10107940 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.132+0000 7fba0dffb700 1 -- 172.21.15.37:0/698997956 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fba04019850 con 0x7fba10107940 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.132+0000 7fba0dffb700 1 --2- 172.21.15.37:0/698997956 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fba00038f70 0x7fba0003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.132+0000 7fba0dffb700 1 -- 172.21.15.37:0/698997956 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fba04011e00 con 0x7fba10107940 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.132+0000 7fba0ffff700 1 --2- 172.21.15.37:0/698997956 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fba00038f70 0x7fba0003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.132+0000 7fba171d7700 1 -- 172.21.15.37:0/698997956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba10068ec0 con 0x7fba10107940 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.133+0000 7fba0ffff700 1 --2- 172.21.15.37:0/698997956 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fba00038f70 0x7fba0003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc006fd0 tx=0x7fb9fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.137+0000 7fba0dffb700 1 -- 172.21.15.37:0/698997956 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fba04003c90 con 0x7fba10107940 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.341+0000 7fba171d7700 1 -- 172.21.15.37:0/698997956 --> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fba1007be40 con 0x7fba00038f70 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.578+0000 7fba0dffb700 1 -- 172.21.15.37:0/698997956 <== mgr.14120 v2:172.21.15.37:6800/401419890 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fba1007be40 con 0x7fba00038f70 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.580+0000 7fba171d7700 1 -- 172.21.15.37:0/698997956 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fba00038f70 msgr2=0x7fba0003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.580+0000 7fba171d7700 1 --2- 172.21.15.37:0/698997956 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fba00038f70 0x7fba0003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc006fd0 tx=0x7fb9fc006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:24.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.580+0000 7fba171d7700 1 -- 172.21.15.37:0/698997956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 msgr2=0x7fba1007cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:24.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.580+0000 7fba171d7700 1 --2- 172.21.15.37:0/698997956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba1007cb50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fba04011850 tx=0x7fba04011c10 comp rx=0 tx=0).stop 2024-03-26T06:13:24.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.580+0000 7fba171d7700 1 -- 172.21.15.37:0/698997956 shutdown_connections 2024-03-26T06:13:24.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.580+0000 7fba171d7700 1 --2- 172.21.15.37:0/698997956 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fba00038f70 0x7fba0003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:24.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.580+0000 7fba171d7700 1 --2- 172.21.15.37:0/698997956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10107940 0x7fba1007cb50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:24.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.580+0000 7fba171d7700 1 -- 172.21.15.37:0/698997956 >> 172.21.15.37:0/698997956 conn(0x7fba10102f40 msgr2=0x7fba10103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:24.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.580+0000 7fba171d7700 1 -- 172.21.15.37:0/698997956 shutdown_connections 2024-03-26T06:13:24.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:23.580+0000 7fba171d7700 1 -- 172.21.15.37:0/698997956 wait complete. 2024-03-26T06:13:24.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:24 smithi037 ceph-mon[31974]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:13:24.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:24 smithi037 ceph-mon[31974]: Generating ssh key... 2024-03-26T06:13:24.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:24 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:24.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:24 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:25.454 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/5b51G256dU7INFRedpjcZ/EMzCqlqs9iAq5UYYr0gdT3zx91RNoBzMBa5txxiaPd6+5iZn5v7pWM8xDkHsHaM2Uk5J1oq2OsdLriH30NeAmJOaPEyAHy+2zgSe0MCa2ivDnDe+KbnaWbcCu/OcV6IJxHWgmM1loD3f/Qkb1hrbKsj5uAZDzGtZ5CfiHRsyx95l0cEJgW9t3NSZrOYNjBmZ//lZtpx8XgugKHEQySeVXdAAexVk4569AVnc9GY6Qn43n4XYUR4WJx3xjT45cLWzexa+T2bdi09EzoInxw12dR5bZjuWffZtc0iOKS/erFAm8aRuijBpVwzusSDW7H+1f1yn7goiM/HcYdYYJYosorC6Y9XZar3EBIN5jWWOgHyRQnxEMqmPnUKESj9BqXZKmhwFRalgtv6BAFCHFDSlf8PiSDwEHkqrXgGLkjKnw8P10pisMj1rkc43GB3W58zEGH4WZ9VNV867p+Zx62tjCFoYFdD2J+HwcyUHTec= ceph-a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.670+0000 7fd76078e700 1 Processor -- start 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.670+0000 7fd76078e700 1 -- start start 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.670+0000 7fd76078e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.670+0000 7fd76078e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75807b800 con 0x7fd75807ce10 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.670+0000 7fd75e52a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.670+0000 7fd75e52a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75807b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59400/0 (socket says 172.21.15.37:59400) 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.670+0000 7fd75e52a700 1 -- 172.21.15.37:0/728881890 learned_addr learned my addr 172.21.15.37:0/728881890 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.671+0000 7fd75e52a700 1 -- 172.21.15.37:0/728881890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75807b980 con 0x7fd75807ce10 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.671+0000 7fd75e52a700 1 --2- 172.21.15.37:0/728881890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75807b2c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd7540098a0 tx=0x7fd754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17647f9030542aeb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.671+0000 7fd75d528700 1 -- 172.21.15.37:0/728881890 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd754004160 con 0x7fd75807ce10 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.671+0000 7fd75d528700 1 -- 172.21.15.37:0/728881890 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd754004300 con 0x7fd75807ce10 2024-03-26T06:13:25.455 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.671+0000 7fd75d528700 1 -- 172.21.15.37:0/728881890 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd754011b50 con 0x7fd75807ce10 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.672+0000 7fd76078e700 1 -- 172.21.15.37:0/728881890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 msgr2=0x7fd75807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.672+0000 7fd76078e700 1 --2- 172.21.15.37:0/728881890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75807b2c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd7540098a0 tx=0x7fd754009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.672+0000 7fd76078e700 1 -- 172.21.15.37:0/728881890 shutdown_connections 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.672+0000 7fd76078e700 1 --2- 172.21.15.37:0/728881890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75807b2c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.672+0000 7fd76078e700 1 -- 172.21.15.37:0/728881890 >> 172.21.15.37:0/728881890 conn(0x7fd758102ee0 msgr2=0x7fd758105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.672+0000 7fd76078e700 1 -- 172.21.15.37:0/728881890 shutdown_connections 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.672+0000 7fd76078e700 1 -- 172.21.15.37:0/728881890 wait complete. 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.672+0000 7fd76078e700 1 Processor -- start 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd76078e700 1 -- start start 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd76078e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd76078e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75810deb0 con 0x7fd75807ce10 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd75e52a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd75e52a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75819dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59402/0 (socket says 172.21.15.37:59402) 2024-03-26T06:13:25.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd75e52a700 1 -- 172.21.15.37:0/1245474914 learned_addr learned my addr 172.21.15.37:0/1245474914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd75e52a700 1 -- 172.21.15.37:0/1245474914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75819dff0 con 0x7fd75807ce10 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd75e52a700 1 --2- 172.21.15.37:0/1245474914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75819dab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd754009fb0 tx=0x7fd754011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd74f7fe700 1 -- 172.21.15.37:0/1245474914 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7540038b0 con 0x7fd75807ce10 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd76078e700 1 -- 172.21.15.37:0/1245474914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75819e280 con 0x7fd75807ce10 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd74f7fe700 1 -- 172.21.15.37:0/1245474914 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd754011e50 con 0x7fd75807ce10 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd74f7fe700 1 -- 172.21.15.37:0/1245474914 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd754019420 con 0x7fd75807ce10 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.673+0000 7fd76078e700 1 -- 172.21.15.37:0/1245474914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7581a0f70 con 0x7fd75807ce10 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.674+0000 7fd74f7fe700 1 -- 172.21.15.37:0/1245474914 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fd754019720 con 0x7fd75807ce10 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.674+0000 7fd76078e700 1 -- 172.21.15.37:0/1245474914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd758068ec0 con 0x7fd75807ce10 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.674+0000 7fd74f7fe700 1 --2- 172.21.15.37:0/1245474914 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd744038ed0 0x7fd74403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.675+0000 7fd74f7fe700 1 -- 172.21.15.37:0/1245474914 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd75404d120 con 0x7fd75807ce10 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.675+0000 7fd75dd29700 1 --2- 172.21.15.37:0/1245474914 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd744038ed0 0x7fd74403b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.675+0000 7fd75dd29700 1 --2- 172.21.15.37:0/1245474914 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd744038ed0 0x7fd74403b390 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd748006fd0 tx=0x7fd748006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.679+0000 7fd74f7fe700 1 -- 172.21.15.37:0/1245474914 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd754003a50 con 0x7fd75807ce10 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.884+0000 7fd76078e700 1 -- 172.21.15.37:0/1245474914 --> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd758104170 con 0x7fd744038ed0 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.885+0000 7fd74f7fe700 1 -- 172.21.15.37:0/1245474914 <== mgr.14120 v2:172.21.15.37:6800/401419890 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fd758104170 con 0x7fd744038ed0 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.887+0000 7fd76078e700 1 -- 172.21.15.37:0/1245474914 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd744038ed0 msgr2=0x7fd74403b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.887+0000 7fd76078e700 1 --2- 172.21.15.37:0/1245474914 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd744038ed0 0x7fd74403b390 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd748006fd0 tx=0x7fd748006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.887+0000 7fd76078e700 1 -- 172.21.15.37:0/1245474914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 msgr2=0x7fd75819dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.887+0000 7fd76078e700 1 --2- 172.21.15.37:0/1245474914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75819dab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd754009fb0 tx=0x7fd754011bf0 comp rx=0 tx=0).stop 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.887+0000 7fd76078e700 1 -- 172.21.15.37:0/1245474914 shutdown_connections 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.887+0000 7fd76078e700 1 --2- 172.21.15.37:0/1245474914 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fd744038ed0 0x7fd74403b390 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.887+0000 7fd76078e700 1 --2- 172.21.15.37:0/1245474914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd75807ce10 0x7fd75819dab0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.887+0000 7fd76078e700 1 -- 172.21.15.37:0/1245474914 >> 172.21.15.37:0/1245474914 conn(0x7fd758102ee0 msgr2=0x7fd758103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.887+0000 7fd76078e700 1 -- 172.21.15.37:0/1245474914 shutdown_connections 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:24.887+0000 7fd76078e700 1 -- 172.21.15.37:0/1245474914 wait complete. 2024-03-26T06:13:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-26T06:13:25.459 INFO:teuthology.orchestra.run.smithi037.stdout:Adding key to root@localhost authorized_keys... 2024-03-26T06:13:25.459 INFO:teuthology.orchestra.run.smithi037.stdout:Adding host smithi037... 2024-03-26T06:13:25.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:25 smithi037 ceph-mon[31974]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:13:26.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:26 smithi037 ceph-mon[31974]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:13:27.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:27 smithi037 ceph-mon[31974]: Deploying cephadm binary to smithi037 2024-03-26T06:13:31.161 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Added host 'smithi037' with addr '172.21.15.37' 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.905+0000 7fb020e82700 1 Processor -- start 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.905+0000 7fb020e82700 1 -- start start 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.905+0000 7fb020e82700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c106c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.905+0000 7fb020e82700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c1071d0 con 0x7fb01c1067e0 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.906+0000 7fb01a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c106c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.906+0000 7fb01a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c106c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59408/0 (socket says 172.21.15.37:59408) 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.906+0000 7fb01a59c700 1 -- 172.21.15.37:0/3161771142 learned_addr learned my addr 172.21.15.37:0/3161771142 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.906+0000 7fb01a59c700 1 -- 172.21.15.37:0/3161771142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c1079e0 con 0x7fb01c1067e0 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.906+0000 7fb01a59c700 1 --2- 172.21.15.37:0/3161771142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c106c00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009e80 tx=0x7fb00c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5233e65451df2cd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.906+0000 7fb01959a700 1 -- 172.21.15.37:0/3161771142 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb00c017070 con 0x7fb01c1067e0 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.906+0000 7fb01959a700 1 -- 172.21.15.37:0/3161771142 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb00c00b8c0 con 0x7fb01c1067e0 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.906+0000 7fb01959a700 1 -- 172.21.15.37:0/3161771142 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb00c0045c0 con 0x7fb01c1067e0 2024-03-26T06:13:31.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.907+0000 7fb020e82700 1 -- 172.21.15.37:0/3161771142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 msgr2=0x7fb01c106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.907+0000 7fb020e82700 1 --2- 172.21.15.37:0/3161771142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c106c00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009e80 tx=0x7fb00c00b330 comp rx=0 tx=0).stop 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.907+0000 7fb020e82700 1 -- 172.21.15.37:0/3161771142 shutdown_connections 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.907+0000 7fb020e82700 1 --2- 172.21.15.37:0/3161771142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c106c00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.907+0000 7fb020e82700 1 -- 172.21.15.37:0/3161771142 >> 172.21.15.37:0/3161771142 conn(0x7fb01c101e40 msgr2=0x7fb01c104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.907+0000 7fb020e82700 1 -- 172.21.15.37:0/3161771142 shutdown_connections 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.907+0000 7fb020e82700 1 -- 172.21.15.37:0/3161771142 wait complete. 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.908+0000 7fb020e82700 1 Processor -- start 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.908+0000 7fb020e82700 1 -- start start 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.908+0000 7fb020e82700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c191990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.908+0000 7fb020e82700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c10ad90 con 0x7fb01c1067e0 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.908+0000 7fb01a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c191990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.908+0000 7fb01a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c191990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59412/0 (socket says 172.21.15.37:59412) 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.908+0000 7fb01a59c700 1 -- 172.21.15.37:0/4180415171 learned_addr learned my addr 172.21.15.37:0/4180415171 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.909+0000 7fb01a59c700 1 -- 172.21.15.37:0/4180415171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c191ed0 con 0x7fb01c1067e0 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.909+0000 7fb01a59c700 1 --2- 172.21.15.37:0/4180415171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c191990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009e50 tx=0x7fb00c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:31.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.909+0000 7fb0137fe700 1 -- 172.21.15.37:0/4180415171 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb00c020020 con 0x7fb01c1067e0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.909+0000 7fb0137fe700 1 -- 172.21.15.37:0/4180415171 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb00c003b50 con 0x7fb01c1067e0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.909+0000 7fb020e82700 1 -- 172.21.15.37:0/4180415171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c1900c0 con 0x7fb01c1067e0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.909+0000 7fb0137fe700 1 -- 172.21.15.37:0/4180415171 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb00c017040 con 0x7fb01c1067e0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.909+0000 7fb020e82700 1 -- 172.21.15.37:0/4180415171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c190600 con 0x7fb01c1067e0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.910+0000 7fb0137fe700 1 -- 172.21.15.37:0/4180415171 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fb00c023020 con 0x7fb01c1067e0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.910+0000 7fb020e82700 1 -- 172.21.15.37:0/4180415171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c063410 con 0x7fb01c1067e0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.910+0000 7fb0137fe700 1 --2- 172.21.15.37:0/4180415171 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb008038ed0 0x7fb00803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.910+0000 7fb0137fe700 1 -- 172.21.15.37:0/4180415171 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb00c00f050 con 0x7fb01c1067e0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.910+0000 7fb019d9b700 1 --2- 172.21.15.37:0/4180415171 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb008038ed0 0x7fb00803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.911+0000 7fb019d9b700 1 --2- 172.21.15.37:0/4180415171 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb008038ed0 0x7fb00803b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb004006fd0 tx=0x7fb004006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:25.915+0000 7fb0137fe700 1 -- 172.21.15.37:0/4180415171 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb00c02a020 con 0x7fb01c1067e0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:26.123+0000 7fb020e82700 1 -- 172.21.15.37:0/4180415171 --> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}) v1 -- 0x7fb01c195700 con 0x7fb008038ed0 2024-03-26T06:13:31.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:30.521+0000 7fb0137fe700 1 -- 172.21.15.37:0/4180415171 <== mgr.14120 v2:172.21.15.37:6800/401419890 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fb01c195700 con 0x7fb008038ed0 2024-03-26T06:13:31.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:30.523+0000 7fb020e82700 1 -- 172.21.15.37:0/4180415171 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb008038ed0 msgr2=0x7fb00803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:31.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:30.523+0000 7fb020e82700 1 --2- 172.21.15.37:0/4180415171 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb008038ed0 0x7fb00803b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb004006fd0 tx=0x7fb004006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:31.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:30.523+0000 7fb020e82700 1 -- 172.21.15.37:0/4180415171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 msgr2=0x7fb01c191990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:31.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:30.523+0000 7fb020e82700 1 --2- 172.21.15.37:0/4180415171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c191990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009e50 tx=0x7fb00c00bd30 comp rx=0 tx=0).stop 2024-03-26T06:13:31.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:30.523+0000 7fb020e82700 1 -- 172.21.15.37:0/4180415171 shutdown_connections 2024-03-26T06:13:31.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:30.523+0000 7fb020e82700 1 --2- 172.21.15.37:0/4180415171 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb008038ed0 0x7fb00803b390 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:31.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:30.523+0000 7fb020e82700 1 --2- 172.21.15.37:0/4180415171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01c1067e0 0x7fb01c191990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:31.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:30.523+0000 7fb020e82700 1 -- 172.21.15.37:0/4180415171 >> 172.21.15.37:0/4180415171 conn(0x7fb01c101e40 msgr2=0x7fb01c102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:31.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:30.523+0000 7fb020e82700 1 -- 172.21.15.37:0/4180415171 shutdown_connections 2024-03-26T06:13:31.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:30.524+0000 7fb020e82700 1 -- 172.21.15.37:0/4180415171 wait complete. 2024-03-26T06:13:31.165 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mon service... 2024-03-26T06:13:31.827 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:31 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:31.828 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:31 smithi037 ceph-mon[31974]: Added host smithi037 2024-03-26T06:13:31.828 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:31 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:13:33.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:32 smithi037 ceph-mon[31974]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:13:33.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:32 smithi037 ceph-mon[31974]: Saving service mon spec with placement count:5 2024-03-26T06:13:33.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:32 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:33.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-26T06:13:33.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.624+0000 7ff0bfeaf700 1 Processor -- start 2024-03-26T06:13:33.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.624+0000 7ff0bfeaf700 1 -- start start 2024-03-26T06:13:33.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.624+0000 7ff0bfeaf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8107170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:33.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.624+0000 7ff0bfeaf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8107740 con 0x7ff0b8106d50 2024-03-26T06:13:33.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.624+0000 7ff0bdc4b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:33.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.624+0000 7ff0bdc4b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8107170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60058/0 (socket says 172.21.15.37:60058) 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.624+0000 7ff0bdc4b700 1 -- 172.21.15.37:0/3962963546 learned_addr learned my addr 172.21.15.37:0/3962963546 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.625+0000 7ff0bdc4b700 1 -- 172.21.15.37:0/3962963546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b8107a70 con 0x7ff0b8106d50 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.625+0000 7ff0bdc4b700 1 --2- 172.21.15.37:0/3962963546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8107170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40098a0 tx=0x7ff0b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7eb3e05ead3368 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.625+0000 7ff0bcc49700 1 -- 172.21.15.37:0/3962963546 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0b4004160 con 0x7ff0b8106d50 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.625+0000 7ff0bcc49700 1 -- 172.21.15.37:0/3962963546 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0b4004300 con 0x7ff0b8106d50 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.625+0000 7ff0bcc49700 1 -- 172.21.15.37:0/3962963546 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0b4011b50 con 0x7ff0b8106d50 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.626+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/3962963546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 msgr2=0x7ff0b8107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.626+0000 7ff0bfeaf700 1 --2- 172.21.15.37:0/3962963546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8107170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40098a0 tx=0x7ff0b4009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.626+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/3962963546 shutdown_connections 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.626+0000 7ff0bfeaf700 1 --2- 172.21.15.37:0/3962963546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8107170 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.626+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/3962963546 >> 172.21.15.37:0/3962963546 conn(0x7ff0b81015f0 msgr2=0x7ff0b8103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.626+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/3962963546 shutdown_connections 2024-03-26T06:13:33.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.626+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/3962963546 wait complete. 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.626+0000 7ff0bfeaf700 1 Processor -- start 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.626+0000 7ff0bfeaf700 1 -- start start 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0bfeaf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8106330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0bfeaf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b810ae20 con 0x7ff0b8106d50 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0bdc4b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8106330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0bdc4b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8106330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60074/0 (socket says 172.21.15.37:60074) 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0bdc4b700 1 -- 172.21.15.37:0/4150691242 learned_addr learned my addr 172.21.15.37:0/4150691242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0bdc4b700 1 -- 172.21.15.37:0/4150691242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b819e5e0 con 0x7ff0b8106d50 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0bdc4b700 1 --2- 172.21.15.37:0/4150691242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8106330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4011b60 tx=0x7ff0b4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0aeffd700 1 -- 172.21.15.37:0/4150691242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0b4003970 con 0x7ff0b8106d50 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0aeffd700 1 -- 172.21.15.37:0/4150691242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0b4003b10 con 0x7ff0b8106d50 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/4150691242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b819e7b0 con 0x7ff0b8106d50 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0aeffd700 1 -- 172.21.15.37:0/4150691242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0b4024610 con 0x7ff0b8106d50 2024-03-26T06:13:33.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.627+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/4150691242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b819eb50 con 0x7ff0b8106d50 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.628+0000 7ff0aeffd700 1 -- 172.21.15.37:0/4150691242 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7ff0b401a430 con 0x7ff0b8106d50 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.628+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/4150691242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b8063410 con 0x7ff0b8106d50 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.629+0000 7ff0aeffd700 1 --2- 172.21.15.37:0/4150691242 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7ff0a4038f20 0x7ff0a403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.629+0000 7ff0aeffd700 1 -- 172.21.15.37:0/4150691242 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff0b4011e00 con 0x7ff0b8106d50 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.629+0000 7ff0bd44a700 1 --2- 172.21.15.37:0/4150691242 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7ff0a4038f20 0x7ff0a403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.629+0000 7ff0bd44a700 1 --2- 172.21.15.37:0/4150691242 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7ff0a4038f20 0x7ff0a403b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8006fd0 tx=0x7ff0a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.633+0000 7ff0aeffd700 1 -- 172.21.15.37:0/4150691242 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff0b400e260 con 0x7ff0b8106d50 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.857+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/4150691242 --> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff0b81027f0 con 0x7ff0a4038f20 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.862+0000 7ff0aeffd700 1 -- 172.21.15.37:0/4150691242 <== mgr.14120 v2:172.21.15.37:6800/401419890 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff0b81027f0 con 0x7ff0a4038f20 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.866+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/4150691242 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7ff0a4038f20 msgr2=0x7ff0a403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.866+0000 7ff0bfeaf700 1 --2- 172.21.15.37:0/4150691242 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7ff0a4038f20 0x7ff0a403b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8006fd0 tx=0x7ff0a8006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:33.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.866+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/4150691242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 msgr2=0x7ff0b8106330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:33.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.866+0000 7ff0bfeaf700 1 --2- 172.21.15.37:0/4150691242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8106330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4011b60 tx=0x7ff0b4011c40 comp rx=0 tx=0).stop 2024-03-26T06:13:33.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.866+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/4150691242 shutdown_connections 2024-03-26T06:13:33.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.866+0000 7ff0bfeaf700 1 --2- 172.21.15.37:0/4150691242 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7ff0a4038f20 0x7ff0a403b3e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:33.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.866+0000 7ff0bfeaf700 1 --2- 172.21.15.37:0/4150691242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8106d50 0x7ff0b8106330 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:33.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.866+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/4150691242 >> 172.21.15.37:0/4150691242 conn(0x7ff0b81015f0 msgr2=0x7ff0b81020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:33.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.866+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/4150691242 shutdown_connections 2024-03-26T06:13:33.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:31.866+0000 7ff0bfeaf700 1 -- 172.21.15.37:0/4150691242 wait complete. 2024-03-26T06:13:33.598 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mgr service... 2024-03-26T06:13:34.339 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:34 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:34.340 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:34 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:35.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-26T06:13:35.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.052+0000 7f9fcb7cf700 1 Processor -- start 2024-03-26T06:13:35.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.052+0000 7f9fcb7cf700 1 -- start start 2024-03-26T06:13:35.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.053+0000 7f9fcb7cf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc410a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:35.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.053+0000 7f9fcb7cf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc407ae70 con 0x7f9fc4108340 2024-03-26T06:13:35.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.053+0000 7f9fc956b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc410a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:35.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.053+0000 7f9fc956b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc410a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60082/0 (socket says 172.21.15.37:60082) 2024-03-26T06:13:35.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.053+0000 7f9fc956b700 1 -- 172.21.15.37:0/2310808858 learned_addr learned my addr 172.21.15.37:0/2310808858 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:35.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.053+0000 7f9fc956b700 1 -- 172.21.15.37:0/2310808858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc410ac70 con 0x7f9fc4108340 2024-03-26T06:13:35.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.053+0000 7f9fc956b700 1 --2- 172.21.15.37:0/2310808858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc410a730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00098a0 tx=0x7f9fc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289ec6c4fd9cf0e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:35.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.054+0000 7f9fbbfff700 1 -- 172.21.15.37:0/2310808858 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fc0004160 con 0x7f9fc4108340 2024-03-26T06:13:35.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.054+0000 7f9fbbfff700 1 -- 172.21.15.37:0/2310808858 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc0004300 con 0x7f9fc4108340 2024-03-26T06:13:35.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.054+0000 7f9fbbfff700 1 -- 172.21.15.37:0/2310808858 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fc0011b20 con 0x7f9fc4108340 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.054+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/2310808858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 msgr2=0x7f9fc410a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.054+0000 7f9fcb7cf700 1 --2- 172.21.15.37:0/2310808858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc410a730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00098a0 tx=0x7f9fc0009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.054+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/2310808858 shutdown_connections 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.054+0000 7f9fcb7cf700 1 --2- 172.21.15.37:0/2310808858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc410a730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.054+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/2310808858 >> 172.21.15.37:0/2310808858 conn(0x7f9fc4101df0 msgr2=0x7f9fc4104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.055+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/2310808858 shutdown_connections 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.055+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/2310808858 wait complete. 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.055+0000 7f9fcb7cf700 1 Processor -- start 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.055+0000 7f9fcb7cf700 1 -- start start 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.056+0000 7f9fcb7cf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc41995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.056+0000 7f9fcb7cf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc4106fc0 con 0x7f9fc4108340 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.056+0000 7f9fc956b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc41995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.056+0000 7f9fc956b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc41995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60094/0 (socket says 172.21.15.37:60094) 2024-03-26T06:13:35.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.056+0000 7f9fc956b700 1 -- 172.21.15.37:0/3035276630 learned_addr learned my addr 172.21.15.37:0/3035276630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:35.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.056+0000 7f9fc956b700 1 -- 172.21.15.37:0/3035276630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc4199af0 con 0x7f9fc4108340 2024-03-26T06:13:35.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.056+0000 7f9fc956b700 1 --2- 172.21.15.37:0/3035276630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc41995b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00116a0 tx=0x7f9fc0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:35.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.056+0000 7f9fba7fc700 1 -- 172.21.15.37:0/3035276630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fc0003a40 con 0x7f9fc4108340 2024-03-26T06:13:35.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.056+0000 7f9fba7fc700 1 -- 172.21.15.37:0/3035276630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc0003be0 con 0x7f9fc4108340 2024-03-26T06:13:35.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.056+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/3035276630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc4199de0 con 0x7f9fc4108340 2024-03-26T06:13:35.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.056+0000 7f9fba7fc700 1 -- 172.21.15.37:0/3035276630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fc001a720 con 0x7f9fc4108340 2024-03-26T06:13:35.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.057+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/3035276630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc419ca70 con 0x7f9fc4108340 2024-03-26T06:13:35.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.057+0000 7f9fba7fc700 1 -- 172.21.15.37:0/3035276630 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f9fc0024430 con 0x7f9fc4108340 2024-03-26T06:13:35.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.058+0000 7f9fba7fc700 1 --2- 172.21.15.37:0/3035276630 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f9fb003d350 0x7f9fb003f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:35.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.058+0000 7f9fba7fc700 1 -- 172.21.15.37:0/3035276630 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9fc004d3c0 con 0x7f9fc4108340 2024-03-26T06:13:35.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.058+0000 7f9fc8d6a700 1 --2- 172.21.15.37:0/3035276630 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f9fb003d350 0x7f9fb003f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.058+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/3035276630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc4063410 con 0x7f9fc4108340 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.058+0000 7f9fc8d6a700 1 --2- 172.21.15.37:0/3035276630 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f9fb003d350 0x7f9fb003f810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006fd0 tx=0x7f9fb4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.063+0000 7f9fba7fc700 1 -- 172.21.15.37:0/3035276630 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9fc0011e20 con 0x7f9fc4108340 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.339+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/3035276630 --> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9fc402d8d0 con 0x7f9fb003d350 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.347+0000 7f9fba7fc700 1 -- 172.21.15.37:0/3035276630 <== mgr.14120 v2:172.21.15.37:6800/401419890 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9fc402d8d0 con 0x7f9fb003d350 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.349+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/3035276630 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f9fb003d350 msgr2=0x7f9fb003f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.349+0000 7f9fcb7cf700 1 --2- 172.21.15.37:0/3035276630 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f9fb003d350 0x7f9fb003f810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006fd0 tx=0x7f9fb4006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.349+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/3035276630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 msgr2=0x7f9fc41995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.349+0000 7f9fcb7cf700 1 --2- 172.21.15.37:0/3035276630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc41995b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00116a0 tx=0x7f9fc0011ba0 comp rx=0 tx=0).stop 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.349+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/3035276630 shutdown_connections 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.349+0000 7f9fcb7cf700 1 --2- 172.21.15.37:0/3035276630 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f9fb003d350 0x7f9fb003f810 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.349+0000 7f9fcb7cf700 1 --2- 172.21.15.37:0/3035276630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc4108340 0x7f9fc41995b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.349+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/3035276630 >> 172.21.15.37:0/3035276630 conn(0x7f9fc4101df0 msgr2=0x7f9fc4102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:35.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.349+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/3035276630 shutdown_connections 2024-03-26T06:13:35.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:34.349+0000 7f9fcb7cf700 1 -- 172.21.15.37:0/3035276630 wait complete. 2024-03-26T06:13:35.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:35 smithi037 ceph-mon[31974]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:13:35.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:35 smithi037 ceph-mon[31974]: Saving service mgr spec with placement count:2 2024-03-26T06:13:35.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:35 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:36.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.107+0000 7f21e05dc700 1 Processor -- start 2024-03-26T06:13:36.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.107+0000 7f21e05dc700 1 -- start start 2024-03-26T06:13:36.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.107+0000 7f21e05dc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d8106010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.107+0000 7f21e05dc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d81065e0 con 0x7f21d8105bf0 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.107+0000 7f21de378700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d8106010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.108+0000 7f21de378700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d8106010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60098/0 (socket says 172.21.15.37:60098) 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.108+0000 7f21de378700 1 -- 172.21.15.37:0/27954607 learned_addr learned my addr 172.21.15.37:0/27954607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.108+0000 7f21de378700 1 -- 172.21.15.37:0/27954607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8108b80 con 0x7f21d8105bf0 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.108+0000 7f21de378700 1 --2- 172.21.15.37:0/27954607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d8106010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f21c80098a0 tx=0x7f21c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b878d10d293ab7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.108+0000 7f21dd376700 1 -- 172.21.15.37:0/27954607 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21c8004160 con 0x7f21d8105bf0 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.108+0000 7f21dd376700 1 -- 172.21.15.37:0/27954607 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21c8004300 con 0x7f21d8105bf0 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.108+0000 7f21dd376700 1 -- 172.21.15.37:0/27954607 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21c8011b20 con 0x7f21d8105bf0 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.109+0000 7f21e05dc700 1 -- 172.21.15.37:0/27954607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 msgr2=0x7f21d8106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.109+0000 7f21e05dc700 1 --2- 172.21.15.37:0/27954607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d8106010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f21c80098a0 tx=0x7f21c8009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:36.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.109+0000 7f21e05dc700 1 -- 172.21.15.37:0/27954607 shutdown_connections 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.109+0000 7f21e05dc700 1 --2- 172.21.15.37:0/27954607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d8106010 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.109+0000 7f21e05dc700 1 -- 172.21.15.37:0/27954607 >> 172.21.15.37:0/27954607 conn(0x7f21d8101540 msgr2=0x7f21d8103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.109+0000 7f21e05dc700 1 -- 172.21.15.37:0/27954607 shutdown_connections 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.109+0000 7f21e05dc700 1 -- 172.21.15.37:0/27954607 wait complete. 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21e05dc700 1 Processor -- start 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21e05dc700 1 -- start start 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21e05dc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21e05dc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d810bf30 con 0x7f21d8105bf0 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21de378700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21de378700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d819b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60116/0 (socket says 172.21.15.37:60116) 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21de378700 1 -- 172.21.15.37:0/894771649 learned_addr learned my addr 172.21.15.37:0/894771649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21de378700 1 -- 172.21.15.37:0/894771649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d819bdb0 con 0x7f21d8105bf0 2024-03-26T06:13:36.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21de378700 1 --2- 172.21.15.37:0/894771649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d819b870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f21c80116a0 tx=0x7f21c8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21cf7fe700 1 -- 172.21.15.37:0/894771649 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21c8003a40 con 0x7f21d8105bf0 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21cf7fe700 1 -- 172.21.15.37:0/894771649 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21c8003be0 con 0x7f21d8105bf0 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21e05dc700 1 -- 172.21.15.37:0/894771649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d819c040 con 0x7f21d8105bf0 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.110+0000 7f21cf7fe700 1 -- 172.21.15.37:0/894771649 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21c801a720 con 0x7f21d8105bf0 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.111+0000 7f21e05dc700 1 -- 172.21.15.37:0/894771649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d819c4a0 con 0x7f21d8105bf0 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.111+0000 7f21cf7fe700 1 -- 172.21.15.37:0/894771649 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f21c801aa20 con 0x7f21d8105bf0 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.112+0000 7f21cf7fe700 1 --2- 172.21.15.37:0/894771649 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f21c4038ed0 0x7f21c403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.112+0000 7f21e05dc700 1 -- 172.21.15.37:0/894771649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d8068ec0 con 0x7f21d8105bf0 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.112+0000 7f21cf7fe700 1 -- 172.21.15.37:0/894771649 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f21c804d620 con 0x7f21d8105bf0 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.112+0000 7f21ddb77700 1 --2- 172.21.15.37:0/894771649 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f21c4038ed0 0x7f21c403b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.112+0000 7f21ddb77700 1 --2- 172.21.15.37:0/894771649 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f21c4038ed0 0x7f21c403b390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21d4006fd0 tx=0x7f21d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:36.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.117+0000 7f21cf7fe700 1 -- 172.21.15.37:0/894771649 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f21c8011e20 con 0x7f21d8105bf0 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.320+0000 7f21e05dc700 1 -- 172.21.15.37:0/894771649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f21d819ed30 con 0x7f21d8105bf0 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.323+0000 7f21cf7fe700 1 -- 172.21.15.37:0/894771649 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f21c80247b0 con 0x7f21d8105bf0 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.325+0000 7f21e05dc700 1 -- 172.21.15.37:0/894771649 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f21c4038ed0 msgr2=0x7f21c403b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.325+0000 7f21e05dc700 1 --2- 172.21.15.37:0/894771649 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f21c4038ed0 0x7f21c403b390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21d4006fd0 tx=0x7f21d4006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.325+0000 7f21e05dc700 1 -- 172.21.15.37:0/894771649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 msgr2=0x7f21d819b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.325+0000 7f21e05dc700 1 --2- 172.21.15.37:0/894771649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d819b870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f21c80116a0 tx=0x7f21c8011ba0 comp rx=0 tx=0).stop 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.325+0000 7f21e05dc700 1 -- 172.21.15.37:0/894771649 shutdown_connections 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.325+0000 7f21e05dc700 1 --2- 172.21.15.37:0/894771649 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f21c4038ed0 0x7f21c403b390 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.325+0000 7f21e05dc700 1 --2- 172.21.15.37:0/894771649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d8105bf0 0x7f21d819b870 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.325+0000 7f21e05dc700 1 -- 172.21.15.37:0/894771649 >> 172.21.15.37:0/894771649 conn(0x7f21d8101540 msgr2=0x7f21d8101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.325+0000 7f21e05dc700 1 -- 172.21.15.37:0/894771649 shutdown_connections 2024-03-26T06:13:36.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:36.325+0000 7f21e05dc700 1 -- 172.21.15.37:0/894771649 wait complete. 2024-03-26T06:13:37.164 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:36 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:37.164 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:36 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/894771649' entity='client.admin' 2024-03-26T06:13:38.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.368+0000 7fb69d1b4700 1 Processor -- start 2024-03-26T06:13:38.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.368+0000 7fb69d1b4700 1 -- start start 2024-03-26T06:13:38.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.368+0000 7fb69d1b4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698106c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.368+0000 7fb69d1b4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6981071d0 con 0x7fb6981067e0 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.368+0000 7fb696d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698106c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.369+0000 7fb696d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698106c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60128/0 (socket says 172.21.15.37:60128) 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.369+0000 7fb696d9d700 1 -- 172.21.15.37:0/4277223791 learned_addr learned my addr 172.21.15.37:0/4277223791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.369+0000 7fb696d9d700 1 -- 172.21.15.37:0/4277223791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6981079e0 con 0x7fb6981067e0 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.369+0000 7fb696d9d700 1 --2- 172.21.15.37:0/4277223791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698106c00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0098a0 tx=0x7fb68c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a580baaab6e4a4b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.369+0000 7fb695d9b700 1 -- 172.21.15.37:0/4277223791 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb68c004160 con 0x7fb6981067e0 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.369+0000 7fb695d9b700 1 -- 172.21.15.37:0/4277223791 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb68c004300 con 0x7fb6981067e0 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.369+0000 7fb695d9b700 1 -- 172.21.15.37:0/4277223791 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb68c011b20 con 0x7fb6981067e0 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.370+0000 7fb69d1b4700 1 -- 172.21.15.37:0/4277223791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 msgr2=0x7fb698106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.370+0000 7fb69d1b4700 1 --2- 172.21.15.37:0/4277223791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698106c00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0098a0 tx=0x7fb68c009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.370+0000 7fb69d1b4700 1 -- 172.21.15.37:0/4277223791 shutdown_connections 2024-03-26T06:13:38.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.370+0000 7fb69d1b4700 1 --2- 172.21.15.37:0/4277223791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698106c00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.370+0000 7fb69d1b4700 1 -- 172.21.15.37:0/4277223791 >> 172.21.15.37:0/4277223791 conn(0x7fb698101e40 msgr2=0x7fb698104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.370+0000 7fb69d1b4700 1 -- 172.21.15.37:0/4277223791 shutdown_connections 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.370+0000 7fb69d1b4700 1 -- 172.21.15.37:0/4277223791 wait complete. 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.371+0000 7fb69d1b4700 1 Processor -- start 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.371+0000 7fb69d1b4700 1 -- start start 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.371+0000 7fb69d1b4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698191990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.371+0000 7fb69d1b4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69810ad90 con 0x7fb6981067e0 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.372+0000 7fb696d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698191990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.372+0000 7fb696d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698191990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60136/0 (socket says 172.21.15.37:60136) 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.372+0000 7fb696d9d700 1 -- 172.21.15.37:0/2285429102 learned_addr learned my addr 172.21.15.37:0/2285429102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.372+0000 7fb696d9d700 1 -- 172.21.15.37:0/2285429102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb698191ed0 con 0x7fb6981067e0 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.372+0000 7fb696d9d700 1 --2- 172.21.15.37:0/2285429102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698191990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0116a0 tx=0x7fb68c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:38.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.372+0000 7fb67ffff700 1 -- 172.21.15.37:0/2285429102 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb68c003a40 con 0x7fb6981067e0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.372+0000 7fb69d1b4700 1 -- 172.21.15.37:0/2285429102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6981900c0 con 0x7fb6981067e0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.372+0000 7fb67ffff700 1 -- 172.21.15.37:0/2285429102 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb68c003be0 con 0x7fb6981067e0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.373+0000 7fb67ffff700 1 -- 172.21.15.37:0/2285429102 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb68c01a720 con 0x7fb6981067e0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.373+0000 7fb69d1b4700 1 -- 172.21.15.37:0/2285429102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb698190580 con 0x7fb6981067e0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.373+0000 7fb67ffff700 1 -- 172.21.15.37:0/2285429102 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fb68c01aa70 con 0x7fb6981067e0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.373+0000 7fb67ffff700 1 --2- 172.21.15.37:0/2285429102 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb680038f20 0x7fb68003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.373+0000 7fb67ffff700 1 -- 172.21.15.37:0/2285429102 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb68c024840 con 0x7fb6981067e0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.374+0000 7fb69659c700 1 --2- 172.21.15.37:0/2285429102 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb680038f20 0x7fb68003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.374+0000 7fb69659c700 1 --2- 172.21.15.37:0/2285429102 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb680038f20 0x7fb68003b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb688006fd0 tx=0x7fb688006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.374+0000 7fb69d1b4700 1 -- 172.21.15.37:0/2285429102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb698063410 con 0x7fb6981067e0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.379+0000 7fb67ffff700 1 -- 172.21.15.37:0/2285429102 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb68c011e20 con 0x7fb6981067e0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.588+0000 7fb69d1b4700 1 -- 172.21.15.37:0/2285429102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fb698195580 con 0x7fb6981067e0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.591+0000 7fb67ffff700 1 -- 172.21.15.37:0/2285429102 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fb68c020240 con 0x7fb6981067e0 2024-03-26T06:13:38.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.593+0000 7fb69d1b4700 1 -- 172.21.15.37:0/2285429102 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb680038f20 msgr2=0x7fb68003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:38.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.593+0000 7fb69d1b4700 1 --2- 172.21.15.37:0/2285429102 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb680038f20 0x7fb68003b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb688006fd0 tx=0x7fb688006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:38.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.593+0000 7fb69d1b4700 1 -- 172.21.15.37:0/2285429102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 msgr2=0x7fb698191990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:38.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.593+0000 7fb69d1b4700 1 --2- 172.21.15.37:0/2285429102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698191990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0116a0 tx=0x7fb68c011ba0 comp rx=0 tx=0).stop 2024-03-26T06:13:38.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.593+0000 7fb69d1b4700 1 -- 172.21.15.37:0/2285429102 shutdown_connections 2024-03-26T06:13:38.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.593+0000 7fb69d1b4700 1 --2- 172.21.15.37:0/2285429102 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fb680038f20 0x7fb68003b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:38.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.593+0000 7fb69d1b4700 1 --2- 172.21.15.37:0/2285429102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6981067e0 0x7fb698191990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:38.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.593+0000 7fb69d1b4700 1 -- 172.21.15.37:0/2285429102 >> 172.21.15.37:0/2285429102 conn(0x7fb698101e40 msgr2=0x7fb698102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:38.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.593+0000 7fb69d1b4700 1 -- 172.21.15.37:0/2285429102 shutdown_connections 2024-03-26T06:13:38.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:37.593+0000 7fb69d1b4700 1 -- 172.21.15.37:0/2285429102 wait complete. 2024-03-26T06:13:38.154 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling the dashboard module... 2024-03-26T06:13:38.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:38 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2285429102' entity='client.admin' 2024-03-26T06:13:38.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:38 smithi037 ceph-mon[31974]: from='mgr.14120 172.21.15.37:0/3270133098' entity='mgr.x' 2024-03-26T06:13:38.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:38 smithi037 ceph-mon[31974]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:13:39.769 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:39 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2849564494' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-26T06:13:39.769 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:39 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: ignoring --setuser ceph since I am not root 2024-03-26T06:13:39.769 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:39 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: ignoring --setgroup ceph since I am not root 2024-03-26T06:13:40.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:40 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:40.183+0000 7f67f5fa1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T06:13:40.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:40 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:40.279+0000 7f67f5fa1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T06:13:40.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.615+0000 7f7f007c3700 1 Processor -- start 2024-03-26T06:13:40.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.615+0000 7f7f007c3700 1 -- start start 2024-03-26T06:13:40.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.615+0000 7f7f007c3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef807b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:40.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.615+0000 7f7f007c3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef807b8b0 con 0x7f7ef807ce30 2024-03-26T06:13:40.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.615+0000 7f7efe55f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef807b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:40.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.615+0000 7f7efe55f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef807b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60582/0 (socket says 172.21.15.37:60582) 2024-03-26T06:13:40.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.615+0000 7f7efe55f700 1 -- 172.21.15.37:0/4133611880 learned_addr learned my addr 172.21.15.37:0/4133611880 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:40.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.615+0000 7f7efe55f700 1 -- 172.21.15.37:0/4133611880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef807ba30 con 0x7f7ef807ce30 2024-03-26T06:13:40.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.616+0000 7f7efe55f700 1 --2- 172.21.15.37:0/4133611880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef807b2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40098a0 tx=0x7f7ef4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f849313ab95d3cdf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.616+0000 7f7efd55d700 1 -- 172.21.15.37:0/4133611880 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ef4004160 con 0x7f7ef807ce30 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.616+0000 7f7efd55d700 1 -- 172.21.15.37:0/4133611880 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ef4004300 con 0x7f7ef807ce30 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.616+0000 7f7efd55d700 1 -- 172.21.15.37:0/4133611880 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ef4011b20 con 0x7f7ef807ce30 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.616+0000 7f7f007c3700 1 -- 172.21.15.37:0/4133611880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 msgr2=0x7f7ef807b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.616+0000 7f7f007c3700 1 --2- 172.21.15.37:0/4133611880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef807b2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40098a0 tx=0x7f7ef4009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.616+0000 7f7f007c3700 1 -- 172.21.15.37:0/4133611880 shutdown_connections 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.616+0000 7f7f007c3700 1 --2- 172.21.15.37:0/4133611880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef807b2e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.616+0000 7f7f007c3700 1 -- 172.21.15.37:0/4133611880 >> 172.21.15.37:0/4133611880 conn(0x7f7ef8102f40 msgr2=0x7f7ef8105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.616+0000 7f7f007c3700 1 -- 172.21.15.37:0/4133611880 shutdown_connections 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.616+0000 7f7f007c3700 1 -- 172.21.15.37:0/4133611880 wait complete. 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.617+0000 7f7f007c3700 1 Processor -- start 2024-03-26T06:13:40.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.617+0000 7f7f007c3700 1 -- start start 2024-03-26T06:13:40.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.617+0000 7f7f007c3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:40.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.617+0000 7f7f007c3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef810df40 con 0x7f7ef807ce30 2024-03-26T06:13:40.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.617+0000 7f7efe55f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:40.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.617+0000 7f7efe55f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef819da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60598/0 (socket says 172.21.15.37:60598) 2024-03-26T06:13:40.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.617+0000 7f7efe55f700 1 -- 172.21.15.37:0/2849564494 learned_addr learned my addr 172.21.15.37:0/2849564494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:40.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.617+0000 7f7efe55f700 1 -- 172.21.15.37:0/2849564494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef819df90 con 0x7f7ef807ce30 2024-03-26T06:13:40.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.617+0000 7f7efe55f700 1 --2- 172.21.15.37:0/2849564494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef819da50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4011b60 tx=0x7f7ef4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:40.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.618+0000 7f7eef7fe700 1 -- 172.21.15.37:0/2849564494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ef4003b60 con 0x7f7ef807ce30 2024-03-26T06:13:40.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.618+0000 7f7eef7fe700 1 -- 172.21.15.37:0/2849564494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ef401a910 con 0x7f7ef807ce30 2024-03-26T06:13:40.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.618+0000 7f7eef7fe700 1 -- 172.21.15.37:0/2849564494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ef4024930 con 0x7f7ef807ce30 2024-03-26T06:13:40.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.618+0000 7f7f007c3700 1 -- 172.21.15.37:0/2849564494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef819e220 con 0x7f7ef807ce30 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.618+0000 7f7f007c3700 1 -- 172.21.15.37:0/2849564494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef819e680 con 0x7f7ef807ce30 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.618+0000 7f7eef7fe700 1 -- 172.21.15.37:0/2849564494 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f7ef4022020 con 0x7f7ef807ce30 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.619+0000 7f7eef7fe700 1 --2- 172.21.15.37:0/2849564494 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f7ee4038f70 0x7f7ee403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.619+0000 7f7f007c3700 1 -- 172.21.15.37:0/2849564494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef8063410 con 0x7f7ef807ce30 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.619+0000 7f7eef7fe700 1 -- 172.21.15.37:0/2849564494 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7ef404d030 con 0x7f7ef807ce30 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.619+0000 7f7efdd5e700 1 --2- 172.21.15.37:0/2849564494 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f7ee4038f70 0x7f7ee403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.619+0000 7f7efdd5e700 1 --2- 172.21.15.37:0/2849564494 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f7ee4038f70 0x7f7ee403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8006fd0 tx=0x7f7ee8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.624+0000 7f7eef7fe700 1 -- 172.21.15.37:0/2849564494 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7ef400b3e0 con 0x7f7ef807ce30 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:38.903+0000 7f7f007c3700 1 -- 172.21.15.37:0/2849564494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f7ef81a1050 con 0x7f7ef807ce30 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.595+0000 7f7eef7fe700 1 -- 172.21.15.37:0/2849564494 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f7ef4016260 con 0x7f7ef807ce30 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.597+0000 7f7eef7fe700 1 -- 172.21.15.37:0/2849564494 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f7ef4024ad0 con 0x7f7ef807ce30 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.598+0000 7f7f007c3700 1 -- 172.21.15.37:0/2849564494 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f7ee4038f70 msgr2=0x7f7ee403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:40.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.598+0000 7f7f007c3700 1 --2- 172.21.15.37:0/2849564494 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f7ee4038f70 0x7f7ee403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8006fd0 tx=0x7f7ee8006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:40.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.598+0000 7f7f007c3700 1 -- 172.21.15.37:0/2849564494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 msgr2=0x7f7ef819da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:40.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.598+0000 7f7f007c3700 1 --2- 172.21.15.37:0/2849564494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef819da50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4011b60 tx=0x7f7ef4011c40 comp rx=0 tx=0).stop 2024-03-26T06:13:40.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.598+0000 7f7f007c3700 1 -- 172.21.15.37:0/2849564494 shutdown_connections 2024-03-26T06:13:40.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.598+0000 7f7f007c3700 1 --2- 172.21.15.37:0/2849564494 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7f7ee4038f70 0x7f7ee403b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:40.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.598+0000 7f7f007c3700 1 --2- 172.21.15.37:0/2849564494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807ce30 0x7f7ef819da50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:40.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.598+0000 7f7f007c3700 1 -- 172.21.15.37:0/2849564494 >> 172.21.15.37:0/2849564494 conn(0x7f7ef8102f40 msgr2=0x7f7ef81039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:40.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.599+0000 7f7f007c3700 1 -- 172.21.15.37:0/2849564494 shutdown_connections 2024-03-26T06:13:40.368 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:39.599+0000 7f7f007c3700 1 -- 172.21.15.37:0/2849564494 wait complete. 2024-03-26T06:13:40.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:40 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2849564494' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-26T06:13:40.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:40 smithi037 ceph-mon[31974]: mgrmap e9: x(active, since 21s) 2024-03-26T06:13:41.296 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:40 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:40.955+0000 7f67f5fa1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T06:13:41.296 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:41.050+0000 7f67f5fa1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T06:13:41.296 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:41.198+0000 7f67f5fa1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T06:13:41.296 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:41.296+0000 7f67f5fa1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T06:13:41.595 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:41.383+0000 7f67f5fa1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T06:13:41.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-26T06:13:41.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-26T06:13:41.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T06:13:41.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T06:13:41.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.777+0000 7fddeb238700 1 Processor -- start 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.777+0000 7fddeb238700 1 -- start start 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.777+0000 7fddeb238700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde4107d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.777+0000 7fddeb238700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde4108300 con 0x7fdde4107910 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.777+0000 7fdde8fd4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde4107d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.777+0000 7fdde8fd4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde4107d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60616/0 (socket says 172.21.15.37:60616) 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.777+0000 7fdde8fd4700 1 -- 172.21.15.37:0/966724438 learned_addr learned my addr 172.21.15.37:0/966724438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.778+0000 7fdde8fd4700 1 -- 172.21.15.37:0/966724438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde4108ac0 con 0x7fdde4107910 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.778+0000 7fdde8fd4700 1 --2- 172.21.15.37:0/966724438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde4107d30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fddd80098a0 tx=0x7fddd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7736e8fb90e244 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.778+0000 7fdde37fe700 1 -- 172.21.15.37:0/966724438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddd8004160 con 0x7fdde4107910 2024-03-26T06:13:41.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.778+0000 7fdde37fe700 1 -- 172.21.15.37:0/966724438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fddd8004300 con 0x7fdde4107910 2024-03-26T06:13:41.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.778+0000 7fdde37fe700 1 -- 172.21.15.37:0/966724438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddd8011b20 con 0x7fdde4107910 2024-03-26T06:13:41.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.779+0000 7fddeb238700 1 -- 172.21.15.37:0/966724438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 msgr2=0x7fdde4107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:41.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.779+0000 7fddeb238700 1 --2- 172.21.15.37:0/966724438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde4107d30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fddd80098a0 tx=0x7fddd8009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:41.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.779+0000 7fddeb238700 1 -- 172.21.15.37:0/966724438 shutdown_connections 2024-03-26T06:13:41.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.779+0000 7fddeb238700 1 --2- 172.21.15.37:0/966724438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde4107d30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:41.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.779+0000 7fddeb238700 1 -- 172.21.15.37:0/966724438 >> 172.21.15.37:0/966724438 conn(0x7fdde4102f50 msgr2=0x7fdde4105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:41.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.779+0000 7fddeb238700 1 -- 172.21.15.37:0/966724438 shutdown_connections 2024-03-26T06:13:41.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.779+0000 7fddeb238700 1 -- 172.21.15.37:0/966724438 wait complete. 2024-03-26T06:13:41.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.779+0000 7fddeb238700 1 Processor -- start 2024-03-26T06:13:41.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fddeb238700 1 -- start start 2024-03-26T06:13:41.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fddeb238700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde407cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:41.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fddeb238700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde410be70 con 0x7fdde4107910 2024-03-26T06:13:41.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fdde8fd4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde407cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:41.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fdde8fd4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde407cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60628/0 (socket says 172.21.15.37:60628) 2024-03-26T06:13:41.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fdde8fd4700 1 -- 172.21.15.37:0/4016498613 learned_addr learned my addr 172.21.15.37:0/4016498613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:41.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fdde8fd4700 1 -- 172.21.15.37:0/4016498613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde407d090 con 0x7fdde4107910 2024-03-26T06:13:41.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fdde8fd4700 1 --2- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde407cb50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fddd80116a0 tx=0x7fddd8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:41.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fdde1ffb700 1 -- 172.21.15.37:0/4016498613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddd8003a40 con 0x7fdde4107910 2024-03-26T06:13:41.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fddeb238700 1 -- 172.21.15.37:0/4016498613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde407b280 con 0x7fdde4107910 2024-03-26T06:13:41.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fdde1ffb700 1 -- 172.21.15.37:0/4016498613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fddd8003be0 con 0x7fdde4107910 2024-03-26T06:13:41.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.780+0000 7fdde1ffb700 1 -- 172.21.15.37:0/4016498613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddd801a720 con 0x7fdde4107910 2024-03-26T06:13:41.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.781+0000 7fddeb238700 1 -- 172.21.15.37:0/4016498613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde407b740 con 0x7fdde4107910 2024-03-26T06:13:41.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.781+0000 7fdde1ffb700 1 -- 172.21.15.37:0/4016498613 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fddd801aa20 con 0x7fdde4107910 2024-03-26T06:13:41.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.782+0000 7fdde1ffb700 1 --2- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fddd4038f20 0x7fddd403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:41.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.782+0000 7fddeb238700 1 -- 172.21.15.37:0/4016498613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde4068ec0 con 0x7fdde4107910 2024-03-26T06:13:41.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.782+0000 7fdde1ffb700 1 -- 172.21.15.37:0/4016498613 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fddd804d510 con 0x7fdde4107910 2024-03-26T06:13:41.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.782+0000 7fdde3fff700 1 -- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fddd4038f20 msgr2=0x7fddd403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/401419890 2024-03-26T06:13:41.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.782+0000 7fdde3fff700 1 --2- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fddd4038f20 0x7fddd403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T06:13:41.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.787+0000 7fdde1ffb700 1 -- 172.21.15.37:0/4016498613 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fddd8011e20 con 0x7fdde4107910 2024-03-26T06:13:41.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.983+0000 7fdde3fff700 1 -- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fddd4038f20 msgr2=0x7fddd403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/401419890 2024-03-26T06:13:41.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:40.983+0000 7fdde3fff700 1 --2- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fddd4038f20 0x7fddd403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T06:13:41.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.051+0000 7fddeb238700 1 -- 172.21.15.37:0/4016498613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fdde407bdc0 con 0x7fdde4107910 2024-03-26T06:13:41.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.052+0000 7fdde1ffb700 1 -- 172.21.15.37:0/4016498613 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fddd800f2e0 con 0x7fdde4107910 2024-03-26T06:13:41.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.054+0000 7fddeb238700 1 -- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fddd4038f20 msgr2=0x7fddd403b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T06:13:41.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.054+0000 7fddeb238700 1 --2- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fddd4038f20 0x7fddd403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:41.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.054+0000 7fddeb238700 1 -- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 msgr2=0x7fdde407cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:41.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.054+0000 7fddeb238700 1 --2- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde407cb50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fddd80116a0 tx=0x7fddd8011ba0 comp rx=0 tx=0).stop 2024-03-26T06:13:41.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.054+0000 7fddeb238700 1 -- 172.21.15.37:0/4016498613 shutdown_connections 2024-03-26T06:13:41.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.054+0000 7fddeb238700 1 --2- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fddd4038f20 0x7fddd403b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:41.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.054+0000 7fddeb238700 1 --2- 172.21.15.37:0/4016498613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4107910 0x7fdde407cb50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:41.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.054+0000 7fddeb238700 1 -- 172.21.15.37:0/4016498613 >> 172.21.15.37:0/4016498613 conn(0x7fdde4102f50 msgr2=0x7fdde4103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:41.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.054+0000 7fddeb238700 1 -- 172.21.15.37:0/4016498613 shutdown_connections 2024-03-26T06:13:41.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:41.054+0000 7fddeb238700 1 -- 172.21.15.37:0/4016498613 wait complete. 2024-03-26T06:13:41.669 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-03-26T06:13:41.669 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 9... 2024-03-26T06:13:41.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:41 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/4016498613' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T06:13:41.860 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:41.653+0000 7f67f5fa1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T06:13:41.861 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:41.769+0000 7f67f5fa1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T06:13:41.861 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:41.859+0000 7f67f5fa1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T06:13:42.110 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:41.961+0000 7f67f5fa1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T06:13:42.447 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:42 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:42.143+0000 7f67f5fa1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T06:13:43.447 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:43 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:43.134+0000 7f67f5fa1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T06:13:43.447 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:43 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:43.269+0000 7f67f5fa1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T06:13:43.895 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:43 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:43.608+0000 7f67f5fa1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T06:13:43.895 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:43 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:43.701+0000 7f67f5fa1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T06:13:44.197 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:43 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:43.906+0000 7f67f5fa1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:13:44.197 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:44 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:44.099+0000 7f67f5fa1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T06:13:44.660 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:44 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:44.394+0000 7f67f5fa1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:13:44.660 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:44 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:44.660+0000 7f67f5fa1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T06:13:44.947 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:44 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:44.906+0000 7f67f5fa1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T06:13:46.197 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:45 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:45.739+0000 7f67f5fa1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T06:13:46.651 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:46.325+0000 7f67f5fa1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T06:13:46.651 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:46.427+0000 7f67f5fa1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T06:13:46.652 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:13:46.651+0000 7f67f5fa1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T06:13:46.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: Active manager daemon x restarted 2024-03-26T06:13:46.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: Activating manager daemon x 2024-03-26T06:13:46.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: osdmap e3: 0 total, 0 up, 0 in 2024-03-26T06:13:46.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: mgrmap e10: x(active, starting, since 0.00343629s) 2024-03-26T06:13:46.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:13:46.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T06:13:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T06:13:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T06:13:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T06:13:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: Manager daemon x is now available 2024-03-26T06:13:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:13:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:13:47.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T06:13:47.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T06:13:47.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:47 smithi037 ceph-mon[31974]: mgrmap e11: x(active, since 1.00631s) 2024-03-26T06:13:48.490 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-26T06:13:48.490 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-26T06:13:48.490 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T06:13:48.490 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-26T06:13:48.490 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.110+0000 7fdbc6e46700 1 Processor -- start 2024-03-26T06:13:48.490 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.110+0000 7fdbc6e46700 1 -- start start 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.110+0000 7fdbc6e46700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc0106580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.110+0000 7fdbc6e46700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc007ae50 con 0x7fdbc0106160 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.111+0000 7fdbc4be2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc0106580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.111+0000 7fdbc4be2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc0106580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60630/0 (socket says 172.21.15.37:60630) 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.111+0000 7fdbc4be2700 1 -- 172.21.15.37:0/554315964 learned_addr learned my addr 172.21.15.37:0/554315964 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.111+0000 7fdbc4be2700 1 -- 172.21.15.37:0/554315964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc0106ac0 con 0x7fdbc0106160 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.111+0000 7fdbc4be2700 1 --2- 172.21.15.37:0/554315964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc0106580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40098a0 tx=0x7fdbb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabb0dae32161344 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.111+0000 7fdbbf7fe700 1 -- 172.21.15.37:0/554315964 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb4004160 con 0x7fdbc0106160 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.112+0000 7fdbbf7fe700 1 -- 172.21.15.37:0/554315964 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdbb4004300 con 0x7fdbc0106160 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.112+0000 7fdbbf7fe700 1 -- 172.21.15.37:0/554315964 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb4011b20 con 0x7fdbc0106160 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.112+0000 7fdbc6e46700 1 -- 172.21.15.37:0/554315964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 msgr2=0x7fdbc0106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.112+0000 7fdbc6e46700 1 --2- 172.21.15.37:0/554315964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc0106580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40098a0 tx=0x7fdbb4009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.112+0000 7fdbc6e46700 1 -- 172.21.15.37:0/554315964 shutdown_connections 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.112+0000 7fdbc6e46700 1 --2- 172.21.15.37:0/554315964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc0106580 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.112+0000 7fdbc6e46700 1 -- 172.21.15.37:0/554315964 >> 172.21.15.37:0/554315964 conn(0x7fdbc0101e80 msgr2=0x7fdbc01042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.112+0000 7fdbc6e46700 1 -- 172.21.15.37:0/554315964 shutdown_connections 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.113+0000 7fdbc6e46700 1 -- 172.21.15.37:0/554315964 wait complete. 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.113+0000 7fdbc6e46700 1 Processor -- start 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.113+0000 7fdbc6e46700 1 -- start start 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.113+0000 7fdbc6e46700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc019da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.113+0000 7fdbc6e46700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc0107b90 con 0x7fdbc0106160 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.114+0000 7fdbc4be2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc019da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.114+0000 7fdbc4be2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc019da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60646/0 (socket says 172.21.15.37:60646) 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.114+0000 7fdbc4be2700 1 -- 172.21.15.37:0/2238619656 learned_addr learned my addr 172.21.15.37:0/2238619656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.114+0000 7fdbc4be2700 1 -- 172.21.15.37:0/2238619656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc019df80 con 0x7fdbc0106160 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.114+0000 7fdbc4be2700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc019da40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40116a0 tx=0x7fdbb4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:48.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.114+0000 7fdbbdffb700 1 -- 172.21.15.37:0/2238619656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb4003a40 con 0x7fdbc0106160 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.114+0000 7fdbbdffb700 1 -- 172.21.15.37:0/2238619656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdbb4003be0 con 0x7fdbc0106160 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.114+0000 7fdbbdffb700 1 -- 172.21.15.37:0/2238619656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb401a750 con 0x7fdbc0106160 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.114+0000 7fdbc6e46700 1 -- 172.21.15.37:0/2238619656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc019e210 con 0x7fdbc0106160 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.114+0000 7fdbc6e46700 1 -- 172.21.15.37:0/2238619656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc019e5b0 con 0x7fdbc0106160 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.115+0000 7fdbbdffb700 1 -- 172.21.15.37:0/2238619656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fdbb401aa00 con 0x7fdbc0106160 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.115+0000 7fdbbdffb700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 0x7fdbb003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.115+0000 7fdbc6e46700 1 -- 172.21.15.37:0/2238619656 --> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbc0068ec0 con 0x7fdbb0038f80 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.115+0000 7fdbbdffb700 1 -- 172.21.15.37:0/2238619656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdbb404d420 con 0x7fdbc0106160 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.115+0000 7fdbbffff700 1 -- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 msgr2=0x7fdbb003b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/401419890 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.115+0000 7fdbbffff700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 0x7fdbb003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.316+0000 7fdbbffff700 1 -- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 msgr2=0x7fdbb003b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/401419890 2024-03-26T06:13:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.316+0000 7fdbbffff700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 0x7fdbb003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.717+0000 7fdbbffff700 1 -- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 msgr2=0x7fdbb003b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/401419890 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:42.717+0000 7fdbbffff700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 0x7fdbb003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:43.518+0000 7fdbbffff700 1 -- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 msgr2=0x7fdbb003b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/401419890 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:43.518+0000 7fdbbffff700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 0x7fdbb003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:45.119+0000 7fdbbffff700 1 -- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 msgr2=0x7fdbb003b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/401419890 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:45.119+0000 7fdbbffff700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 0x7fdbb003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:46.667+0000 7fdbbdffb700 1 -- 172.21.15.37:0/2238619656 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 10) v1 ==== 45886+0+0 (secure 0 0 0) 0x7fdbb402d910 con 0x7fdbc0106160 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:46.667+0000 7fdbbdffb700 1 -- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 msgr2=0x7fdbb003b440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:46.667+0000 7fdbbdffb700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 0x7fdbb003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.668+0000 7fdbbdffb700 1 -- 172.21.15.37:0/2238619656 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7fdbb404ed90 con 0x7fdbc0106160 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.669+0000 7fdbbdffb700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdbb003c840 0x7fdbb003ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.669+0000 7fdbbdffb700 1 -- 172.21.15.37:0/2238619656 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbb4001d30 con 0x7fdbb003c840 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.669+0000 7fdbbffff700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdbb003c840 0x7fdbb003ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.670+0000 7fdbbffff700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdbb003c840 0x7fdbb003ec30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdbac003a10 tx=0x7fdbac009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.670+0000 7fdbbdffb700 1 -- 172.21.15.37:0/2238619656 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fdbb4001d30 con 0x7fdbb003c840 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.677+0000 7fdbc6e46700 1 -- 172.21.15.37:0/2238619656 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fdbc0062220 con 0x7fdbb003c840 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.677+0000 7fdbbdffb700 1 -- 172.21.15.37:0/2238619656 <== mgr.14152 v2:172.21.15.37:6800/641545212 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fdbc0062220 con 0x7fdbb003c840 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.678+0000 7fdbc6e46700 1 -- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdbb003c840 msgr2=0x7fdbb003ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.678+0000 7fdbc6e46700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdbb003c840 0x7fdbb003ec30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdbac003a10 tx=0x7fdbac009270 comp rx=0 tx=0).stop 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.678+0000 7fdbc6e46700 1 -- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 msgr2=0x7fdbc019da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.678+0000 7fdbc6e46700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc019da40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40116a0 tx=0x7fdbb4011ba0 comp rx=0 tx=0).stop 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.678+0000 7fdbc6e46700 1 -- 172.21.15.37:0/2238619656 shutdown_connections 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.678+0000 7fdbc6e46700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdbb003c840 0x7fdbb003ec30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.678+0000 7fdbc6e46700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbc0106160 0x7fdbc019da40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.678+0000 7fdbc6e46700 1 --2- 172.21.15.37:0/2238619656 >> [v2:172.21.15.37:6800/401419890,v1:172.21.15.37:6801/401419890] conn(0x7fdbb0038f80 0x7fdbb003b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:48.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.678+0000 7fdbc6e46700 1 -- 172.21.15.37:0/2238619656 >> 172.21.15.37:0/2238619656 conn(0x7fdbc0101e80 msgr2=0x7fdbc0194620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:48.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.678+0000 7fdbc6e46700 1 -- 172.21.15.37:0/2238619656 shutdown_connections 2024-03-26T06:13:48.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:47.678+0000 7fdbc6e46700 1 -- 172.21.15.37:0/2238619656 wait complete. 2024-03-26T06:13:48.496 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 9 is available 2024-03-26T06:13:48.496 INFO:teuthology.orchestra.run.smithi037.stdout:Generating a dashboard self-signed certificate... 2024-03-26T06:13:48.826 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:48 smithi037 ceph-mon[31974]: [26/Mar/2024:06:13:47] ENGINE Bus STARTING 2024-03-26T06:13:48.826 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:48 smithi037 ceph-mon[31974]: [26/Mar/2024:06:13:48] ENGINE Serving on https://172.21.15.37:7150 2024-03-26T06:13:48.826 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:48 smithi037 ceph-mon[31974]: [26/Mar/2024:06:13:48] ENGINE Serving on http://172.21.15.37:8765 2024-03-26T06:13:48.826 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:48 smithi037 ceph-mon[31974]: [26/Mar/2024:06:13:48] ENGINE Bus STARTED 2024-03-26T06:13:49.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:49 smithi037 ceph-mon[31974]: mgrmap e12: x(active, since 2s) 2024-03-26T06:13:49.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:13:49.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:49 smithi037 ceph-mon[31974]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:13:49.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:13:49.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:13:50.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-26T06:13:50.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.280+0000 7fb80e50c700 1 Processor -- start 2024-03-26T06:13:50.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.280+0000 7fb80e50c700 1 -- start start 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.281+0000 7fb80e50c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80810a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.281+0000 7fb80e50c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80807ae90 con 0x7fb808108320 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.281+0000 7fb807fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80810a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.281+0000 7fb807fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80810a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36764/0 (socket says 172.21.15.37:36764) 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.281+0000 7fb807fff700 1 -- 172.21.15.37:0/235296718 learned_addr learned my addr 172.21.15.37:0/235296718 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.281+0000 7fb807fff700 1 -- 172.21.15.37:0/235296718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80810ac50 con 0x7fb808108320 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.281+0000 7fb807fff700 1 --2- 172.21.15.37:0/235296718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80810a710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80098a0 tx=0x7fb7f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e021ce8217602 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.282+0000 7fb806ffd700 1 -- 172.21.15.37:0/235296718 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7f8004160 con 0x7fb808108320 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.282+0000 7fb806ffd700 1 -- 172.21.15.37:0/235296718 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7f8004300 con 0x7fb808108320 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.282+0000 7fb806ffd700 1 -- 172.21.15.37:0/235296718 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7f8011b20 con 0x7fb808108320 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.282+0000 7fb80e50c700 1 -- 172.21.15.37:0/235296718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 msgr2=0x7fb80810a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.282+0000 7fb80e50c700 1 --2- 172.21.15.37:0/235296718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80810a710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80098a0 tx=0x7fb7f8009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.282+0000 7fb80e50c700 1 -- 172.21.15.37:0/235296718 shutdown_connections 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.282+0000 7fb80e50c700 1 --2- 172.21.15.37:0/235296718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80810a710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.282+0000 7fb80e50c700 1 -- 172.21.15.37:0/235296718 >> 172.21.15.37:0/235296718 conn(0x7fb808101e10 msgr2=0x7fb808104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.282+0000 7fb80e50c700 1 -- 172.21.15.37:0/235296718 shutdown_connections 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.282+0000 7fb80e50c700 1 -- 172.21.15.37:0/235296718 wait complete. 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.283+0000 7fb80e50c700 1 Processor -- start 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.283+0000 7fb80e50c700 1 -- start start 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.283+0000 7fb80e50c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.283+0000 7fb80e50c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80810bcc0 con 0x7fb808108320 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.283+0000 7fb807fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.283+0000 7fb807fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80819b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36778/0 (socket says 172.21.15.37:36778) 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.283+0000 7fb807fff700 1 -- 172.21.15.37:0/1894492781 learned_addr learned my addr 172.21.15.37:0/1894492781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.284+0000 7fb807fff700 1 -- 172.21.15.37:0/1894492781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80819bd50 con 0x7fb808108320 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.284+0000 7fb807fff700 1 --2- 172.21.15.37:0/1894492781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80819b810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80116a0 tx=0x7fb7f8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:50.239 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.284+0000 7fb8057fa700 1 -- 172.21.15.37:0/1894492781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7f8003a40 con 0x7fb808108320 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.284+0000 7fb8057fa700 1 -- 172.21.15.37:0/1894492781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7f8003be0 con 0x7fb808108320 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.284+0000 7fb80e50c700 1 -- 172.21.15.37:0/1894492781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80819bfe0 con 0x7fb808108320 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.284+0000 7fb8057fa700 1 -- 172.21.15.37:0/1894492781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7f801a750 con 0x7fb808108320 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.284+0000 7fb80e50c700 1 -- 172.21.15.37:0/1894492781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80819c440 con 0x7fb808108320 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.285+0000 7fb80e50c700 1 -- 172.21.15.37:0/1894492781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb808068ec0 con 0x7fb808108320 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.285+0000 7fb8057fa700 1 -- 172.21.15.37:0/1894492781 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fb7f801aa70 con 0x7fb808108320 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.286+0000 7fb8057fa700 1 --2- 172.21.15.37:0/1894492781 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb7f0038f20 0x7fb7f003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.286+0000 7fb8057fa700 1 -- 172.21.15.37:0/1894492781 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb7f804d660 con 0x7fb808108320 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.286+0000 7fb8077fe700 1 --2- 172.21.15.37:0/1894492781 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb7f0038f20 0x7fb7f003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.286+0000 7fb8077fe700 1 --2- 172.21.15.37:0/1894492781 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb7f0038f20 0x7fb7f003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc006fd0 tx=0x7fb7fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.290+0000 7fb8057fa700 1 -- 172.21.15.37:0/1894492781 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb7f8011e20 con 0x7fb808108320 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.504+0000 7fb80e50c700 1 -- 172.21.15.37:0/1894492781 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb80819ee50 con 0x7fb7f0038f20 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.599+0000 7fb8057fa700 1 -- 172.21.15.37:0/1894492781 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb80819ee50 con 0x7fb7f0038f20 2024-03-26T06:13:50.240 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.602+0000 7fb80e50c700 1 -- 172.21.15.37:0/1894492781 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb7f0038f20 msgr2=0x7fb7f003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.602+0000 7fb80e50c700 1 --2- 172.21.15.37:0/1894492781 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb7f0038f20 0x7fb7f003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc006fd0 tx=0x7fb7fc006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.602+0000 7fb80e50c700 1 -- 172.21.15.37:0/1894492781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 msgr2=0x7fb80819b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.602+0000 7fb80e50c700 1 --2- 172.21.15.37:0/1894492781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80819b810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80116a0 tx=0x7fb7f8011ba0 comp rx=0 tx=0).stop 2024-03-26T06:13:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.602+0000 7fb80e50c700 1 -- 172.21.15.37:0/1894492781 shutdown_connections 2024-03-26T06:13:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.602+0000 7fb80e50c700 1 --2- 172.21.15.37:0/1894492781 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb7f0038f20 0x7fb7f003b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.602+0000 7fb80e50c700 1 --2- 172.21.15.37:0/1894492781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb808108320 0x7fb80819b810 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.602+0000 7fb80e50c700 1 -- 172.21.15.37:0/1894492781 >> 172.21.15.37:0/1894492781 conn(0x7fb808101e10 msgr2=0x7fb8081028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.602+0000 7fb80e50c700 1 -- 172.21.15.37:0/1894492781 shutdown_connections 2024-03-26T06:13:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:49.602+0000 7fb80e50c700 1 -- 172.21.15.37:0/1894492781 wait complete. 2024-03-26T06:13:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial admin user... 2024-03-26T06:13:51.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:51 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:13:51.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:51 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:13:52.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:52 smithi037 ceph-mon[31974]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:13:52.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Vgtn6cGqTmgyTtDBrbSUd.RUvsbabxWOC0TXF7EEAj.ymyJXjwSQe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711433631, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-26T06:13:52.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.729+0000 7f5264e12700 1 Processor -- start 2024-03-26T06:13:52.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.729+0000 7f5264e12700 1 -- start start 2024-03-26T06:13:52.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.729+0000 7f5264e12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f5260107cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:52.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.729+0000 7f5264e12700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5260108290 con 0x7f52601078a0 2024-03-26T06:13:52.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.729+0000 7f525e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f5260107cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:52.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.729+0000 7f525e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f5260107cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36786/0 (socket says 172.21.15.37:36786) 2024-03-26T06:13:52.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.729+0000 7f525e59c700 1 -- 172.21.15.37:0/1667918027 learned_addr learned my addr 172.21.15.37:0/1667918027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:52.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.729+0000 7f525e59c700 1 -- 172.21.15.37:0/1667918027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5260108aa0 con 0x7f52601078a0 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.730+0000 7f525e59c700 1 --2- 172.21.15.37:0/1667918027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f5260107cc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f52500098a0 tx=0x7f5250009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e793fd51ab6badbc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.730+0000 7f525d59a700 1 -- 172.21.15.37:0/1667918027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5250004160 con 0x7f52601078a0 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.730+0000 7f525d59a700 1 -- 172.21.15.37:0/1667918027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5250004300 con 0x7f52601078a0 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.730+0000 7f525d59a700 1 -- 172.21.15.37:0/1667918027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5250011b20 con 0x7f52601078a0 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.730+0000 7f5264e12700 1 -- 172.21.15.37:0/1667918027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 msgr2=0x7f5260107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.730+0000 7f5264e12700 1 --2- 172.21.15.37:0/1667918027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f5260107cc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f52500098a0 tx=0x7f5250009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.731+0000 7f5264e12700 1 -- 172.21.15.37:0/1667918027 shutdown_connections 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.731+0000 7f5264e12700 1 --2- 172.21.15.37:0/1667918027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f5260107cc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.731+0000 7f5264e12700 1 -- 172.21.15.37:0/1667918027 >> 172.21.15.37:0/1667918027 conn(0x7f5260102ee0 msgr2=0x7f5260105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.731+0000 7f5264e12700 1 -- 172.21.15.37:0/1667918027 shutdown_connections 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.731+0000 7f5264e12700 1 -- 172.21.15.37:0/1667918027 wait complete. 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.731+0000 7f5264e12700 1 Processor -- start 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.731+0000 7f5264e12700 1 -- start start 2024-03-26T06:13:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.731+0000 7f5264e12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f526007cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.731+0000 7f5264e12700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526010be50 con 0x7f52601078a0 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.732+0000 7f525e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f526007cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.732+0000 7f525e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f526007cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36802/0 (socket says 172.21.15.37:36802) 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.732+0000 7f525e59c700 1 -- 172.21.15.37:0/1353313937 learned_addr learned my addr 172.21.15.37:0/1353313937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.732+0000 7f525e59c700 1 -- 172.21.15.37:0/1353313937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526007d090 con 0x7f52601078a0 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.732+0000 7f525e59c700 1 --2- 172.21.15.37:0/1353313937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f526007cb50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f52500117e0 tx=0x7f5250011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.732+0000 7f524f7fe700 1 -- 172.21.15.37:0/1353313937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5250003bf0 con 0x7f52601078a0 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.732+0000 7f524f7fe700 1 -- 172.21.15.37:0/1353313937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f525001a910 con 0x7f52601078a0 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.732+0000 7f524f7fe700 1 -- 172.21.15.37:0/1353313937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52500249a0 con 0x7f52601078a0 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.732+0000 7f5264e12700 1 -- 172.21.15.37:0/1353313937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526007b280 con 0x7f52601078a0 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.732+0000 7f5264e12700 1 -- 172.21.15.37:0/1353313937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526007b740 con 0x7f52601078a0 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.733+0000 7f524f7fe700 1 -- 172.21.15.37:0/1353313937 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f5250022020 con 0x7f52601078a0 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.733+0000 7f524f7fe700 1 --2- 172.21.15.37:0/1353313937 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5248038f80 0x7f524803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.733+0000 7f524f7fe700 1 -- 172.21.15.37:0/1353313937 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f525004d280 con 0x7f52601078a0 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.733+0000 7f525dd9b700 1 --2- 172.21.15.37:0/1353313937 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5248038f80 0x7f524803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.734+0000 7f525dd9b700 1 --2- 172.21.15.37:0/1353313937 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5248038f80 0x7f524803b440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5254006fd0 tx=0x7f5254006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.734+0000 7f5264e12700 1 -- 172.21.15.37:0/1353313937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5260068ec0 con 0x7f52601078a0 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.739+0000 7f524f7fe700 1 -- 172.21.15.37:0/1353313937 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5250011e20 con 0x7f52601078a0 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:50.995+0000 7f5264e12700 1 -- 172.21.15.37:0/1353313937 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f526007bde0 con 0x7f5248038f80 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:51.242+0000 7f524f7fe700 1 -- 172.21.15.37:0/1353313937 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f526007bde0 con 0x7f5248038f80 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:51.244+0000 7f5264e12700 1 -- 172.21.15.37:0/1353313937 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5248038f80 msgr2=0x7f524803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:51.244+0000 7f5264e12700 1 --2- 172.21.15.37:0/1353313937 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5248038f80 0x7f524803b440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5254006fd0 tx=0x7f5254006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:51.244+0000 7f5264e12700 1 -- 172.21.15.37:0/1353313937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 msgr2=0x7f526007cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:51.244+0000 7f5264e12700 1 --2- 172.21.15.37:0/1353313937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f526007cb50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f52500117e0 tx=0x7f5250011c10 comp rx=0 tx=0).stop 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:51.244+0000 7f5264e12700 1 -- 172.21.15.37:0/1353313937 shutdown_connections 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:51.244+0000 7f5264e12700 1 --2- 172.21.15.37:0/1353313937 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5248038f80 0x7f524803b440 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:51.244+0000 7f5264e12700 1 --2- 172.21.15.37:0/1353313937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52601078a0 0x7f526007cb50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:52.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:51.244+0000 7f5264e12700 1 -- 172.21.15.37:0/1353313937 >> 172.21.15.37:0/1353313937 conn(0x7f5260102ee0 msgr2=0x7f52601039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:52.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:51.245+0000 7f5264e12700 1 -- 172.21.15.37:0/1353313937 shutdown_connections 2024-03-26T06:13:52.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:51.245+0000 7f5264e12700 1 -- 172.21.15.37:0/1353313937 wait complete. 2024-03-26T06:13:52.651 INFO:teuthology.orchestra.run.smithi037.stdout:Fetching dashboard port number... 2024-03-26T06:13:53.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 8443 2024-03-26T06:13:53.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.133+0000 7effed0db700 1 Processor -- start 2024-03-26T06:13:53.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.133+0000 7effed0db700 1 -- start start 2024-03-26T06:13:53.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.134+0000 7effed0db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe8107d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:53.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.134+0000 7effed0db700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe8108310 con 0x7effe8107920 2024-03-26T06:13:53.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.134+0000 7effe6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe8107d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:53.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.134+0000 7effe6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe8107d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36818/0 (socket says 172.21.15.37:36818) 2024-03-26T06:13:53.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.134+0000 7effe6d9d700 1 -- 172.21.15.37:0/2002289937 learned_addr learned my addr 172.21.15.37:0/2002289937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:53.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.134+0000 7effe6d9d700 1 -- 172.21.15.37:0/2002289937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe8108ad0 con 0x7effe8107920 2024-03-26T06:13:53.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.134+0000 7effe6d9d700 1 --2- 172.21.15.37:0/2002289937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe8107d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7effdc0098a0 tx=0x7effdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6164ba5632798b01 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:53.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.134+0000 7effe5d9b700 1 -- 172.21.15.37:0/2002289937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effdc004160 con 0x7effe8107920 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.134+0000 7effe5d9b700 1 -- 172.21.15.37:0/2002289937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effdc004300 con 0x7effe8107920 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.135+0000 7effe5d9b700 1 -- 172.21.15.37:0/2002289937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effdc011b20 con 0x7effe8107920 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.135+0000 7effed0db700 1 -- 172.21.15.37:0/2002289937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 msgr2=0x7effe8107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.135+0000 7effed0db700 1 --2- 172.21.15.37:0/2002289937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe8107d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7effdc0098a0 tx=0x7effdc009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.135+0000 7effed0db700 1 -- 172.21.15.37:0/2002289937 shutdown_connections 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.135+0000 7effed0db700 1 --2- 172.21.15.37:0/2002289937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe8107d40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.135+0000 7effed0db700 1 -- 172.21.15.37:0/2002289937 >> 172.21.15.37:0/2002289937 conn(0x7effe8102f40 msgr2=0x7effe8105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.135+0000 7effed0db700 1 -- 172.21.15.37:0/2002289937 shutdown_connections 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.135+0000 7effed0db700 1 -- 172.21.15.37:0/2002289937 wait complete. 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.136+0000 7effed0db700 1 Processor -- start 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.136+0000 7effed0db700 1 -- start start 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.136+0000 7effed0db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe807cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:53.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.136+0000 7effed0db700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe810be80 con 0x7effe8107920 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.136+0000 7effe6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe807cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.136+0000 7effe6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe807cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36822/0 (socket says 172.21.15.37:36822) 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.136+0000 7effe6d9d700 1 -- 172.21.15.37:0/947310557 learned_addr learned my addr 172.21.15.37:0/947310557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.137+0000 7effe6d9d700 1 -- 172.21.15.37:0/947310557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe807d090 con 0x7effe8107920 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.137+0000 7effe6d9d700 1 --2- 172.21.15.37:0/947310557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe807cb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7effdc0117e0 tx=0x7effdc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.137+0000 7effcffff700 1 -- 172.21.15.37:0/947310557 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effdc003bf0 con 0x7effe8107920 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.137+0000 7effed0db700 1 -- 172.21.15.37:0/947310557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe807b280 con 0x7effe8107920 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.137+0000 7effcffff700 1 -- 172.21.15.37:0/947310557 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effdc01a910 con 0x7effe8107920 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.137+0000 7effcffff700 1 -- 172.21.15.37:0/947310557 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effdc0249a0 con 0x7effe8107920 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.137+0000 7effed0db700 1 -- 172.21.15.37:0/947310557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe807b740 con 0x7effe8107920 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.138+0000 7effcffff700 1 -- 172.21.15.37:0/947310557 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7effdc022020 con 0x7effe8107920 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.138+0000 7effed0db700 1 -- 172.21.15.37:0/947310557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe8063410 con 0x7effe8107920 2024-03-26T06:13:53.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.138+0000 7effcffff700 1 --2- 172.21.15.37:0/947310557 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7effd0038f20 0x7effd003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.138+0000 7effcffff700 1 -- 172.21.15.37:0/947310557 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7effdc04d440 con 0x7effe8107920 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.139+0000 7effe659c700 1 --2- 172.21.15.37:0/947310557 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7effd0038f20 0x7effd003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.139+0000 7effe659c700 1 --2- 172.21.15.37:0/947310557 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7effd0038f20 0x7effd003b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7effd8006fd0 tx=0x7effd8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.143+0000 7effcffff700 1 -- 172.21.15.37:0/947310557 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7effdc011e20 con 0x7effe8107920 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.329+0000 7effcffff700 1 -- 172.21.15.37:0/947310557 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7effdc02d910 con 0x7effe8107920 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.351+0000 7effed0db700 1 -- 172.21.15.37:0/947310557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7effe807be00 con 0x7effe8107920 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.351+0000 7effcffff700 1 -- 172.21.15.37:0/947310557 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7effdc00b9d0 con 0x7effe8107920 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.353+0000 7effed0db700 1 -- 172.21.15.37:0/947310557 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7effd0038f20 msgr2=0x7effd003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.353+0000 7effed0db700 1 --2- 172.21.15.37:0/947310557 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7effd0038f20 0x7effd003b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7effd8006fd0 tx=0x7effd8006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.353+0000 7effed0db700 1 -- 172.21.15.37:0/947310557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 msgr2=0x7effe807cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.353+0000 7effed0db700 1 --2- 172.21.15.37:0/947310557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe807cb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7effdc0117e0 tx=0x7effdc011c10 comp rx=0 tx=0).stop 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.353+0000 7effed0db700 1 -- 172.21.15.37:0/947310557 shutdown_connections 2024-03-26T06:13:53.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.353+0000 7effed0db700 1 --2- 172.21.15.37:0/947310557 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7effd0038f20 0x7effd003b3e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:53.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.353+0000 7effed0db700 1 --2- 172.21.15.37:0/947310557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8107920 0x7effe807cb50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:53.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.353+0000 7effed0db700 1 -- 172.21.15.37:0/947310557 >> 172.21.15.37:0/947310557 conn(0x7effe8102f40 msgr2=0x7effe81039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:53.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.354+0000 7effed0db700 1 -- 172.21.15.37:0/947310557 shutdown_connections 2024-03-26T06:13:53.979 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:53.354+0000 7effed0db700 1 -- 172.21.15.37:0/947310557 wait complete. 2024-03-26T06:13:53.984 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T06:13:53.984 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-26T06:13:53.992 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T06:13:53.992 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-26T06:13:53.992 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-26T06:13:53.992 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-26T06:13:53.994 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph Dashboard is now available at: 2024-03-26T06:13:53.994 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:13:53.994 INFO:teuthology.orchestra.run.smithi037.stdout: URL: https://smithi037.front.sepia.ceph.com:8443/ 2024-03-26T06:13:53.995 INFO:teuthology.orchestra.run.smithi037.stdout: User: admin 2024-03-26T06:13:53.995 INFO:teuthology.orchestra.run.smithi037.stdout: Password: xye9phv6rz 2024-03-26T06:13:53.995 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:13:53.995 INFO:teuthology.orchestra.run.smithi037.stdout:Saving cluster configuration to /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config directory 2024-03-26T06:13:53.995 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling autotune for osd_memory_target 2024-03-26T06:13:54.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:54 smithi037 ceph-mon[31974]: mgrmap e13: x(active, since 6s) 2024-03-26T06:13:54.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:54 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/947310557' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-26T06:13:55.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.432+0000 7f0aedbb2700 1 Processor -- start 2024-03-26T06:13:55.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.432+0000 7f0aedbb2700 1 -- start start 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.433+0000 7f0aedbb2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae810a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.433+0000 7f0aedbb2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae807adf0 con 0x7f0ae8108350 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.433+0000 7f0ae77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae810a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.433+0000 7f0ae77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae810a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36826/0 (socket says 172.21.15.37:36826) 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.433+0000 7f0ae77fe700 1 -- 172.21.15.37:0/1137731233 learned_addr learned my addr 172.21.15.37:0/1137731233 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.433+0000 7f0ae77fe700 1 -- 172.21.15.37:0/1137731233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae810ac80 con 0x7f0ae8108350 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.433+0000 7f0ae77fe700 1 --2- 172.21.15.37:0/1137731233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae810a740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80098a0 tx=0x7f0ad8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1992fdc01c573708 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.433+0000 7f0ae67fc700 1 -- 172.21.15.37:0/1137731233 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ad8004160 con 0x7f0ae8108350 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.433+0000 7f0ae67fc700 1 -- 172.21.15.37:0/1137731233 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ad8004300 con 0x7f0ae8108350 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.433+0000 7f0ae67fc700 1 -- 172.21.15.37:0/1137731233 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ad8011b20 con 0x7f0ae8108350 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.434+0000 7f0aedbb2700 1 -- 172.21.15.37:0/1137731233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 msgr2=0x7f0ae810a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:55.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.434+0000 7f0aedbb2700 1 --2- 172.21.15.37:0/1137731233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae810a740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80098a0 tx=0x7f0ad8009bb0 comp rx=0 tx=0).stop 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.434+0000 7f0aedbb2700 1 -- 172.21.15.37:0/1137731233 shutdown_connections 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.434+0000 7f0aedbb2700 1 --2- 172.21.15.37:0/1137731233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae810a740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.434+0000 7f0aedbb2700 1 -- 172.21.15.37:0/1137731233 >> 172.21.15.37:0/1137731233 conn(0x7f0ae8101de0 msgr2=0x7f0ae8104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.434+0000 7f0aedbb2700 1 -- 172.21.15.37:0/1137731233 shutdown_connections 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.434+0000 7f0aedbb2700 1 -- 172.21.15.37:0/1137731233 wait complete. 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.435+0000 7f0aedbb2700 1 Processor -- start 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.435+0000 7f0aedbb2700 1 -- start start 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.435+0000 7f0aedbb2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.435+0000 7f0aedbb2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae810bc80 con 0x7f0ae8108350 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.435+0000 7f0ae77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.435+0000 7f0ae77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36840/0 (socket says 172.21.15.37:36840) 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.435+0000 7f0ae77fe700 1 -- 172.21.15.37:0/2371446461 learned_addr learned my addr 172.21.15.37:0/2371446461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.435+0000 7f0ae77fe700 1 -- 172.21.15.37:0/2371446461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae819bcc0 con 0x7f0ae8108350 2024-03-26T06:13:55.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.436+0000 7f0ae77fe700 1 --2- 172.21.15.37:0/2371446461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae819b780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8011b60 tx=0x7f0ad8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.436+0000 7f0ae4ff9700 1 -- 172.21.15.37:0/2371446461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ad8003b60 con 0x7f0ae8108350 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.436+0000 7f0ae4ff9700 1 -- 172.21.15.37:0/2371446461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ad801a910 con 0x7f0ae8108350 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.436+0000 7f0aedbb2700 1 -- 172.21.15.37:0/2371446461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae819bf50 con 0x7f0ae8108350 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.436+0000 7f0ae4ff9700 1 -- 172.21.15.37:0/2371446461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ad8024930 con 0x7f0ae8108350 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.436+0000 7f0aedbb2700 1 -- 172.21.15.37:0/2371446461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae819c3b0 con 0x7f0ae8108350 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.437+0000 7f0ae4ff9700 1 -- 172.21.15.37:0/2371446461 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f0ad8022020 con 0x7f0ae8108350 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.437+0000 7f0aedbb2700 1 -- 172.21.15.37:0/2371446461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae8063410 con 0x7f0ae8108350 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.437+0000 7f0ae4ff9700 1 --2- 172.21.15.37:0/2371446461 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ad0039010 0x7f0ad003b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.437+0000 7f0ae4ff9700 1 -- 172.21.15.37:0/2371446461 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0ad804d2a0 con 0x7f0ae8108350 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.437+0000 7f0ae6ffd700 1 --2- 172.21.15.37:0/2371446461 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ad0039010 0x7f0ad003b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.438+0000 7f0ae6ffd700 1 --2- 172.21.15.37:0/2371446461 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ad0039010 0x7f0ad003b4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0adc006fd0 tx=0x7f0adc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.442+0000 7f0ae4ff9700 1 -- 172.21.15.37:0/2371446461 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0ad800b3e0 con 0x7f0ae8108350 2024-03-26T06:13:55.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.645+0000 7f0aedbb2700 1 -- 172.21.15.37:0/2371446461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f0ae819ee40 con 0x7f0ae8108350 2024-03-26T06:13:55.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.645+0000 7f0ae4ff9700 1 -- 172.21.15.37:0/2371446461 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0ad8016260 con 0x7f0ae8108350 2024-03-26T06:13:55.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.648+0000 7f0aedbb2700 1 -- 172.21.15.37:0/2371446461 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ad0039010 msgr2=0x7f0ad003b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:55.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.648+0000 7f0aedbb2700 1 --2- 172.21.15.37:0/2371446461 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ad0039010 0x7f0ad003b4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0adc006fd0 tx=0x7f0adc006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:55.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.648+0000 7f0aedbb2700 1 -- 172.21.15.37:0/2371446461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 msgr2=0x7f0ae819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:55.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.648+0000 7f0aedbb2700 1 --2- 172.21.15.37:0/2371446461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae819b780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8011b60 tx=0x7f0ad8011c40 comp rx=0 tx=0).stop 2024-03-26T06:13:55.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.648+0000 7f0aedbb2700 1 -- 172.21.15.37:0/2371446461 shutdown_connections 2024-03-26T06:13:55.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.648+0000 7f0aedbb2700 1 --2- 172.21.15.37:0/2371446461 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ad0039010 0x7f0ad003b4d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:55.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.648+0000 7f0aedbb2700 1 --2- 172.21.15.37:0/2371446461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae8108350 0x7f0ae819b780 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:55.225 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.648+0000 7f0aedbb2700 1 -- 172.21.15.37:0/2371446461 >> 172.21.15.37:0/2371446461 conn(0x7f0ae8101de0 msgr2=0x7f0ae8102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:55.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.648+0000 7f0aedbb2700 1 -- 172.21.15.37:0/2371446461 shutdown_connections 2024-03-26T06:13:55.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:54.648+0000 7f0aedbb2700 1 -- 172.21.15.37:0/2371446461 wait complete. 2024-03-26T06:13:56.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.667+0000 7f005c41a700 1 Processor -- start 2024-03-26T06:13:56.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.667+0000 7f005c41a700 1 -- start start 2024-03-26T06:13:56.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.667+0000 7f005c41a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f0054106b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:56.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.667+0000 7f005c41a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0054107130 con 0x7f0054106740 2024-03-26T06:13:56.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.668+0000 7f005a1b6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f0054106b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:56.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.668+0000 7f005a1b6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f0054106b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36856/0 (socket says 172.21.15.37:36856) 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.668+0000 7f005a1b6700 1 -- 172.21.15.37:0/993817135 learned_addr learned my addr 172.21.15.37:0/993817135 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.668+0000 7f005a1b6700 1 -- 172.21.15.37:0/993817135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0054107940 con 0x7f0054106740 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.668+0000 7f005a1b6700 1 --2- 172.21.15.37:0/993817135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f0054106b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0044009e80 tx=0x7f004400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813cf0780aad3acc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.668+0000 7f00591b4700 1 -- 172.21.15.37:0/993817135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0044017070 con 0x7f0054106740 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.668+0000 7f00591b4700 1 -- 172.21.15.37:0/993817135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f004400b8c0 con 0x7f0054106740 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.668+0000 7f00591b4700 1 -- 172.21.15.37:0/993817135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00440045c0 con 0x7f0054106740 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.669+0000 7f005c41a700 1 -- 172.21.15.37:0/993817135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 msgr2=0x7f0054106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.669+0000 7f005c41a700 1 --2- 172.21.15.37:0/993817135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f0054106b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0044009e80 tx=0x7f004400b330 comp rx=0 tx=0).stop 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.669+0000 7f005c41a700 1 -- 172.21.15.37:0/993817135 shutdown_connections 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.669+0000 7f005c41a700 1 --2- 172.21.15.37:0/993817135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f0054106b60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.669+0000 7f005c41a700 1 -- 172.21.15.37:0/993817135 >> 172.21.15.37:0/993817135 conn(0x7f0054101de0 msgr2=0x7f0054104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.669+0000 7f005c41a700 1 -- 172.21.15.37:0/993817135 shutdown_connections 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.669+0000 7f005c41a700 1 -- 172.21.15.37:0/993817135 wait complete. 2024-03-26T06:13:56.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.669+0000 7f005c41a700 1 Processor -- start 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f005c41a700 1 -- start start 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f005c41a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f005407aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f005c41a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005410acf0 con 0x7f0054106740 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f005a1b6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f005407aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f005a1b6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f005407aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36860/0 (socket says 172.21.15.37:36860) 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f005a1b6700 1 -- 172.21.15.37:0/4163040099 learned_addr learned my addr 172.21.15.37:0/4163040099 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f005a1b6700 1 -- 172.21.15.37:0/4163040099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005419e4e0 con 0x7f0054106740 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f005a1b6700 1 --2- 172.21.15.37:0/4163040099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f005407aa90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0044003fa0 tx=0x7f0044004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f00537fe700 1 -- 172.21.15.37:0/4163040099 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0044020020 con 0x7f0054106740 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f005c41a700 1 -- 172.21.15.37:0/4163040099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005419e6b0 con 0x7f0054106740 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f00537fe700 1 -- 172.21.15.37:0/4163040099 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f004400bba0 con 0x7f0054106740 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f00537fe700 1 -- 172.21.15.37:0/4163040099 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0044017040 con 0x7f0054106740 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.670+0000 7f005c41a700 1 -- 172.21.15.37:0/4163040099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005419eb30 con 0x7f0054106740 2024-03-26T06:13:56.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.671+0000 7f00537fe700 1 -- 172.21.15.37:0/4163040099 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f0044023020 con 0x7f0054106740 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.671+0000 7f005c41a700 1 -- 172.21.15.37:0/4163040099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0054063410 con 0x7f0054106740 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.671+0000 7f00537fe700 1 --2- 172.21.15.37:0/4163040099 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f003c038fc0 0x7f003c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.671+0000 7f00537fe700 1 -- 172.21.15.37:0/4163040099 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f004400f050 con 0x7f0054106740 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.672+0000 7f00599b5700 1 --2- 172.21.15.37:0/4163040099 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f003c038fc0 0x7f003c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.672+0000 7f00599b5700 1 --2- 172.21.15.37:0/4163040099 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f003c038fc0 0x7f003c03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0048006fd0 tx=0x7f0048006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.676+0000 7f00537fe700 1 -- 172.21.15.37:0/4163040099 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f004402a020 con 0x7f0054106740 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.994+0000 7f005c41a700 1 -- 172.21.15.37:0/4163040099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f005419ed00 con 0x7f0054106740 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.996+0000 7f00537fe700 1 -- 172.21.15.37:0/4163040099 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f004402d350 con 0x7f0054106740 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.998+0000 7f005c41a700 1 -- 172.21.15.37:0/4163040099 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f003c038fc0 msgr2=0x7f003c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.998+0000 7f005c41a700 1 --2- 172.21.15.37:0/4163040099 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f003c038fc0 0x7f003c03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0048006fd0 tx=0x7f0048006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.998+0000 7f005c41a700 1 -- 172.21.15.37:0/4163040099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 msgr2=0x7f005407aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.998+0000 7f005c41a700 1 --2- 172.21.15.37:0/4163040099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f005407aa90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0044003fa0 tx=0x7f0044004080 comp rx=0 tx=0).stop 2024-03-26T06:13:56.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.998+0000 7f005c41a700 1 -- 172.21.15.37:0/4163040099 shutdown_connections 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.998+0000 7f005c41a700 1 --2- 172.21.15.37:0/4163040099 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f003c038fc0 0x7f003c03b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.998+0000 7f005c41a700 1 --2- 172.21.15.37:0/4163040099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0054106740 0x7f005407aa90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.998+0000 7f005c41a700 1 -- 172.21.15.37:0/4163040099 >> 172.21.15.37:0/4163040099 conn(0x7f0054101de0 msgr2=0x7f0054102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.998+0000 7f005c41a700 1 -- 172.21.15.37:0/4163040099 shutdown_connections 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-26T06:13:55.998+0000 7f005c41a700 1 -- 172.21.15.37:0/4163040099 wait complete. 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout:Or, if you are only running a single cluster on this host: 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:13:56.660 INFO:teuthology.orchestra.run.smithi037.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-26T06:13:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:13:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: ceph telemetry on 2024-03-26T06:13:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:13:56.661 INFO:teuthology.orchestra.run.smithi037.stdout:For more information see: 2024-03-26T06:13:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:13:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-26T06:13:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:13:56.661 INFO:teuthology.orchestra.run.smithi037.stdout:Bootstrap complete. 2024-03-26T06:13:56.697 INFO:tasks.cephadm:Fetching config... 2024-03-26T06:13:56.697 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:13:56.697 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-26T06:13:56.713 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-26T06:13:56.713 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:13:56.713 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-26T06:13:56.768 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-26T06:13:56.769 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:13:56.769 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-26T06:13:56.835 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-26T06:13:56.836 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:13:56.836 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-26T06:13:56.890 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-26T06:13:56.891 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/5b51G256dU7INFRedpjcZ/EMzCqlqs9iAq5UYYr0gdT3zx91RNoBzMBa5txxiaPd6+5iZn5v7pWM8xDkHsHaM2Uk5J1oq2OsdLriH30NeAmJOaPEyAHy+2zgSe0MCa2ivDnDe+KbnaWbcCu/OcV6IJxHWgmM1loD3f/Qkb1hrbKsj5uAZDzGtZ5CfiHRsyx95l0cEJgW9t3NSZrOYNjBmZ//lZtpx8XgugKHEQySeVXdAAexVk4569AVnc9GY6Qn43n4XYUR4WJx3xjT45cLWzexa+T2bdi09EzoInxw12dR5bZjuWffZtc0iOKS/erFAm8aRuijBpVwzusSDW7H+1f1yn7goiM/HcYdYYJYosorC6Y9XZar3EBIN5jWWOgHyRQnxEMqmPnUKESj9BqXZKmhwFRalgtv6BAFCHFDSlf8PiSDwEHkqrXgGLkjKnw8P10pisMj1rkc43GB3W58zEGH4WZ9VNV867p+Zx62tjCFoYFdD2J+HwcyUHTec= ceph-a86fb218-eb37-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T06:13:56.968 INFO:teuthology.orchestra.run.smithi037.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/5b51G256dU7INFRedpjcZ/EMzCqlqs9iAq5UYYr0gdT3zx91RNoBzMBa5txxiaPd6+5iZn5v7pWM8xDkHsHaM2Uk5J1oq2OsdLriH30NeAmJOaPEyAHy+2zgSe0MCa2ivDnDe+KbnaWbcCu/OcV6IJxHWgmM1loD3f/Qkb1hrbKsj5uAZDzGtZ5CfiHRsyx95l0cEJgW9t3NSZrOYNjBmZ//lZtpx8XgugKHEQySeVXdAAexVk4569AVnc9GY6Qn43n4XYUR4WJx3xjT45cLWzexa+T2bdi09EzoInxw12dR5bZjuWffZtc0iOKS/erFAm8aRuijBpVwzusSDW7H+1f1yn7goiM/HcYdYYJYosorC6Y9XZar3EBIN5jWWOgHyRQnxEMqmPnUKESj9BqXZKmhwFRalgtv6BAFCHFDSlf8PiSDwEHkqrXgGLkjKnw8P10pisMj1rkc43GB3W58zEGH4WZ9VNV867p+Zx62tjCFoYFdD2J+HwcyUHTec= ceph-a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:13:56.982 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/5b51G256dU7INFRedpjcZ/EMzCqlqs9iAq5UYYr0gdT3zx91RNoBzMBa5txxiaPd6+5iZn5v7pWM8xDkHsHaM2Uk5J1oq2OsdLriH30NeAmJOaPEyAHy+2zgSe0MCa2ivDnDe+KbnaWbcCu/OcV6IJxHWgmM1loD3f/Qkb1hrbKsj5uAZDzGtZ5CfiHRsyx95l0cEJgW9t3NSZrOYNjBmZ//lZtpx8XgugKHEQySeVXdAAexVk4569AVnc9GY6Qn43n4XYUR4WJx3xjT45cLWzexa+T2bdi09EzoInxw12dR5bZjuWffZtc0iOKS/erFAm8aRuijBpVwzusSDW7H+1f1yn7goiM/HcYdYYJYosorC6Y9XZar3EBIN5jWWOgHyRQnxEMqmPnUKESj9BqXZKmhwFRalgtv6BAFCHFDSlf8PiSDwEHkqrXgGLkjKnw8P10pisMj1rkc43GB3W58zEGH4WZ9VNV867p+Zx62tjCFoYFdD2J+HwcyUHTec= ceph-a86fb218-eb37-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T06:13:57.036 INFO:teuthology.orchestra.run.smithi066.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/5b51G256dU7INFRedpjcZ/EMzCqlqs9iAq5UYYr0gdT3zx91RNoBzMBa5txxiaPd6+5iZn5v7pWM8xDkHsHaM2Uk5J1oq2OsdLriH30NeAmJOaPEyAHy+2zgSe0MCa2ivDnDe+KbnaWbcCu/OcV6IJxHWgmM1loD3f/Qkb1hrbKsj5uAZDzGtZ5CfiHRsyx95l0cEJgW9t3NSZrOYNjBmZ//lZtpx8XgugKHEQySeVXdAAexVk4569AVnc9GY6Qn43n4XYUR4WJx3xjT45cLWzexa+T2bdi09EzoInxw12dR5bZjuWffZtc0iOKS/erFAm8aRuijBpVwzusSDW7H+1f1yn7goiM/HcYdYYJYosorC6Y9XZar3EBIN5jWWOgHyRQnxEMqmPnUKESj9BqXZKmhwFRalgtv6BAFCHFDSlf8PiSDwEHkqrXgGLkjKnw8P10pisMj1rkc43GB3W58zEGH4WZ9VNV867p+Zx62tjCFoYFdD2J+HwcyUHTec= ceph-a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:13:57.055 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/5b51G256dU7INFRedpjcZ/EMzCqlqs9iAq5UYYr0gdT3zx91RNoBzMBa5txxiaPd6+5iZn5v7pWM8xDkHsHaM2Uk5J1oq2OsdLriH30NeAmJOaPEyAHy+2zgSe0MCa2ivDnDe+KbnaWbcCu/OcV6IJxHWgmM1loD3f/Qkb1hrbKsj5uAZDzGtZ5CfiHRsyx95l0cEJgW9t3NSZrOYNjBmZ//lZtpx8XgugKHEQySeVXdAAexVk4569AVnc9GY6Qn43n4XYUR4WJx3xjT45cLWzexa+T2bdi09EzoInxw12dR5bZjuWffZtc0iOKS/erFAm8aRuijBpVwzusSDW7H+1f1yn7goiM/HcYdYYJYosorC6Y9XZar3EBIN5jWWOgHyRQnxEMqmPnUKESj9BqXZKmhwFRalgtv6BAFCHFDSlf8PiSDwEHkqrXgGLkjKnw8P10pisMj1rkc43GB3W58zEGH4WZ9VNV867p+Zx62tjCFoYFdD2J+HwcyUHTec= ceph-a86fb218-eb37-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T06:13:57.095 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/5b51G256dU7INFRedpjcZ/EMzCqlqs9iAq5UYYr0gdT3zx91RNoBzMBa5txxiaPd6+5iZn5v7pWM8xDkHsHaM2Uk5J1oq2OsdLriH30NeAmJOaPEyAHy+2zgSe0MCa2ivDnDe+KbnaWbcCu/OcV6IJxHWgmM1loD3f/Qkb1hrbKsj5uAZDzGtZ5CfiHRsyx95l0cEJgW9t3NSZrOYNjBmZ//lZtpx8XgugKHEQySeVXdAAexVk4569AVnc9GY6Qn43n4XYUR4WJx3xjT45cLWzexa+T2bdi09EzoInxw12dR5bZjuWffZtc0iOKS/erFAm8aRuijBpVwzusSDW7H+1f1yn7goiM/HcYdYYJYosorC6Y9XZar3EBIN5jWWOgHyRQnxEMqmPnUKESj9BqXZKmhwFRalgtv6BAFCHFDSlf8PiSDwEHkqrXgGLkjKnw8P10pisMj1rkc43GB3W58zEGH4WZ9VNV867p+Zx62tjCFoYFdD2J+HwcyUHTec= ceph-a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:13:57.110 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-26T06:13:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:13:57.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:13:56 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/4163040099' entity='client.admin' 2024-03-26T06:13:59.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.183+0000 7fbfba382700 1 -- 172.21.15.37:0/1439153417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfb4102380 msgr2=0x7fbfb4104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:59.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.183+0000 7fbfba382700 1 --2- 172.21.15.37:0/1439153417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfb4102380 0x7fbfb4104770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009910 tx=0x7fbfa4009c20 comp rx=0 tx=0).stop 2024-03-26T06:13:59.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.183+0000 7fbfba382700 1 -- 172.21.15.37:0/1439153417 shutdown_connections 2024-03-26T06:13:59.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.183+0000 7fbfba382700 1 --2- 172.21.15.37:0/1439153417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfb4102380 0x7fbfb4104770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:59.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.183+0000 7fbfba382700 1 -- 172.21.15.37:0/1439153417 >> 172.21.15.37:0/1439153417 conn(0x7fbfb40fbcf0 msgr2=0x7fbfb40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:59.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.183+0000 7fbfba382700 1 -- 172.21.15.37:0/1439153417 shutdown_connections 2024-03-26T06:13:59.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.183+0000 7fbfba382700 1 -- 172.21.15.37:0/1439153417 wait complete. 2024-03-26T06:13:59.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.183+0000 7fbfba382700 1 Processor -- start 2024-03-26T06:13:59.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.183+0000 7fbfba382700 1 -- start start 2024-03-26T06:13:59.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfba382700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfb4102380 0x7fbfb4101c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:59.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfba382700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb41078e0 con 0x7fbfb4102380 2024-03-26T06:13:59.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfb3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfb4102380 0x7fbfb4101c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:59.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfb3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfb4102380 0x7fbfb4101c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49044/0 (socket says 172.21.15.37:49044) 2024-03-26T06:13:59.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfb3fff700 1 -- 172.21.15.37:0/3879936181 learned_addr learned my addr 172.21.15.37:0/3879936181 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:13:59.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfb3fff700 1 -- 172.21.15.37:0/3879936181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4102140 con 0x7fbfb4102380 2024-03-26T06:13:59.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfb3fff700 1 --2- 172.21.15.37:0/3879936181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfb4102380 0x7fbfb4101c00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4004fa0 tx=0x7fbfa4005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:59.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfb17fa700 1 -- 172.21.15.37:0/3879936181 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfa4027070 con 0x7fbfb4102380 2024-03-26T06:13:59.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfba382700 1 -- 172.21.15.37:0/3879936181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb4100330 con 0x7fbfb4102380 2024-03-26T06:13:59.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfb17fa700 1 -- 172.21.15.37:0/3879936181 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfa400be40 con 0x7fbfb4102380 2024-03-26T06:13:59.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfb17fa700 1 -- 172.21.15.37:0/3879936181 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfa400f460 con 0x7fbfb4102380 2024-03-26T06:13:59.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.184+0000 7fbfba382700 1 -- 172.21.15.37:0/3879936181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb4100870 con 0x7fbfb4102380 2024-03-26T06:13:59.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.185+0000 7fbfb17fa700 1 -- 172.21.15.37:0/3879936181 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fbfa402a070 con 0x7fbfb4102380 2024-03-26T06:13:59.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.186+0000 7fbfb17fa700 1 --2- 172.21.15.37:0/3879936181 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fbf9c038fc0 0x7fbf9c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:13:59.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.186+0000 7fbfb17fa700 1 -- 172.21.15.37:0/3879936181 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbfa401e030 con 0x7fbfb4102380 2024-03-26T06:13:59.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.186+0000 7fbfb37fe700 1 --2- 172.21.15.37:0/3879936181 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fbf9c038fc0 0x7fbf9c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:13:59.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.186+0000 7fbfba382700 1 -- 172.21.15.37:0/3879936181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb4108060 con 0x7fbfb4102380 2024-03-26T06:13:59.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.186+0000 7fbfb37fe700 1 --2- 172.21.15.37:0/3879936181 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fbf9c038fc0 0x7fbf9c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006fd0 tx=0x7fbfa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:13:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.191+0000 7fbfb17fa700 1 -- 172.21.15.37:0/3879936181 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fbfa4013070 con 0x7fbfb4102380 2024-03-26T06:13:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.402+0000 7fbfba382700 1 -- 172.21.15.37:0/3879936181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fbfb4063410 con 0x7fbfb4102380 2024-03-26T06:13:59.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.404+0000 7fbfb17fa700 1 -- 172.21.15.37:0/3879936181 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbfa4022470 con 0x7fbfb4102380 2024-03-26T06:13:59.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.408+0000 7fbfba382700 1 -- 172.21.15.37:0/3879936181 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fbf9c038fc0 msgr2=0x7fbf9c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:59.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.408+0000 7fbfba382700 1 --2- 172.21.15.37:0/3879936181 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fbf9c038fc0 0x7fbf9c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006fd0 tx=0x7fbfa8006e40 comp rx=0 tx=0).stop 2024-03-26T06:13:59.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.408+0000 7fbfba382700 1 -- 172.21.15.37:0/3879936181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfb4102380 msgr2=0x7fbfb4101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:13:59.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.408+0000 7fbfba382700 1 --2- 172.21.15.37:0/3879936181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfb4102380 0x7fbfb4101c00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4004fa0 tx=0x7fbfa4005080 comp rx=0 tx=0).stop 2024-03-26T06:13:59.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.408+0000 7fbfba382700 1 -- 172.21.15.37:0/3879936181 shutdown_connections 2024-03-26T06:13:59.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.408+0000 7fbfba382700 1 --2- 172.21.15.37:0/3879936181 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fbf9c038fc0 0x7fbf9c03b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:59.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.408+0000 7fbfba382700 1 --2- 172.21.15.37:0/3879936181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfb4102380 0x7fbfb4101c00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:13:59.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.408+0000 7fbfba382700 1 -- 172.21.15.37:0/3879936181 >> 172.21.15.37:0/3879936181 conn(0x7fbfb40fbcf0 msgr2=0x7fbfb40fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:13:59.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.408+0000 7fbfba382700 1 -- 172.21.15.37:0/3879936181 shutdown_connections 2024-03-26T06:13:59.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:13:59.408+0000 7fbfba382700 1 -- 172.21.15.37:0/3879936181 wait complete. 2024-03-26T06:14:00.274 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-26T06:14:00.274 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-26T06:14:00.539 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:00 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3879936181' entity='client.admin' 2024-03-26T06:14:00.539 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:00.539 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:00.539 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:14:00.540 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:00.540 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:00.540 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:00.797 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:14:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.914+0000 7fb1bdc17700 1 -- 172.21.15.37:0/757044314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b8100710 msgr2=0x7fb1b8100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.914+0000 7fb1bdc17700 1 --2- 172.21.15.37:0/757044314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b8100710 0x7fb1b8100b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009910 tx=0x7fb1a8009c20 comp rx=0 tx=0).stop 2024-03-26T06:14:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.914+0000 7fb1bdc17700 1 -- 172.21.15.37:0/757044314 shutdown_connections 2024-03-26T06:14:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.914+0000 7fb1bdc17700 1 --2- 172.21.15.37:0/757044314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b8100710 0x7fb1b8100b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.914+0000 7fb1bdc17700 1 -- 172.21.15.37:0/757044314 >> 172.21.15.37:0/757044314 conn(0x7fb1b80fbcb0 msgr2=0x7fb1b80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.914+0000 7fb1bdc17700 1 -- 172.21.15.37:0/757044314 shutdown_connections 2024-03-26T06:14:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.914+0000 7fb1bdc17700 1 -- 172.21.15.37:0/757044314 wait complete. 2024-03-26T06:14:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.915+0000 7fb1bdc17700 1 Processor -- start 2024-03-26T06:14:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.915+0000 7fb1bdc17700 1 -- start start 2024-03-26T06:14:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.915+0000 7fb1bdc17700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b8100710 0x7fb1b8069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:02.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.915+0000 7fb1bdc17700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b81067a0 con 0x7fb1b8100710 2024-03-26T06:14:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.915+0000 7fb1b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b8100710 0x7fb1b8069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.915+0000 7fb1b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b8100710 0x7fb1b8069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49054/0 (socket says 172.21.15.37:49054) 2024-03-26T06:14:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.915+0000 7fb1b77fe700 1 -- 172.21.15.37:0/3283656990 learned_addr learned my addr 172.21.15.37:0/3283656990 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:14:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.915+0000 7fb1b77fe700 1 -- 172.21.15.37:0/3283656990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b8199c70 con 0x7fb1b8100710 2024-03-26T06:14:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.916+0000 7fb1b77fe700 1 --2- 172.21.15.37:0/3283656990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b8100710 0x7fb1b8069c50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8006010 tx=0x7fb1a8005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.916+0000 7fb1b4ff9700 1 -- 172.21.15.37:0/3283656990 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1a8027070 con 0x7fb1b8100710 2024-03-26T06:14:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.916+0000 7fb1b4ff9700 1 -- 172.21.15.37:0/3283656990 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1a800be40 con 0x7fb1b8100710 2024-03-26T06:14:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.916+0000 7fb1bdc17700 1 -- 172.21.15.37:0/3283656990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b8199e40 con 0x7fb1b8100710 2024-03-26T06:14:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.916+0000 7fb1b4ff9700 1 -- 172.21.15.37:0/3283656990 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1a800f460 con 0x7fb1b8100710 2024-03-26T06:14:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.916+0000 7fb1bdc17700 1 -- 172.21.15.37:0/3283656990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b819a1e0 con 0x7fb1b8100710 2024-03-26T06:14:02.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.917+0000 7fb1b4ff9700 1 -- 172.21.15.37:0/3283656990 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fb1a802a070 con 0x7fb1b8100710 2024-03-26T06:14:02.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.917+0000 7fb1bdc17700 1 -- 172.21.15.37:0/3283656990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b8106f20 con 0x7fb1b8100710 2024-03-26T06:14:02.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.917+0000 7fb1b4ff9700 1 --2- 172.21.15.37:0/3283656990 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1a0039020 0x7fb1a003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:02.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.917+0000 7fb1b4ff9700 1 -- 172.21.15.37:0/3283656990 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb1a801e030 con 0x7fb1b8100710 2024-03-26T06:14:02.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.917+0000 7fb1b6ffd700 1 --2- 172.21.15.37:0/3283656990 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1a0039020 0x7fb1a003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:02.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.917+0000 7fb1b6ffd700 1 --2- 172.21.15.37:0/3283656990 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1a0039020 0x7fb1a003b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac006fd0 tx=0x7fb1ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:02.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:02.922+0000 7fb1b4ff9700 1 -- 172.21.15.37:0/3283656990 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb1a8013070 con 0x7fb1b8100710 2024-03-26T06:14:03.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.132+0000 7fb1bdc17700 1 -- 172.21.15.37:0/3283656990 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1b802daf0 con 0x7fb1a0039020 2024-03-26T06:14:03.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.135+0000 7fb1b4ff9700 1 -- 172.21.15.37:0/3283656990 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb1b802daf0 con 0x7fb1a0039020 2024-03-26T06:14:03.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.137+0000 7fb1bdc17700 1 -- 172.21.15.37:0/3283656990 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1a0039020 msgr2=0x7fb1a003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.137+0000 7fb1bdc17700 1 --2- 172.21.15.37:0/3283656990 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1a0039020 0x7fb1a003b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac006fd0 tx=0x7fb1ac006e40 comp rx=0 tx=0).stop 2024-03-26T06:14:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.138+0000 7fb1bdc17700 1 -- 172.21.15.37:0/3283656990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b8100710 msgr2=0x7fb1b8069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.138+0000 7fb1bdc17700 1 --2- 172.21.15.37:0/3283656990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b8100710 0x7fb1b8069c50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8006010 tx=0x7fb1a8005050 comp rx=0 tx=0).stop 2024-03-26T06:14:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.138+0000 7fb1bdc17700 1 -- 172.21.15.37:0/3283656990 shutdown_connections 2024-03-26T06:14:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.138+0000 7fb1bdc17700 1 --2- 172.21.15.37:0/3283656990 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1a0039020 0x7fb1a003b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.138+0000 7fb1bdc17700 1 --2- 172.21.15.37:0/3283656990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b8100710 0x7fb1b8069c50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.138+0000 7fb1bdc17700 1 -- 172.21.15.37:0/3283656990 >> 172.21.15.37:0/3283656990 conn(0x7fb1b80fbcb0 msgr2=0x7fb1b80fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.138+0000 7fb1bdc17700 1 -- 172.21.15.37:0/3283656990 shutdown_connections 2024-03-26T06:14:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:03.138+0000 7fb1bdc17700 1 -- 172.21.15.37:0/3283656990 wait complete. 2024-03-26T06:14:03.992 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi066 2024-03-26T06:14:03.992 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-26T06:14:03.992 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/etc/ceph/ceph.conf 2024-03-26T06:14:04.011 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-26T06:14:04.011 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:14:04.069 INFO:tasks.cephadm:Adding host smithi066 to orchestrator... 2024-03-26T06:14:04.069 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch host add smithi066 2024-03-26T06:14:04.305 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:14:04.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:04 smithi037 ceph-mon[31974]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:14:04.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:04.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:04.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:04.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:04.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:04 smithi037 ceph-mon[31974]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-26T06:14:04.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:04 smithi037 ceph-mon[31974]: Updating smithi037:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:14:04.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:04 smithi037 ceph-mon[31974]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:14:04.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:04.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:04.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:05.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:05 smithi037 ceph-mon[31974]: Updating smithi037:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:14:05.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.953+0000 7f0982c9a700 1 -- 172.21.15.37:0/486060493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f097c1029c0 msgr2=0x7f097c102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.953+0000 7f0982c9a700 1 --2- 172.21.15.37:0/486060493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f097c1029c0 0x7f097c102da0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0970009910 tx=0x7f0970009c20 comp rx=0 tx=0).stop 2024-03-26T06:14:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.953+0000 7f0982c9a700 1 -- 172.21.15.37:0/486060493 shutdown_connections 2024-03-26T06:14:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.953+0000 7f0982c9a700 1 --2- 172.21.15.37:0/486060493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f097c1029c0 0x7f097c102da0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.953+0000 7f0982c9a700 1 -- 172.21.15.37:0/486060493 >> 172.21.15.37:0/486060493 conn(0x7f097c0fc190 msgr2=0x7f097c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.953+0000 7f0982c9a700 1 -- 172.21.15.37:0/486060493 shutdown_connections 2024-03-26T06:14:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.954+0000 7f0982c9a700 1 -- 172.21.15.37:0/486060493 wait complete. 2024-03-26T06:14:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.954+0000 7f0982c9a700 1 Processor -- start 2024-03-26T06:14:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.954+0000 7f0982c9a700 1 -- start start 2024-03-26T06:14:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.954+0000 7f0982c9a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f097c1029c0 0x7f097c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.954+0000 7f0982c9a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c10be40 con 0x7f097c1029c0 2024-03-26T06:14:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.955+0000 7f0980a36700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f097c1029c0 0x7f097c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.955+0000 7f0980a36700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f097c1029c0 0x7f097c197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49082/0 (socket says 172.21.15.37:49082) 2024-03-26T06:14:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.955+0000 7f0980a36700 1 -- 172.21.15.37:0/1902476726 learned_addr learned my addr 172.21.15.37:0/1902476726 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:14:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.955+0000 7f0980a36700 1 -- 172.21.15.37:0/1902476726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c1978b0 con 0x7f097c1029c0 2024-03-26T06:14:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.955+0000 7f0980a36700 1 --2- 172.21.15.37:0/1902476726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f097c1029c0 0x7f097c197370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0970006010 tx=0x7f0970005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.955+0000 7f0979ffb700 1 -- 172.21.15.37:0/1902476726 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0970026070 con 0x7f097c1029c0 2024-03-26T06:14:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.955+0000 7f0979ffb700 1 -- 172.21.15.37:0/1902476726 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f097000bc30 con 0x7f097c1029c0 2024-03-26T06:14:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.955+0000 7f0982c9a700 1 -- 172.21.15.37:0/1902476726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097c197b40 con 0x7f097c1029c0 2024-03-26T06:14:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.955+0000 7f0979ffb700 1 -- 172.21.15.37:0/1902476726 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0970003ab0 con 0x7f097c1029c0 2024-03-26T06:14:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.955+0000 7f0982c9a700 1 -- 172.21.15.37:0/1902476726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097c197fa0 con 0x7f097c1029c0 2024-03-26T06:14:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.956+0000 7f0979ffb700 1 -- 172.21.15.37:0/1902476726 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f0970029070 con 0x7f097c1029c0 2024-03-26T06:14:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.956+0000 7f0982c9a700 1 -- 172.21.15.37:0/1902476726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097c06a2e0 con 0x7f097c1029c0 2024-03-26T06:14:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.956+0000 7f0979ffb700 1 --2- 172.21.15.37:0/1902476726 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f096c038fc0 0x7f096c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.957+0000 7f0979ffb700 1 -- 172.21.15.37:0/1902476726 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f097001e070 con 0x7f097c1029c0 2024-03-26T06:14:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.957+0000 7f097bfff700 1 --2- 172.21.15.37:0/1902476726 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f096c038fc0 0x7f096c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.957+0000 7f097bfff700 1 --2- 172.21.15.37:0/1902476726 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f096c038fc0 0x7f096c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0968006fd0 tx=0x7f0968006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:05.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:05.962+0000 7f0979ffb700 1 -- 172.21.15.37:0/1902476726 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0970051070 con 0x7f097c1029c0 2024-03-26T06:14:06.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:06.171+0000 7f0982c9a700 1 -- 172.21.15.37:0/1902476726 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi066", "target": ["mon-mgr", ""]}) v1 -- 0x7f097c198740 con 0x7f096c038fc0 2024-03-26T06:14:07.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:06 smithi037 ceph-mon[31974]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi066", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:14:08.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:07 smithi037 ceph-mon[31974]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:08.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:07 smithi037 ceph-mon[31974]: Deploying cephadm binary to smithi066 2024-03-26T06:14:10.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:09 smithi037 ceph-mon[31974]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:10.527 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi066' with addr '172.21.15.66' 2024-03-26T06:14:10.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:10.527+0000 7f0979ffb700 1 -- 172.21.15.37:0/1902476726 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f097c198740 con 0x7f096c038fc0 2024-03-26T06:14:10.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:10.530+0000 7f0982c9a700 1 -- 172.21.15.37:0/1902476726 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f096c038fc0 msgr2=0x7f096c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:10.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:10.530+0000 7f0982c9a700 1 --2- 172.21.15.37:0/1902476726 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f096c038fc0 0x7f096c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0968006fd0 tx=0x7f0968006e40 comp rx=0 tx=0).stop 2024-03-26T06:14:10.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:10.530+0000 7f0982c9a700 1 -- 172.21.15.37:0/1902476726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f097c1029c0 msgr2=0x7f097c197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:10.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:10.530+0000 7f0982c9a700 1 --2- 172.21.15.37:0/1902476726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f097c1029c0 0x7f097c197370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0970006010 tx=0x7f0970005050 comp rx=0 tx=0).stop 2024-03-26T06:14:10.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:10.530+0000 7f0982c9a700 1 -- 172.21.15.37:0/1902476726 shutdown_connections 2024-03-26T06:14:10.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:10.530+0000 7f0982c9a700 1 --2- 172.21.15.37:0/1902476726 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f096c038fc0 0x7f096c03b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:10.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:10.530+0000 7f0982c9a700 1 --2- 172.21.15.37:0/1902476726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f097c1029c0 0x7f097c197370 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:10.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:10.530+0000 7f0982c9a700 1 -- 172.21.15.37:0/1902476726 >> 172.21.15.37:0/1902476726 conn(0x7f097c0fc190 msgr2=0x7f097c103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:10.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:10.530+0000 7f0982c9a700 1 -- 172.21.15.37:0/1902476726 shutdown_connections 2024-03-26T06:14:10.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:10.530+0000 7f0982c9a700 1 -- 172.21.15.37:0/1902476726 wait complete. 2024-03-26T06:14:11.211 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T06:14:11.455 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:14:11.830 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:11 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:11.830 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:11 smithi037 ceph-mon[31974]: Added host smithi066 2024-03-26T06:14:11.830 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:11 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:11.830 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:11 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:12.856 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:12 smithi037 ceph-mon[31974]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:12.856 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:12 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:13.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.058+0000 7fcd05abf700 1 -- 172.21.15.37:0/2681786302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd001066b0 msgr2=0x7fcd00106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.058+0000 7fcd05abf700 1 --2- 172.21.15.37:0/2681786302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd001066b0 0x7fcd00106a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009910 tx=0x7fccf0009c20 comp rx=0 tx=0).stop 2024-03-26T06:14:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.058+0000 7fcd05abf700 1 -- 172.21.15.37:0/2681786302 shutdown_connections 2024-03-26T06:14:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.058+0000 7fcd05abf700 1 --2- 172.21.15.37:0/2681786302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd001066b0 0x7fcd00106a90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.058+0000 7fcd05abf700 1 -- 172.21.15.37:0/2681786302 >> 172.21.15.37:0/2681786302 conn(0x7fcd00077320 msgr2=0x7fcd000fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.058+0000 7fcd05abf700 1 -- 172.21.15.37:0/2681786302 shutdown_connections 2024-03-26T06:14:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.058+0000 7fcd05abf700 1 -- 172.21.15.37:0/2681786302 wait complete. 2024-03-26T06:14:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.059+0000 7fcd05abf700 1 Processor -- start 2024-03-26T06:14:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.059+0000 7fcd05abf700 1 -- start start 2024-03-26T06:14:13.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.059+0000 7fcd05abf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd001066b0 0x7fcd00198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.059+0000 7fcd05abf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0010acf0 con 0x7fcd001066b0 2024-03-26T06:14:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.059+0000 7fccff7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd001066b0 0x7fcd00198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.059+0000 7fccff7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd001066b0 0x7fcd00198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52576/0 (socket says 172.21.15.37:52576) 2024-03-26T06:14:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.059+0000 7fccff7fe700 1 -- 172.21.15.37:0/1007767127 learned_addr learned my addr 172.21.15.37:0/1007767127 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:14:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.060+0000 7fccff7fe700 1 -- 172.21.15.37:0/1007767127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00198680 con 0x7fcd001066b0 2024-03-26T06:14:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.060+0000 7fccff7fe700 1 --2- 172.21.15.37:0/1007767127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd001066b0 0x7fcd00198140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fccf0005850 tx=0x7fccf0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.060+0000 7fccfcff9700 1 -- 172.21.15.37:0/1007767127 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf0026070 con 0x7fcd001066b0 2024-03-26T06:14:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.060+0000 7fcd05abf700 1 -- 172.21.15.37:0/1007767127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd00198970 con 0x7fcd001066b0 2024-03-26T06:14:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.060+0000 7fccfcff9700 1 -- 172.21.15.37:0/1007767127 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccf000bbd0 con 0x7fcd001066b0 2024-03-26T06:14:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.060+0000 7fccfcff9700 1 -- 172.21.15.37:0/1007767127 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf0003970 con 0x7fcd001066b0 2024-03-26T06:14:13.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.060+0000 7fcd05abf700 1 -- 172.21.15.37:0/1007767127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd00198e30 con 0x7fcd001066b0 2024-03-26T06:14:13.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.061+0000 7fccfcff9700 1 -- 172.21.15.37:0/1007767127 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fccf0029070 con 0x7fcd001066b0 2024-03-26T06:14:13.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.061+0000 7fcd05abf700 1 -- 172.21.15.37:0/1007767127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd00101df0 con 0x7fcd001066b0 2024-03-26T06:14:13.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.061+0000 7fccfcff9700 1 --2- 172.21.15.37:0/1007767127 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcce8038fc0 0x7fcce803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:13.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.061+0000 7fccfcff9700 1 -- 172.21.15.37:0/1007767127 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fccf001e070 con 0x7fcd001066b0 2024-03-26T06:14:13.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.061+0000 7fccfeffd700 1 --2- 172.21.15.37:0/1007767127 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcce8038fc0 0x7fcce803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:13.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.061+0000 7fccfeffd700 1 --2- 172.21.15.37:0/1007767127 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcce8038fc0 0x7fcce803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fccf4006fd0 tx=0x7fccf4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:13.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.066+0000 7fccfcff9700 1 -- 172.21.15.37:0/1007767127 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fccf0013070 con 0x7fcd001066b0 2024-03-26T06:14:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.295+0000 7fcd05abf700 1 -- 172.21.15.37:0/1007767127 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd00069770 con 0x7fcce8038fc0 2024-03-26T06:14:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.296+0000 7fccfcff9700 1 -- 172.21.15.37:0/1007767127 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fcd00069770 con 0x7fcce8038fc0 2024-03-26T06:14:13.296 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:14:13.296 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}] 2024-03-26T06:14:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.298+0000 7fcd05abf700 1 -- 172.21.15.37:0/1007767127 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcce8038fc0 msgr2=0x7fcce803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.298+0000 7fcd05abf700 1 --2- 172.21.15.37:0/1007767127 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcce8038fc0 0x7fcce803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fccf4006fd0 tx=0x7fccf4006e40 comp rx=0 tx=0).stop 2024-03-26T06:14:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.298+0000 7fcd05abf700 1 -- 172.21.15.37:0/1007767127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd001066b0 msgr2=0x7fcd00198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.298+0000 7fcd05abf700 1 --2- 172.21.15.37:0/1007767127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd001066b0 0x7fcd00198140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fccf0005850 tx=0x7fccf0005030 comp rx=0 tx=0).stop 2024-03-26T06:14:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.298+0000 7fcd05abf700 1 -- 172.21.15.37:0/1007767127 shutdown_connections 2024-03-26T06:14:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.298+0000 7fcd05abf700 1 --2- 172.21.15.37:0/1007767127 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcce8038fc0 0x7fcce803b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.298+0000 7fcd05abf700 1 --2- 172.21.15.37:0/1007767127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd001066b0 0x7fcd00198140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.298+0000 7fcd05abf700 1 -- 172.21.15.37:0/1007767127 >> 172.21.15.37:0/1007767127 conn(0x7fcd00077320 msgr2=0x7fcd001081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.298+0000 7fcd05abf700 1 -- 172.21.15.37:0/1007767127 shutdown_connections 2024-03-26T06:14:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:13.298+0000 7fcd05abf700 1 -- 172.21.15.37:0/1007767127 wait complete. 2024-03-26T06:14:13.918 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-03-26T06:14:13.918 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-26T06:14:13.918 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-03-26T06:14:13.937 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-26T06:14:13.937 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:14:13.994 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-03-26T06:14:13.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch host add smithi149 2024-03-26T06:14:14.236 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:14:14.463 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:14 smithi037 ceph-mon[31974]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:14.463 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:14 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:14.463 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:14 smithi037 ceph-mon[31974]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:14:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.850+0000 7f11d99a2700 1 -- 172.21.15.37:0/4275512964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d41077b0 msgr2=0x7f11d4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.850+0000 7f11d99a2700 1 --2- 172.21.15.37:0/4275512964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d41077b0 0x7f11d4107b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f11c400b0a0 tx=0x7f11c400b3b0 comp rx=0 tx=0).stop 2024-03-26T06:14:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.850+0000 7f11d99a2700 1 -- 172.21.15.37:0/4275512964 shutdown_connections 2024-03-26T06:14:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.850+0000 7f11d99a2700 1 --2- 172.21.15.37:0/4275512964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d41077b0 0x7f11d4107b90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.850+0000 7f11d99a2700 1 -- 172.21.15.37:0/4275512964 >> 172.21.15.37:0/4275512964 conn(0x7f11d40fd2c0 msgr2=0x7f11d40ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:15.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.850+0000 7f11d99a2700 1 -- 172.21.15.37:0/4275512964 shutdown_connections 2024-03-26T06:14:15.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.850+0000 7f11d99a2700 1 -- 172.21.15.37:0/4275512964 wait complete. 2024-03-26T06:14:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.851+0000 7f11d99a2700 1 Processor -- start 2024-03-26T06:14:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.851+0000 7f11d99a2700 1 -- start start 2024-03-26T06:14:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.851+0000 7f11d99a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d41077b0 0x7f11d4197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.851+0000 7f11d99a2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d410bdf0 con 0x7f11d41077b0 2024-03-26T06:14:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.851+0000 7f11d2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d41077b0 0x7f11d4197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.851+0000 7f11d2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d41077b0 0x7f11d4197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52596/0 (socket says 172.21.15.37:52596) 2024-03-26T06:14:15.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.851+0000 7f11d2ffd700 1 -- 172.21.15.37:0/3580754883 learned_addr learned my addr 172.21.15.37:0/3580754883 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:14:15.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.852+0000 7f11d2ffd700 1 -- 172.21.15.37:0/3580754883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d4197750 con 0x7f11d41077b0 2024-03-26T06:14:15.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.852+0000 7f11d2ffd700 1 --2- 172.21.15.37:0/3580754883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d41077b0 0x7f11d4197210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f11c4015040 tx=0x7f11c4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:15.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.852+0000 7f11d89a0700 1 -- 172.21.15.37:0/3580754883 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11c4017040 con 0x7f11d41077b0 2024-03-26T06:14:15.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.852+0000 7f11d89a0700 1 -- 172.21.15.37:0/3580754883 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11c4022070 con 0x7f11d41077b0 2024-03-26T06:14:15.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.852+0000 7f11d99a2700 1 -- 172.21.15.37:0/3580754883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d41979e0 con 0x7f11d41077b0 2024-03-26T06:14:15.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.852+0000 7f11d89a0700 1 -- 172.21.15.37:0/3580754883 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11c401d5d0 con 0x7f11d41077b0 2024-03-26T06:14:15.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.852+0000 7f11d99a2700 1 -- 172.21.15.37:0/3580754883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d419bae0 con 0x7f11d41077b0 2024-03-26T06:14:15.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.853+0000 7f11d89a0700 1 -- 172.21.15.37:0/3580754883 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f11c4004000 con 0x7f11d41077b0 2024-03-26T06:14:15.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.853+0000 7f11d99a2700 1 -- 172.21.15.37:0/3580754883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d4102ef0 con 0x7f11d41077b0 2024-03-26T06:14:15.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.853+0000 7f11d89a0700 1 --2- 172.21.15.37:0/3580754883 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f11bc038fc0 0x7f11bc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:15.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.853+0000 7f11d89a0700 1 -- 172.21.15.37:0/3580754883 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f11c400f050 con 0x7f11d41077b0 2024-03-26T06:14:15.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.853+0000 7f11d27fc700 1 --2- 172.21.15.37:0/3580754883 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f11bc038fc0 0x7f11bc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:15.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.854+0000 7f11d27fc700 1 --2- 172.21.15.37:0/3580754883 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f11bc038fc0 0x7f11bc03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f11c8006fd0 tx=0x7f11c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:15.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:15.858+0000 7f11d89a0700 1 -- 172.21.15.37:0/3580754883 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f11c4007e20 con 0x7f11d41077b0 2024-03-26T06:14:16.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:16.065+0000 7f11d99a2700 1 -- 172.21.15.37:0/3580754883 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) v1 -- 0x7f11d4198570 con 0x7f11bc038fc0 2024-03-26T06:14:16.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:16 smithi037 ceph-mon[31974]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:17.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:17 smithi037 ceph-mon[31974]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:14:17.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:17.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:17.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:17.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:17.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:14:17.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:17.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:17.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:17 smithi037 ceph-mon[31974]: Updating smithi066:/etc/ceph/ceph.conf 2024-03-26T06:14:18.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:18 smithi037 ceph-mon[31974]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:18.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:18 smithi037 ceph-mon[31974]: Deploying cephadm binary to smithi149 2024-03-26T06:14:18.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:18 smithi037 ceph-mon[31974]: Updating smithi066:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:14:18.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:18 smithi037 ceph-mon[31974]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:14:18.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:18 smithi037 ceph-mon[31974]: Updating smithi066:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:14:20.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:20.414+0000 7f11d89a0700 1 -- 172.21.15.37:0/3580754883 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f11d4198570 con 0x7f11bc038fc0 2024-03-26T06:14:20.415 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-03-26T06:14:20.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:20.416+0000 7f11d99a2700 1 -- 172.21.15.37:0/3580754883 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f11bc038fc0 msgr2=0x7f11bc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:20.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:20.416+0000 7f11d99a2700 1 --2- 172.21.15.37:0/3580754883 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f11bc038fc0 0x7f11bc03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f11c8006fd0 tx=0x7f11c8006e40 comp rx=0 tx=0).stop 2024-03-26T06:14:20.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:20.416+0000 7f11d99a2700 1 -- 172.21.15.37:0/3580754883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d41077b0 msgr2=0x7f11d4197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:20.416+0000 7f11d99a2700 1 --2- 172.21.15.37:0/3580754883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d41077b0 0x7f11d4197210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f11c4015040 tx=0x7f11c4003d50 comp rx=0 tx=0).stop 2024-03-26T06:14:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:20.416+0000 7f11d99a2700 1 -- 172.21.15.37:0/3580754883 shutdown_connections 2024-03-26T06:14:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:20.416+0000 7f11d99a2700 1 --2- 172.21.15.37:0/3580754883 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f11bc038fc0 0x7f11bc03b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:20.416+0000 7f11d99a2700 1 --2- 172.21.15.37:0/3580754883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d41077b0 0x7f11d4197210 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:20.416+0000 7f11d99a2700 1 -- 172.21.15.37:0/3580754883 >> 172.21.15.37:0/3580754883 conn(0x7f11d40fd2c0 msgr2=0x7f11d40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:20.417+0000 7f11d99a2700 1 -- 172.21.15.37:0/3580754883 shutdown_connections 2024-03-26T06:14:20.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:20.417+0000 7f11d99a2700 1 -- 172.21.15.37:0/3580754883 wait complete. 2024-03-26T06:14:20.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:20 smithi037 ceph-mon[31974]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:20.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:20 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:20.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:20 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:20.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:20 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:20.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:20 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:20.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:20 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:21.135 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T06:14:21.376 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:14:21.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:21 smithi037 ceph-mon[31974]: Added host smithi149 2024-03-26T06:14:21.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:21.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:22.680 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:22 smithi037 ceph-mon[31974]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:22.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6393224700 1 -- 172.21.15.37:0/1247081355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c107840 msgr2=0x7f638c107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:22.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6393224700 1 --2- 172.21.15.37:0/1247081355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c107840 0x7f638c107c20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6374009910 tx=0x7f6374009c20 comp rx=0 tx=0).stop 2024-03-26T06:14:22.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6393224700 1 -- 172.21.15.37:0/1247081355 shutdown_connections 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6393224700 1 --2- 172.21.15.37:0/1247081355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c107840 0x7f638c107c20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6393224700 1 -- 172.21.15.37:0/1247081355 >> 172.21.15.37:0/1247081355 conn(0x7f638c0fb930 msgr2=0x7f638c0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6393224700 1 -- 172.21.15.37:0/1247081355 shutdown_connections 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6393224700 1 -- 172.21.15.37:0/1247081355 wait complete. 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6393224700 1 Processor -- start 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6393224700 1 -- start start 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6393224700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c107840 0x7f638c100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6393224700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c10be40 con 0x7f638c107840 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.890+0000 7f6390fc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c107840 0x7f638c100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.891+0000 7f6390fc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c107840 0x7f638c100df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53510/0 (socket says 172.21.15.37:53510) 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.891+0000 7f6390fc0700 1 -- 172.21.15.37:0/2775812217 learned_addr learned my addr 172.21.15.37:0/2775812217 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:14:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.891+0000 7f6390fc0700 1 -- 172.21.15.37:0/2775812217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c102ce0 con 0x7f638c107840 2024-03-26T06:14:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.891+0000 7f6390fc0700 1 --2- 172.21.15.37:0/2775812217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c107840 0x7f638c100df0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6374005850 tx=0x7f6374005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.891+0000 7f6389ffb700 1 -- 172.21.15.37:0/2775812217 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6374026070 con 0x7f638c107840 2024-03-26T06:14:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.891+0000 7f6389ffb700 1 -- 172.21.15.37:0/2775812217 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f637400bc30 con 0x7f638c107840 2024-03-26T06:14:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.891+0000 7f6393224700 1 -- 172.21.15.37:0/2775812217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c101330 con 0x7f638c107840 2024-03-26T06:14:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.891+0000 7f6389ffb700 1 -- 172.21.15.37:0/2775812217 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6374003a20 con 0x7f638c107840 2024-03-26T06:14:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.891+0000 7f6393224700 1 -- 172.21.15.37:0/2775812217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c1017f0 con 0x7f638c107840 2024-03-26T06:14:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.892+0000 7f6389ffb700 1 -- 172.21.15.37:0/2775812217 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f6374029070 con 0x7f638c107840 2024-03-26T06:14:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.892+0000 7f6393224700 1 -- 172.21.15.37:0/2775812217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c063410 con 0x7f638c107840 2024-03-26T06:14:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.892+0000 7f6389ffb700 1 --2- 172.21.15.37:0/2775812217 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6378039020 0x7f637803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.892+0000 7f6389ffb700 1 -- 172.21.15.37:0/2775812217 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f637401e070 con 0x7f638c107840 2024-03-26T06:14:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.892+0000 7f638bfff700 1 --2- 172.21.15.37:0/2775812217 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6378039020 0x7f637803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.893+0000 7f638bfff700 1 --2- 172.21.15.37:0/2775812217 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6378039020 0x7f637803b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f637c006fd0 tx=0x7f637c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:22.897+0000 7f6389ffb700 1 -- 172.21.15.37:0/2775812217 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6374013070 con 0x7f638c107840 2024-03-26T06:14:23.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.110+0000 7f6393224700 1 -- 172.21.15.37:0/2775812217 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f638c101a20 con 0x7f6378039020 2024-03-26T06:14:23.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.111+0000 7f6389ffb700 1 -- 172.21.15.37:0/2775812217 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f638c101a20 con 0x7f6378039020 2024-03-26T06:14:23.111 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:14:23.111 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-03-26T06:14:23.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.113+0000 7f6393224700 1 -- 172.21.15.37:0/2775812217 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6378039020 msgr2=0x7f637803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:23.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.113+0000 7f6393224700 1 --2- 172.21.15.37:0/2775812217 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6378039020 0x7f637803b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f637c006fd0 tx=0x7f637c006e40 comp rx=0 tx=0).stop 2024-03-26T06:14:23.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.113+0000 7f6393224700 1 -- 172.21.15.37:0/2775812217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c107840 msgr2=0x7f638c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:23.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.113+0000 7f6393224700 1 --2- 172.21.15.37:0/2775812217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c107840 0x7f638c100df0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6374005850 tx=0x7f6374005030 comp rx=0 tx=0).stop 2024-03-26T06:14:23.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.113+0000 7f6393224700 1 -- 172.21.15.37:0/2775812217 shutdown_connections 2024-03-26T06:14:23.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.113+0000 7f6393224700 1 --2- 172.21.15.37:0/2775812217 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6378039020 0x7f637803b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:23.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.113+0000 7f6393224700 1 --2- 172.21.15.37:0/2775812217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c107840 0x7f638c100df0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:23.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.113+0000 7f6393224700 1 -- 172.21.15.37:0/2775812217 >> 172.21.15.37:0/2775812217 conn(0x7f638c0fb930 msgr2=0x7f638c0fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:23.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.113+0000 7f6393224700 1 -- 172.21.15.37:0/2775812217 shutdown_connections 2024-03-26T06:14:23.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:23.113+0000 7f6393224700 1 -- 172.21.15.37:0/2775812217 wait complete. 2024-03-26T06:14:23.780 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-26T06:14:23.780 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-26T06:14:24.021 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:14:24.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:23 smithi037 ceph-mon[31974]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:24.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:24.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:23 smithi037 ceph-mon[31974]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:14:25.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.503+0000 7ff490922700 1 -- 172.21.15.37:0/809492016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff48c101110 msgr2=0x7ff48c1014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:25.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.503+0000 7ff490922700 1 --2- 172.21.15.37:0/809492016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff48c101110 0x7ff48c1014f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff48400b0a0 tx=0x7ff48400b3b0 comp rx=0 tx=0).stop 2024-03-26T06:14:25.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.503+0000 7ff490922700 1 -- 172.21.15.37:0/809492016 shutdown_connections 2024-03-26T06:14:25.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.503+0000 7ff490922700 1 --2- 172.21.15.37:0/809492016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff48c101110 0x7ff48c1014f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:25.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.503+0000 7ff490922700 1 -- 172.21.15.37:0/809492016 >> 172.21.15.37:0/809492016 conn(0x7ff48c076d90 msgr2=0x7ff48c0771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:25.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.503+0000 7ff490922700 1 -- 172.21.15.37:0/809492016 shutdown_connections 2024-03-26T06:14:25.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.503+0000 7ff490922700 1 -- 172.21.15.37:0/809492016 wait complete. 2024-03-26T06:14:25.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.504+0000 7ff490922700 1 Processor -- start 2024-03-26T06:14:25.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.504+0000 7ff490922700 1 -- start start 2024-03-26T06:14:25.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.504+0000 7ff490922700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff48c101110 0x7ff48c1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:25.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.504+0000 7ff490922700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c10ad00 con 0x7ff48c101110 2024-03-26T06:14:25.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.504+0000 7ff48a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff48c101110 0x7ff48c1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:25.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.504+0000 7ff48a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff48c101110 0x7ff48c1971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53532/0 (socket says 172.21.15.37:53532) 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.504+0000 7ff48a59c700 1 -- 172.21.15.37:0/1597416697 learned_addr learned my addr 172.21.15.37:0/1597416697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.504+0000 7ff48a59c700 1 -- 172.21.15.37:0/1597416697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c197700 con 0x7ff48c101110 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.505+0000 7ff48a59c700 1 --2- 172.21.15.37:0/1597416697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff48c101110 0x7ff48c1971c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff484015040 tx=0x7ff484003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.505+0000 7ff4837fe700 1 -- 172.21.15.37:0/1597416697 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff484017040 con 0x7ff48c101110 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.505+0000 7ff4837fe700 1 -- 172.21.15.37:0/1597416697 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff484022070 con 0x7ff48c101110 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.505+0000 7ff490922700 1 -- 172.21.15.37:0/1597416697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c197990 con 0x7ff48c101110 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.505+0000 7ff4837fe700 1 -- 172.21.15.37:0/1597416697 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff48401d5d0 con 0x7ff48c101110 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.505+0000 7ff490922700 1 -- 172.21.15.37:0/1597416697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c19ba90 con 0x7ff48c101110 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.506+0000 7ff4837fe700 1 -- 172.21.15.37:0/1597416697 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7ff48401d8d0 con 0x7ff48c101110 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.506+0000 7ff490922700 1 -- 172.21.15.37:0/1597416697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c101ec0 con 0x7ff48c101110 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.506+0000 7ff4837fe700 1 --2- 172.21.15.37:0/1597416697 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff478038fc0 0x7ff47803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.506+0000 7ff4837fe700 1 -- 172.21.15.37:0/1597416697 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff48400f050 con 0x7ff48c101110 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.506+0000 7ff489d9b700 1 --2- 172.21.15.37:0/1597416697 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff478038fc0 0x7ff47803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:25.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.506+0000 7ff489d9b700 1 --2- 172.21.15.37:0/1597416697 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff478038fc0 0x7ff47803b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff474006fd0 tx=0x7ff474006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.511+0000 7ff4837fe700 1 -- 172.21.15.37:0/1597416697 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff484007e20 con 0x7ff48c101110 2024-03-26T06:14:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.722+0000 7ff490922700 1 -- 172.21.15.37:0/1597416697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff48c198520 con 0x7ff48c101110 2024-03-26T06:14:25.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.987+0000 7ff4837fe700 1 -- 172.21.15.37:0/1597416697 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff48402c200 con 0x7ff48c101110 2024-03-26T06:14:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.991+0000 7ff490922700 1 -- 172.21.15.37:0/1597416697 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff478038fc0 msgr2=0x7ff47803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.991+0000 7ff490922700 1 --2- 172.21.15.37:0/1597416697 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff478038fc0 0x7ff47803b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff474006fd0 tx=0x7ff474006e40 comp rx=0 tx=0).stop 2024-03-26T06:14:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.991+0000 7ff490922700 1 -- 172.21.15.37:0/1597416697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff48c101110 msgr2=0x7ff48c1971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.991+0000 7ff490922700 1 --2- 172.21.15.37:0/1597416697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff48c101110 0x7ff48c1971c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff484015040 tx=0x7ff484003d50 comp rx=0 tx=0).stop 2024-03-26T06:14:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.991+0000 7ff490922700 1 -- 172.21.15.37:0/1597416697 shutdown_connections 2024-03-26T06:14:25.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.991+0000 7ff490922700 1 --2- 172.21.15.37:0/1597416697 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff478038fc0 0x7ff47803b480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.991+0000 7ff490922700 1 --2- 172.21.15.37:0/1597416697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff48c101110 0x7ff48c1971c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.991+0000 7ff490922700 1 -- 172.21.15.37:0/1597416697 >> 172.21.15.37:0/1597416697 conn(0x7ff48c076d90 msgr2=0x7ff48c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.991+0000 7ff490922700 1 -- 172.21.15.37:0/1597416697 shutdown_connections 2024-03-26T06:14:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:25.991+0000 7ff490922700 1 -- 172.21.15.37:0/1597416697 wait complete. 2024-03-26T06:14:25.992 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-03-26T06:14:26.311 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:25 smithi037 ceph-mon[31974]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:26.312 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:25 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1597416697' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T06:14:26.656 INFO:tasks.cephadm:Adding mon.a on smithi037 2024-03-26T06:14:26.656 INFO:tasks.cephadm:Adding mon.b on smithi066 2024-03-26T06:14:26.656 INFO:tasks.cephadm:Adding mon.c on smithi149 2024-03-26T06:14:26.656 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi037:172.21.15.37=a;smithi066:172.21.15.66=b;smithi149:172.21.15.149=c' 2024-03-26T06:14:26.891 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:14:27.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1597416697' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T06:14:27.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T06:14:27.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:27.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:27.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:27.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:27.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:14:27.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:27.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:27.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-26T06:14:27.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: Updating smithi149:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:14:27.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:14:27.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:27.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:27.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.411+0000 7ff142a88700 1 -- 172.21.15.149:0/550715438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c1077c0 msgr2=0x7ff13c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.411+0000 7ff142a88700 1 --2- 172.21.15.149:0/550715438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c1077c0 0x7ff13c107ba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff138009910 tx=0x7ff138009c20 comp rx=0 tx=0).stop 2024-03-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.411+0000 7ff142a88700 1 -- 172.21.15.149:0/550715438 shutdown_connections 2024-03-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.411+0000 7ff142a88700 1 --2- 172.21.15.149:0/550715438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c1077c0 0x7ff13c107ba0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.411+0000 7ff142a88700 1 -- 172.21.15.149:0/550715438 >> 172.21.15.149:0/550715438 conn(0x7ff13c0fd2f0 msgr2=0x7ff13c0ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.411+0000 7ff142a88700 1 -- 172.21.15.149:0/550715438 shutdown_connections 2024-03-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.412+0000 7ff142a88700 1 -- 172.21.15.149:0/550715438 wait complete. 2024-03-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.412+0000 7ff142a88700 1 Processor -- start 2024-03-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.412+0000 7ff142a88700 1 -- start start 2024-03-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.412+0000 7ff142a88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c1077c0 0x7ff13c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:28.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.412+0000 7ff142a88700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c10be00 con 0x7ff13c1077c0 2024-03-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.413+0000 7ff140824700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c1077c0 0x7ff13c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.413+0000 7ff140824700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c1077c0 0x7ff13c197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.149:41834/0 (socket says 172.21.15.149:41834) 2024-03-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.413+0000 7ff140824700 1 -- 172.21.15.149:0/3031636225 learned_addr learned my addr 172.21.15.149:0/3031636225 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.413+0000 7ff140824700 1 -- 172.21.15.149:0/3031636225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c197840 con 0x7ff13c1077c0 2024-03-26T06:14:28.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.413+0000 7ff140824700 1 --2- 172.21.15.149:0/3031636225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c1077c0 0x7ff13c197300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff138005850 tx=0x7ff138005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.413+0000 7ff131ffb700 1 -- 172.21.15.149:0/3031636225 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff138026070 con 0x7ff13c1077c0 2024-03-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.413+0000 7ff142a88700 1 -- 172.21.15.149:0/3031636225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c197b30 con 0x7ff13c1077c0 2024-03-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.413+0000 7ff131ffb700 1 -- 172.21.15.149:0/3031636225 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff13800bbd0 con 0x7ff13c1077c0 2024-03-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.413+0000 7ff131ffb700 1 -- 172.21.15.149:0/3031636225 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff138003970 con 0x7ff13c1077c0 2024-03-26T06:14:28.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.414+0000 7ff142a88700 1 -- 172.21.15.149:0/3031636225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c19bbd0 con 0x7ff13c1077c0 2024-03-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.414+0000 7ff142a88700 1 -- 172.21.15.149:0/3031636225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c102f00 con 0x7ff13c1077c0 2024-03-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.415+0000 7ff131ffb700 1 -- 172.21.15.149:0/3031636225 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7ff138029070 con 0x7ff13c1077c0 2024-03-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.415+0000 7ff131ffb700 1 --2- 172.21.15.149:0/3031636225 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff124038fc0 0x7ff12403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.415+0000 7ff131ffb700 1 -- 172.21.15.149:0/3031636225 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff13801e070 con 0x7ff13c1077c0 2024-03-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.415+0000 7ff133fff700 1 --2- 172.21.15.149:0/3031636225 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff124038fc0 0x7ff12403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:28.417 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.415+0000 7ff133fff700 1 --2- 172.21.15.149:0/3031636225 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff124038fc0 0x7ff12403b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff12c006fd0 tx=0x7ff12c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:28.420 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.420+0000 7ff131ffb700 1 -- 172.21.15.149:0/3031636225 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff138013070 con 0x7ff13c1077c0 2024-03-26T06:14:28.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:28 smithi037 ceph-mon[31974]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:28.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:28 smithi037 ceph-mon[31974]: Updating smithi149:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:14:28.624 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.624+0000 7ff142a88700 1 -- 172.21.15.149:0/3031636225 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi066:172.21.15.66=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}) v1 -- 0x7ff13c198660 con 0x7ff124038fc0 2024-03-26T06:14:28.631 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.631+0000 7ff131ffb700 1 -- 172.21.15.149:0/3031636225 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff13c198660 con 0x7ff124038fc0 2024-03-26T06:14:28.631 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mon update... 2024-03-26T06:14:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.633+0000 7ff142a88700 1 -- 172.21.15.149:0/3031636225 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff124038fc0 msgr2=0x7ff12403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.633+0000 7ff142a88700 1 --2- 172.21.15.149:0/3031636225 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff124038fc0 0x7ff12403b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff12c006fd0 tx=0x7ff12c006e40 comp rx=0 tx=0).stop 2024-03-26T06:14:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.633+0000 7ff142a88700 1 -- 172.21.15.149:0/3031636225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c1077c0 msgr2=0x7ff13c197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.633+0000 7ff142a88700 1 --2- 172.21.15.149:0/3031636225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c1077c0 0x7ff13c197300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff138005850 tx=0x7ff138005030 comp rx=0 tx=0).stop 2024-03-26T06:14:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.633+0000 7ff142a88700 1 -- 172.21.15.149:0/3031636225 shutdown_connections 2024-03-26T06:14:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.633+0000 7ff142a88700 1 --2- 172.21.15.149:0/3031636225 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff124038fc0 0x7ff12403b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.633+0000 7ff142a88700 1 --2- 172.21.15.149:0/3031636225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff13c1077c0 0x7ff13c197300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.633+0000 7ff142a88700 1 -- 172.21.15.149:0/3031636225 >> 172.21.15.149:0/3031636225 conn(0x7ff13c0fd2f0 msgr2=0x7ff13c109330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.633+0000 7ff142a88700 1 -- 172.21.15.149:0/3031636225 shutdown_connections 2024-03-26T06:14:28.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:28.633+0000 7ff142a88700 1 -- 172.21.15.149:0/3031636225 wait complete. 2024-03-26T06:14:29.273 DEBUG:teuthology.orchestra.run.smithi066:mon.b> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.b.service 2024-03-26T06:14:29.275 DEBUG:teuthology.orchestra.run.smithi149:mon.c> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.c.service 2024-03-26T06:14:29.278 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T06:14:29.278 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T06:14:29.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:29 smithi037 ceph-mon[31974]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi066:172.21.15.66=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:14:29.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:29 smithi037 ceph-mon[31974]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi066:172.21.15.66=b;smithi149:172.21.15.149=c;count:3 2024-03-26T06:14:29.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:29 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:29.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:29 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:29.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:29 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:29.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:29 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:29.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:29 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:29.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:29 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:29.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:29 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:29.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:29 smithi037 ceph-mon[31974]: Deploying daemon mon.c on smithi149 2024-03-26T06:14:30.364 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:14:30.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:30 smithi037 ceph-mon[31974]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:32.633 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:32 smithi149 systemd[1]: Starting Ceph mon.c for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:14:32.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:32 smithi037 ceph-mon[31974]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:33.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 podman[36654]: 2024-03-26 06:14:32.992095074 +0000 UTC m=+0.018669997 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:14:33.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 podman[36654]: 2024-03-26 06:14:33.160847661 +0000 UTC m=+0.187422585 container create db92c4504829203fa82cf49ed105495d5c25821bfd0cf1126775e0c66a29133c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-26T06:14:33.807 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 podman[36654]: 2024-03-26 06:14:33.49028722 +0000 UTC m=+0.516862148 container init db92c4504829203fa82cf49ed105495d5c25821bfd0cf1126775e0c66a29133c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:14:33.807 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 podman[36654]: 2024-03-26 06:14:33.49427688 +0000 UTC m=+0.520851807 container start db92c4504829203fa82cf49ed105495d5c25821bfd0cf1126775e0c66a29133c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-c, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: ceph version 19.0.0-1964-g7e98f5f6 (7e98f5f6b535defc7fa715b456904ffc0500aa9f) squid (dev), process ceph-mon, pid 2 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: pidfile_write: ignore empty --pid-file 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: load: jerasure load: lrc 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: RocksDB version: 7.9.2 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Git sha 0 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Compile date 2024-03-23 10:35:25 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: DB SUMMARY 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: DB Session ID: VA5LQQ02U02ESWIG0INX 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: CURRENT file: CURRENT 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-26T06:14:33.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.error_if_exists: 0 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.create_if_missing: 0 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.paranoid_checks: 1 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.env: 0x558fc1a4eb20 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.info_log: 0x558fc30b98e0 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.statistics: (nil) 2024-03-26T06:14:33.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.use_fsync: 0 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_log_file_size: 0 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.allow_fallocate: 1 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.use_direct_reads: 0 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.db_log_dir: 2024-03-26T06:14:33.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.wal_dir: 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.write_buffer_manager: 0x558fc308d400 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T06:14:33.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T06:14:33.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T06:14:33.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T06:14:33.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T06:14:33.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.unordered_write: 0 2024-03-26T06:14:33.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T06:14:33.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T06:14:33.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T06:14:33.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T06:14:33.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.row_cache: None 2024-03-26T06:14:33.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.wal_filter: None 2024-03-26T06:14:33.814 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T06:14:33.814 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T06:14:33.814 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.two_write_queues: 0 2024-03-26T06:14:33.814 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T06:14:33.814 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.wal_compression: 0 2024-03-26T06:14:33.814 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.atomic_flush: 0 2024-03-26T06:14:33.814 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T06:14:33.814 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T06:14:33.814 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.log_readahead_size: 0 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_background_jobs: 2 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_background_compactions: -1 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_subcompactions: 1 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T06:14:33.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_open_files: -1 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_background_flushes: -1 2024-03-26T06:14:33.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Compression algorithms supported: 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: kZSTD supported: 0 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: kXpressCompression supported: 0 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: kZlibCompression supported: 1 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: kSnappyCompression supported: 1 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: kLZ4Compression supported: 1 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: kBZip2Compression supported: 0 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T06:14:33.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.merge_operator: 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_filter: None 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558fc30b99e0) 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout: cache_index_and_filter_blocks: 1 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout: pin_top_level_index_and_filter: 1 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_type: 0 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout: data_block_index_type: 0 2024-03-26T06:14:33.818 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_shortening: 1 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: checksum: 4 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: no_block_cache: 0 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache: 0x558fc30bf090 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_name: BinnedLRUCache 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_options: 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: capacity : 536870912 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: num_shard_bits : 4 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: strict_capacity_limit : 0 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: high_pri_pool_ratio: 0.000 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_compressed: (nil) 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: persistent_cache: (nil) 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_size: 4096 2024-03-26T06:14:33.819 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_size_deviation: 10 2024-03-26T06:14:33.820 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_restart_interval: 16 2024-03-26T06:14:33.820 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_block_restart_interval: 1 2024-03-26T06:14:33.820 INFO:journalctl@ceph.mon.c.smithi149.stdout: metadata_block_size: 4096 2024-03-26T06:14:33.820 INFO:journalctl@ceph.mon.c.smithi149.stdout: partition_filters: 0 2024-03-26T06:14:33.820 INFO:journalctl@ceph.mon.c.smithi149.stdout: use_delta_encoding: 1 2024-03-26T06:14:33.820 INFO:journalctl@ceph.mon.c.smithi149.stdout: filter_policy: bloomfilter 2024-03-26T06:14:33.820 INFO:journalctl@ceph.mon.c.smithi149.stdout: whole_key_filtering: 1 2024-03-26T06:14:33.820 INFO:journalctl@ceph.mon.c.smithi149.stdout: verify_compression: 0 2024-03-26T06:14:33.820 INFO:journalctl@ceph.mon.c.smithi149.stdout: read_amp_bytes_per_bit: 0 2024-03-26T06:14:33.820 INFO:journalctl@ceph.mon.c.smithi149.stdout: format_version: 5 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout: enable_index_compression: 1 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_align: 0 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout: max_auto_readahead_size: 262144 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout: prepopulate_block_cache: 0 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout: initial_auto_readahead_size: 8192 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compression: NoCompression 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.num_levels: 7 2024-03-26T06:14:33.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T06:14:33.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T06:14:33.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T06:14:33.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T06:14:33.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T06:14:33.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T06:14:33.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T06:14:33.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T06:14:33.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T06:14:33.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T06:14:33.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T06:14:33.822 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T06:14:33.823 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T06:14:33.823 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T06:14:33.823 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T06:14:33.823 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T06:14:33.823 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T06:14:33.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T06:14:33.825 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.table_properties_collectors: 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.inplace_update_support: 0 2024-03-26T06:14:33.826 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.bloom_locality: 0 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.max_successive_merges: 0 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.ttl: 2592000 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.enable_blob_files: false 2024-03-26T06:14:33.827 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.min_blob_size: 0 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c090b7ba-0503-4bfa-9883-3a3b3c98fb07 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711433673524544, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-26T06:14:33.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711433673524973, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711433673, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c090b7ba-0503-4bfa-9883-3a3b3c98fb07", "db_session_id": "VA5LQQ02U02ESWIG0INX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711433673525052, "job": 1, "event": "recovery_finished"} 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558fc31b2000 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: DB pointer 0x558fc319c000 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** DB Stats ** 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T06:14:33.829 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** Compaction Stats [default] ** 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** Compaction Stats [default] ** 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T06:14:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: Block cache BinnedLRUCache@0x558fc30bf090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-26T06:14:33.831 INFO:journalctl@ceph.mon.c.smithi149.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: using public_addr v2:172.21.15.149:0/0 -> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: starting mon.c rank -1 at public addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] at bind addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(???) e0 preinit fsid a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).mds e1 new map 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout: e1 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout: legacy client fscid: -1 2024-03-26T06:14:33.832 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout: No filesystems configured 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:33.833 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Updating smithi066:/etc/ceph/ceph.conf 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:33.834 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Deploying cephadm binary to smithi149 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Updating smithi066:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Updating smithi066:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Added host smithi149 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:33.835 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1597416697' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1597416697' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Updating smithi149:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:14:33.836 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Updating smithi149:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi066:172.21.15.66=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi066:172.21.15.66=b;smithi149:172.21.15.149=c;count:3 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: Deploying daemon mon.c on smithi149 2024-03-26T06:14:33.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:33.838 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:33.838 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 ceph-mon[36681]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-26T06:14:33.838 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 bash[36654]: db92c4504829203fa82cf49ed105495d5c25821bfd0cf1126775e0c66a29133c 2024-03-26T06:14:33.838 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:33 smithi149 systemd[1]: Started Ceph mon.c for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:14:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:14:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: mon.a calling monitor election 2024-03-26T06:14:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: mon.c calling monitor election 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: monmap e2: 2 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: fsmap 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: mgrmap e13: x(active, since 51s) 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: overall HEALTH_OK 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:38.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:38.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:14:38.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: mon.a calling monitor election 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: mon.c calling monitor election 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: monmap e2: 2 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: fsmap 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T06:14:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: mgrmap e13: x(active, since 51s) 2024-03-26T06:14:38.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: overall HEALTH_OK 2024-03-26T06:14:38.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:38.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:39.693 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.692+0000 7f622ed44700 1 -- 172.21.15.149:0/841628958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62180062a0 msgr2=0x7f6218008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:39.693 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.692+0000 7f622ed44700 1 --2- 172.21.15.149:0/841628958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62180062a0 0x7f6218008710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6228101780 tx=0x7f621c00fe90 comp rx=0 tx=0).stop 2024-03-26T06:14:39.693 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.692+0000 7f622ed44700 1 -- 172.21.15.149:0/841628958 shutdown_connections 2024-03-26T06:14:39.693 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.692+0000 7f622ed44700 1 --2- 172.21.15.149:0/841628958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62180062a0 0x7f6218008710 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.692+0000 7f622ed44700 1 -- 172.21.15.149:0/841628958 >> 172.21.15.149:0/841628958 conn(0x7f62280fc1d0 msgr2=0x7f62280fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.692+0000 7f622ed44700 1 -- 172.21.15.149:0/841628958 shutdown_connections 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.692+0000 7f622ed44700 1 -- 172.21.15.149:0/841628958 wait complete. 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622ed44700 1 Processor -- start 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622ed44700 1 -- start start 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622ed44700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62180062a0 0x7f6228109bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622ed44700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622810b810 0x7f622810a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622ed44700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6228108740 con 0x7f622810b810 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622ed44700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62281085c0 con 0x7f62180062a0 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622cae0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62180062a0 0x7f6228109bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622cae0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62180062a0 0x7f6228109bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:39496/0 (socket says 172.21.15.149:39496) 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622cae0700 1 -- 172.21.15.149:0/3102315345 learned_addr learned my addr 172.21.15.149:0/3102315345 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-26T06:14:39.694 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622cae0700 1 -- 172.21.15.149:0/3102315345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62180062a0 msgr2=0x7f6228109bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-26T06:14:39.695 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622cae0700 1 -- 172.21.15.149:0/3102315345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62180062a0 msgr2=0x7f6228109bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T06:14:39.695 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622cae0700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62180062a0 0x7f6228109bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T06:14:39.695 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f622cae0700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62180062a0 0x7f6228109bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T06:14:39.695 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.693+0000 7f6227fff700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622810b810 0x7f622810a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:39.695 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.694+0000 7f6227fff700 1 -- 172.21.15.149:0/3102315345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62180062a0 msgr2=0x7f6228109bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T06:14:39.695 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.694+0000 7f6227fff700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62180062a0 0x7f6228109bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:39.695 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.694+0000 7f6227fff700 1 -- 172.21.15.149:0/3102315345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622810a650 con 0x7f622810b810 2024-03-26T06:14:39.696 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.694+0000 7f6227fff700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622810b810 0x7f622810a110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f621400b500 tx=0x7f621400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:39.696 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.694+0000 7f6225ffb700 1 -- 172.21.15.149:0/3102315345 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6214011630 con 0x7f622810b810 2024-03-26T06:14:39.696 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.694+0000 7f6225ffb700 1 -- 172.21.15.149:0/3102315345 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6214011cb0 con 0x7f622810b810 2024-03-26T06:14:39.697 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.694+0000 7f622ed44700 1 -- 172.21.15.149:0/3102315345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622810a940 con 0x7f622810b810 2024-03-26T06:14:39.697 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.694+0000 7f6225ffb700 1 -- 172.21.15.149:0/3102315345 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6214010db0 con 0x7f622810b810 2024-03-26T06:14:39.697 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.695+0000 7f622ed44700 1 -- 172.21.15.149:0/3102315345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62281a8080 con 0x7f622810b810 2024-03-26T06:14:39.697 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.695+0000 7f6225ffb700 1 -- 172.21.15.149:0/3102315345 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f6214018020 con 0x7f622810b810 2024-03-26T06:14:39.697 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.695+0000 7f6225ffb700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f620c038d50 0x7f620c03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:39.697 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.695+0000 7f6225ffb700 1 -- 172.21.15.149:0/3102315345 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f621404b380 con 0x7f622810b810 2024-03-26T06:14:39.697 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.695+0000 7f622ed44700 1 -- 172.21.15.149:0/3102315345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6228101e00 con 0x7f622810b810 2024-03-26T06:14:39.697 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.696+0000 7f622cae0700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f620c038d50 0x7f620c03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:39.697 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.696+0000 7f622cae0700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f620c038d50 0x7f620c03b210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f621c005b40 tx=0x7f621c005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:39.700 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.701+0000 7f6225ffb700 1 -- 172.21.15.149:0/3102315345 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f621401f070 con 0x7f622810b810 2024-03-26T06:14:39.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:39.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:39.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:39.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:39 smithi149 ceph-mon[36681]: Deploying daemon mon.b on smithi066 2024-03-26T06:14:39.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:39.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:39.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:39.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:39.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:39 smithi037 ceph-mon[31974]: Deploying daemon mon.b on smithi066 2024-03-26T06:14:39.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:39.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.976+0000 7f622ed44700 1 -- 172.21.15.149:0/3102315345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f62281a8370 con 0x7f622810b810 2024-03-26T06:14:39.976 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.977+0000 7f6225ffb700 1 -- 172.21.15.149:0/3102315345 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f621400e4c0 con 0x7f622810b810 2024-03-26T06:14:39.976 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:14:39.976 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":2,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","modified":"2024-03-26T06:14:33.547445Z","created":"2024-03-26T06:12:39.866886Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T06:14:39.978 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.979+0000 7f622ed44700 1 -- 172.21.15.149:0/3102315345 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f620c038d50 msgr2=0x7f620c03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:39.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.979+0000 7f622ed44700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f620c038d50 0x7f620c03b210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f621c005b40 tx=0x7f621c005a90 comp rx=0 tx=0).stop 2024-03-26T06:14:39.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.979+0000 7f622ed44700 1 -- 172.21.15.149:0/3102315345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622810b810 msgr2=0x7f622810a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:39.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.979+0000 7f622ed44700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622810b810 0x7f622810a110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f621400b500 tx=0x7f621400b8c0 comp rx=0 tx=0).stop 2024-03-26T06:14:39.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.979+0000 7f622ed44700 1 -- 172.21.15.149:0/3102315345 shutdown_connections 2024-03-26T06:14:39.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.979+0000 7f622ed44700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f620c038d50 0x7f620c03b210 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:39.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.979+0000 7f622ed44700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62180062a0 0x7f6228109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:39.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.979+0000 7f622ed44700 1 --2- 172.21.15.149:0/3102315345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622810b810 0x7f622810a110 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:39.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.979+0000 7f622ed44700 1 -- 172.21.15.149:0/3102315345 >> 172.21.15.149:0/3102315345 conn(0x7f62280fc1d0 msgr2=0x7f622806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:39.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.979+0000 7f622ed44700 1 -- 172.21.15.149:0/3102315345 shutdown_connections 2024-03-26T06:14:39.979 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:39.979+0000 7f622ed44700 1 -- 172.21.15.149:0/3102315345 wait complete. 2024-03-26T06:14:39.979 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 2 2024-03-26T06:14:40.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:40 smithi149 ceph-mon[36681]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:40.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:40 smithi149 ceph-mon[36681]: from='client.? 172.21.15.149:0/3102315345' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T06:14:40.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:40 smithi037 ceph-mon[31974]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:40.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:40 smithi037 ceph-mon[31974]: from='client.? 172.21.15.149:0/3102315345' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T06:14:41.437 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T06:14:41.438 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T06:14:41.685 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:14:42.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:42 smithi149 ceph-mon[36681]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:42.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:42 smithi037 ceph-mon[31974]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:48.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:48.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:14:48.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: mon.a calling monitor election 2024-03-26T06:14:48.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: mon.c calling monitor election 2024-03-26T06:14:48.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:48.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: monmap e3: 3 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],b=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: fsmap 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: mgrmap e13: x(active, since 61s) 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-26T06:14:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: mon.b (rank 2) addr [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] is down (out of quorum) 2024-03-26T06:14:48.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:48.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:48.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:48.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:48.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:48.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:14:48.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: mon.a calling monitor election 2024-03-26T06:14:48.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: mon.c calling monitor election 2024-03-26T06:14:48.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:14:48.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:48.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:48.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T06:14:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: monmap e3: 3 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],b=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T06:14:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: fsmap 2024-03-26T06:14:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T06:14:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: mgrmap e13: x(active, since 61s) 2024-03-26T06:14:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-26T06:14:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-26T06:14:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-26T06:14:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: mon.b (rank 2) addr [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] is down (out of quorum) 2024-03-26T06:14:48.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:48.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:48.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:48.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:49.145 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.144+0000 7f0a806d6700 1 -- 172.21.15.149:0/1546516269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a640062a0 msgr2=0x7f0a64008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:49.145 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.144+0000 7f0a806d6700 1 --2- 172.21.15.149:0/1546516269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a640062a0 0x7f0a64008710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0a78102980 tx=0x7f0a6800fe90 comp rx=0 tx=0).stop 2024-03-26T06:14:49.146 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.145+0000 7f0a806d6700 1 -- 172.21.15.149:0/1546516269 shutdown_connections 2024-03-26T06:14:49.146 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.145+0000 7f0a806d6700 1 --2- 172.21.15.149:0/1546516269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a640062a0 0x7f0a64008710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:49.146 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.145+0000 7f0a806d6700 1 -- 172.21.15.149:0/1546516269 >> 172.21.15.149:0/1546516269 conn(0x7f0a780fd3b0 msgr2=0x7f0a780ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:49.146 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.145+0000 7f0a806d6700 1 -- 172.21.15.149:0/1546516269 shutdown_connections 2024-03-26T06:14:49.146 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.145+0000 7f0a806d6700 1 -- 172.21.15.149:0/1546516269 wait complete. 2024-03-26T06:14:49.146 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:48 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:49.146 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:48 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:49.149 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.145+0000 7f0a806d6700 1 Processor -- start 2024-03-26T06:14:49.149 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.145+0000 7f0a806d6700 1 -- start start 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a806d6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a640062a0 0x7f0a780740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a806d6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a78074620 0x7f0a78074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a806d6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a781a37f0 0x7f0a781a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a806d6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a781098f0 con 0x7f0a78074620 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a806d6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a78109770 con 0x7f0a781a37f0 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a806d6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a78109a70 con 0x7f0a640062a0 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a7e472700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a640062a0 0x7f0a780740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a7e472700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a640062a0 0x7f0a780740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:44910/0 (socket says 172.21.15.149:44910) 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a7e472700 1 -- 172.21.15.149:0/2446265562 learned_addr learned my addr 172.21.15.149:0/2446265562 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a7dc71700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a78074620 0x7f0a78074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a7e472700 1 -- 172.21.15.149:0/2446265562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a781a37f0 msgr2=0x7f0a781a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:49.150 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a7e472700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a781a37f0 0x7f0a781a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a7e472700 1 -- 172.21.15.149:0/2446265562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a78074620 msgr2=0x7f0a78074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a7e472700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a78074620 0x7f0a78074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a7e472700 1 -- 172.21.15.149:0/2446265562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a781a6310 con 0x7f0a640062a0 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a7dc71700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a78074620 0x7f0a78074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a7e472700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a640062a0 0x7f0a780740e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a68010010 tx=0x7f0a68012d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.146+0000 7f0a6f7fe700 1 -- 172.21.15.149:0/2446265562 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68011d20 con 0x7f0a640062a0 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.147+0000 7f0a6f7fe700 1 -- 172.21.15.149:0/2446265562 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a6801d410 con 0x7f0a640062a0 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.147+0000 7f0a806d6700 1 -- 172.21.15.149:0/2446265562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a781a65a0 con 0x7f0a640062a0 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.147+0000 7f0a6f7fe700 1 -- 172.21.15.149:0/2446265562 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a680038e0 con 0x7f0a640062a0 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.147+0000 7f0a806d6700 1 -- 172.21.15.149:0/2446265562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a781a6a80 con 0x7f0a640062a0 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.147+0000 7f0a6f7fe700 1 -- 172.21.15.149:0/2446265562 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f0a68003a80 con 0x7f0a640062a0 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.148+0000 7f0a6f7fe700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a5c039450 0x7f0a5c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:49.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.148+0000 7f0a6f7fe700 1 -- 172.21.15.149:0/2446265562 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0a6804ff90 con 0x7f0a640062a0 2024-03-26T06:14:49.152 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.148+0000 7f0a806d6700 1 -- 172.21.15.149:0/2446265562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a78063410 con 0x7f0a640062a0 2024-03-26T06:14:49.152 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.148+0000 7f0a7dc71700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a5c039450 0x7f0a5c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:49.152 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.148+0000 7f0a7dc71700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a5c039450 0x7f0a5c03b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009910 tx=0x7f0a74008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:49.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.153+0000 7f0a6f7fe700 1 -- 172.21.15.149:0/2446265562 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0a6801d8e0 con 0x7f0a640062a0 2024-03-26T06:14:49.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:48 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:49.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:48 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:49.437 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.436+0000 7f0a806d6700 1 -- 172.21.15.149:0/2446265562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0a7802d830 con 0x7f0a640062a0 2024-03-26T06:14:49.437 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.437+0000 7f0a6f7fe700 1 -- 172.21.15.149:0/2446265562 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1279 (secure 0 0 0) 0x7f0a6803c020 con 0x7f0a640062a0 2024-03-26T06:14:49.438 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-26T06:14:49.438 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":3,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","modified":"2024-03-26T06:14:42.819404Z","created":"2024-03-26T06:12:39.866886Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T06:14:49.439 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 -- 172.21.15.149:0/2446265562 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a5c039450 msgr2=0x7f0a5c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:49.439 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a5c039450 0x7f0a5c03b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009910 tx=0x7f0a74008040 comp rx=0 tx=0).stop 2024-03-26T06:14:49.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 -- 172.21.15.149:0/2446265562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a640062a0 msgr2=0x7f0a780740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:49.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a640062a0 0x7f0a780740e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a68010010 tx=0x7f0a68012d30 comp rx=0 tx=0).stop 2024-03-26T06:14:49.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 -- 172.21.15.149:0/2446265562 shutdown_connections 2024-03-26T06:14:49.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a5c039450 0x7f0a5c03b910 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:49.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a781a37f0 0x7f0a781a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:49.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a640062a0 0x7f0a780740e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:49.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 --2- 172.21.15.149:0/2446265562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a78074620 0x7f0a78074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:49.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 -- 172.21.15.149:0/2446265562 >> 172.21.15.149:0/2446265562 conn(0x7f0a780fd3b0 msgr2=0x7f0a78106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:49.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 -- 172.21.15.149:0/2446265562 shutdown_connections 2024-03-26T06:14:49.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:49.439+0000 7f0a806d6700 1 -- 172.21.15.149:0/2446265562 wait complete. 2024-03-26T06:14:49.441 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 3 2024-03-26T06:14:50.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:50.109 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-26T06:14:50.109 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-26T06:14:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:14:50.832 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: mon.b calling monitor election 2024-03-26T06:14:50.832 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: mon.b calling monitor election 2024-03-26T06:14:50.832 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: mon.c calling monitor election 2024-03-26T06:14:50.832 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: mon.a calling monitor election 2024-03-26T06:14:50.832 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T06:14:50.833 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: monmap e3: 3 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],b=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T06:14:50.833 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: fsmap 2024-03-26T06:14:50.833 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T06:14:50.833 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: mgrmap e13: x(active, since 63s) 2024-03-26T06:14:50.834 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-26T06:14:50.834 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: Cluster is now healthy 2024-03-26T06:14:50.834 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: overall HEALTH_OK 2024-03-26T06:14:50.834 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.834 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.834 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:50.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T06:14:50.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:50.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: mon.b calling monitor election 2024-03-26T06:14:50.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: mon.b calling monitor election 2024-03-26T06:14:50.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: mon.c calling monitor election 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: mon.a calling monitor election 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: monmap e3: 3 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],b=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: fsmap 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: mgrmap e13: x(active, since 63s) 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: Cluster is now healthy 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: overall HEALTH_OK 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:50.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T06:14:50.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:50.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: mon.b calling monitor election 2024-03-26T06:14:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: mon.b calling monitor election 2024-03-26T06:14:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: mon.c calling monitor election 2024-03-26T06:14:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: mon.a calling monitor election 2024-03-26T06:14:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T06:14:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: monmap e3: 3 mons at {a=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],b=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T06:14:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: fsmap 2024-03-26T06:14:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T06:14:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: mgrmap e13: x(active, since 63s) 2024-03-26T06:14:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-26T06:14:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: Cluster is now healthy 2024-03-26T06:14:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: overall HEALTH_OK 2024-03-26T06:14:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:50.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:50.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T06:14:50.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:51.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:51 smithi037 ceph-mon[31974]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T06:14:51.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:51 smithi037 ceph-mon[31974]: Reconfiguring daemon mon.a on smithi037 2024-03-26T06:14:51.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:51 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:51.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:51 smithi066 ceph-mon[36504]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T06:14:51.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:51 smithi066 ceph-mon[36504]: Reconfiguring daemon mon.a on smithi037 2024-03-26T06:14:51.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:51 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:51.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:51 smithi149 ceph-mon[36681]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T06:14:51.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:51 smithi149 ceph-mon[36681]: Reconfiguring daemon mon.a on smithi037 2024-03-26T06:14:51.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:51 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:14:52.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:52 smithi037 ceph-mon[31974]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:52.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:52.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:52 smithi066 ceph-mon[36504]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:52.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:52.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:52 smithi149 ceph-mon[36681]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:52.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:53.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.119+0000 7f2f9f305700 1 -- 172.21.15.37:0/3926597874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f98106060 msgr2=0x7f2f9810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:53.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.119+0000 7f2f9f305700 1 --2- 172.21.15.37:0/3926597874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f98106060 0x7f2f9810dc30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000b0d0 tx=0x7f2f9000b3e0 comp rx=0 tx=0).stop 2024-03-26T06:14:53.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.119+0000 7f2f9f305700 1 -- 172.21.15.37:0/3926597874 shutdown_connections 2024-03-26T06:14:53.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.119+0000 7f2f9f305700 1 --2- 172.21.15.37:0/3926597874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f98106060 0x7f2f9810dc30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:53.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.119+0000 7f2f9f305700 1 --2- 172.21.15.37:0/3926597874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f98101740 0x7f2f98101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:53.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.119+0000 7f2f9f305700 1 --2- 172.21.15.37:0/3926597874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f981020f0 0x7f2f98105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:53.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.119+0000 7f2f9f305700 1 -- 172.21.15.37:0/3926597874 >> 172.21.15.37:0/3926597874 conn(0x7f2f980772e0 msgr2=0x7f2f980fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:53.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.120+0000 7f2f9f305700 1 -- 172.21.15.37:0/3926597874 shutdown_connections 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.120+0000 7f2f9f305700 1 -- 172.21.15.37:0/3926597874 wait complete. 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.120+0000 7f2f9f305700 1 Processor -- start 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.120+0000 7f2f9f305700 1 -- start start 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.120+0000 7f2f9f305700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f98101740 0x7f2f9819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.120+0000 7f2f9f305700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f981020f0 0x7f2f9819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.120+0000 7f2f9f305700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f98106060 0x7f2f9819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.120+0000 7f2f9f305700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f98111070 con 0x7f2f981020f0 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.120+0000 7f2f9f305700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f98110ef0 con 0x7f2f98101740 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.120+0000 7f2f9f305700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f981111f0 con 0x7f2f98106060 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.120+0000 7f2f9c8a0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f981020f0 0x7f2f9819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9d0a1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f98101740 0x7f2f9819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9c8a0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f981020f0 0x7f2f9819ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58134/0 (socket says 172.21.15.37:58134) 2024-03-26T06:14:53.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9c8a0700 1 -- 172.21.15.37:0/960540490 learned_addr learned my addr 172.21.15.37:0/960540490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9d8a2700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f98106060 0x7f2f9819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9c8a0700 1 -- 172.21.15.37:0/960540490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f98106060 msgr2=0x7f2f9819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9c8a0700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f98106060 0x7f2f9819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9c8a0700 1 -- 172.21.15.37:0/960540490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f98101740 msgr2=0x7f2f9819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9c8a0700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f98101740 0x7f2f9819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9c8a0700 1 -- 172.21.15.37:0/960540490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9819f740 con 0x7f2f981020f0 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9d0a1700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f98101740 0x7f2f9819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9c8a0700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f981020f0 0x7f2f9819ac70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400e970 tx=0x7f2f9400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f8a7fc700 1 -- 172.21.15.37:0/960540490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9400cbd0 con 0x7f2f981020f0 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f8a7fc700 1 -- 172.21.15.37:0/960540490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f9400cd70 con 0x7f2f981020f0 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9f305700 1 -- 172.21.15.37:0/960540490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9819fa30 con 0x7f2f981020f0 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f8a7fc700 1 -- 172.21.15.37:0/960540490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f940056b0 con 0x7f2f981020f0 2024-03-26T06:14:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.121+0000 7f2f9f305700 1 -- 172.21.15.37:0/960540490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f981a72b0 con 0x7f2f981020f0 2024-03-26T06:14:53.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.122+0000 7f2f8a7fc700 1 -- 172.21.15.37:0/960540490 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f2f94004830 con 0x7f2f981020f0 2024-03-26T06:14:53.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.123+0000 7f2f9f305700 1 -- 172.21.15.37:0/960540490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f981024f0 con 0x7f2f981020f0 2024-03-26T06:14:53.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.123+0000 7f2f8a7fc700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2f84039200 0x7f2f8403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:53.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.123+0000 7f2f8a7fc700 1 -- 172.21.15.37:0/960540490 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2f9404c4b0 con 0x7f2f981020f0 2024-03-26T06:14:53.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.123+0000 7f2f9d0a1700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2f84039200 0x7f2f8403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:53.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.123+0000 7f2f9d0a1700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2f84039200 0x7f2f8403b6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c005950 tx=0x7f2f8c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:53.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.128+0000 7f2f8a7fc700 1 -- 172.21.15.37:0/960540490 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2f94004b20 con 0x7f2f981020f0 2024-03-26T06:14:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.333+0000 7f2f9f305700 1 -- 172.21.15.37:0/960540490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f2f9802d830 con 0x7f2f981020f0 2024-03-26T06:14:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.333+0000 7f2f8a7fc700 1 -- 172.21.15.37:0/960540490 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f2f94016db0 con 0x7f2f981020f0 2024-03-26T06:14:53.333 INFO:teuthology.orchestra.run.smithi037.stdout:# minimal ceph.conf for a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:14:53.334 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-03-26T06:14:53.334 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:14:53.334 INFO:teuthology.orchestra.run.smithi037.stdout: mon_host = [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-03-26T06:14:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 -- 172.21.15.37:0/960540490 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2f84039200 msgr2=0x7f2f8403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2f84039200 0x7f2f8403b6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c005950 tx=0x7f2f8c00a300 comp rx=0 tx=0).stop 2024-03-26T06:14:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 -- 172.21.15.37:0/960540490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f981020f0 msgr2=0x7f2f9819ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:53.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f981020f0 0x7f2f9819ac70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400e970 tx=0x7f2f9400ec80 comp rx=0 tx=0).stop 2024-03-26T06:14:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 -- 172.21.15.37:0/960540490 shutdown_connections 2024-03-26T06:14:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2f84039200 0x7f2f8403b6c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f98106060 0x7f2f9819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f98101740 0x7f2f9819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 --2- 172.21.15.37:0/960540490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f981020f0 0x7f2f9819ac70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 -- 172.21.15.37:0/960540490 >> 172.21.15.37:0/960540490 conn(0x7f2f980772e0 msgr2=0x7f2f98105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 -- 172.21.15.37:0/960540490 shutdown_connections 2024-03-26T06:14:53.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:14:53.337+0000 7f2f9f305700 1 -- 172.21.15.37:0/960540490 wait complete. 2024-03-26T06:14:53.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:53.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:53.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:53 smithi149 ceph-mon[36681]: Reconfiguring mon.b (monmap changed)... 2024-03-26T06:14:53.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:53.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T06:14:53.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:53.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:53 smithi149 ceph-mon[36681]: Reconfiguring daemon mon.b on smithi066 2024-03-26T06:14:53.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:53 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/960540490' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:53.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:53.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:53.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:53 smithi037 ceph-mon[31974]: Reconfiguring mon.b (monmap changed)... 2024-03-26T06:14:53.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:53.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T06:14:53.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:53.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:53 smithi037 ceph-mon[31974]: Reconfiguring daemon mon.b on smithi066 2024-03-26T06:14:53.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:53 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/960540490' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:53.967 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-26T06:14:53.967 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:14:53.967 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T06:14:53.996 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:14:53.996 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:14:54.003 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:54.003 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:54.003 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:53 smithi066 ceph-mon[36504]: Reconfiguring mon.b (monmap changed)... 2024-03-26T06:14:54.003 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:54.003 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T06:14:54.003 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:54.003 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:53 smithi066 ceph-mon[36504]: Reconfiguring daemon mon.b on smithi066 2024-03-26T06:14:54.004 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:53 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/960540490' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:54.063 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-26T06:14:54.063 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T06:14:54.099 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-26T06:14:54.099 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:14:54.171 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-26T06:14:54.172 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T06:14:54.203 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-26T06:14:54.203 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:14:54.272 INFO:tasks.cephadm:Adding mgr.x on smithi037 2024-03-26T06:14:54.272 INFO:tasks.cephadm:Adding mgr.y on smithi066 2024-03-26T06:14:54.272 INFO:tasks.cephadm:Adding mgr.z on smithi149 2024-03-26T06:14:54.272 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch apply mgr '3;smithi037=x;smithi066=y;smithi149=z' 2024-03-26T06:14:54.548 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:14:54.892 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:54 smithi149 ceph-mon[36681]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:54.892 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:54.892 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:54.892 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:54.892 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T06:14:54.892 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:54.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:54 smithi037 ceph-mon[31974]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:54.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:54.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:54.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:54.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T06:14:54.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:55.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:54 smithi066 ceph-mon[36504]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:55.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:55.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:55.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T06:14:55.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T06:14:55.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:55.818 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:55 smithi149 ceph-mon[36681]: Reconfiguring mon.c (monmap changed)... 2024-03-26T06:14:55.818 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:55 smithi149 ceph-mon[36681]: Reconfiguring daemon mon.c on smithi149 2024-03-26T06:14:55.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:55 smithi037 ceph-mon[31974]: Reconfiguring mon.c (monmap changed)... 2024-03-26T06:14:55.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:55 smithi037 ceph-mon[31974]: Reconfiguring daemon mon.c on smithi149 2024-03-26T06:14:56.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:55 smithi066 ceph-mon[36504]: Reconfiguring mon.c (monmap changed)... 2024-03-26T06:14:56.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:55 smithi066 ceph-mon[36504]: Reconfiguring daemon mon.c on smithi149 2024-03-26T06:14:56.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:56 smithi149 ceph-mon[36681]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:56.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:56 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:56.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:56 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:56.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:56 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:56.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:56 smithi037 ceph-mon[31974]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:56.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:56 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:56.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:56 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:56.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:56 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:57.024 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.021+0000 7f6a05722700 1 -- 172.21.15.149:0/4081437341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a00101f10 msgr2=0x7f6a00102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:57.024 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.021+0000 7f6a05722700 1 --2- 172.21.15.149:0/4081437341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a00101f10 0x7f6a00102370 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009990 tx=0x7f69f0009ca0 comp rx=0 tx=0).stop 2024-03-26T06:14:57.024 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.021+0000 7f6a05722700 1 -- 172.21.15.149:0/4081437341 shutdown_connections 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.021+0000 7f6a05722700 1 --2- 172.21.15.149:0/4081437341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a00100d10 0x7f6a00101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.021+0000 7f6a05722700 1 --2- 172.21.15.149:0/4081437341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a001028b0 0x7f6a00109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.021+0000 7f6a05722700 1 --2- 172.21.15.149:0/4081437341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a00101f10 0x7f6a00102370 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.021+0000 7f6a05722700 1 -- 172.21.15.149:0/4081437341 >> 172.21.15.149:0/4081437341 conn(0x7f6a000fc2b0 msgr2=0x7f6a000fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.021+0000 7f6a05722700 1 -- 172.21.15.149:0/4081437341 shutdown_connections 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.021+0000 7f6a05722700 1 -- 172.21.15.149:0/4081437341 wait complete. 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f6a05722700 1 Processor -- start 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f6a05722700 1 -- start start 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f6a05722700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a00100d10 0x7f6a001980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f6a05722700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a00101f10 0x7f6a001985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f6a05722700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a001028b0 0x7f6a0019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:57.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f6a05722700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0010c8d0 con 0x7f6a00100d10 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f6a05722700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0010c750 con 0x7f6a00101f10 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f6a05722700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0010ca50 con 0x7f6a001028b0 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f69ff7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a001028b0 0x7f6a0019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f69ff7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a001028b0 0x7f6a0019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:44938/0 (socket says 172.21.15.149:44938) 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f69ff7fe700 1 -- 172.21.15.149:0/1569949858 learned_addr learned my addr 172.21.15.149:0/1569949858 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f69feffd700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a00100d10 0x7f6a001980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f69fe7fc700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a00101f10 0x7f6a001985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f69ff7fe700 1 -- 172.21.15.149:0/1569949858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a00101f10 msgr2=0x7f6a001985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f69ff7fe700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a00101f10 0x7f6a001985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f69ff7fe700 1 -- 172.21.15.149:0/1569949858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a00100d10 msgr2=0x7f6a001980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.022+0000 7f69ff7fe700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a00100d10 0x7f6a001980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.023+0000 7f69ff7fe700 1 -- 172.21.15.149:0/1569949858 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0019fd00 con 0x7f6a001028b0 2024-03-26T06:14:57.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.023+0000 7f69feffd700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a00100d10 0x7f6a001980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.023+0000 7f69ff7fe700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a001028b0 0x7f6a0019f660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f69f8007f10 tx=0x7f69f800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.023+0000 7f69e7fff700 1 -- 172.21.15.149:0/1569949858 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f800ddd0 con 0x7f6a001028b0 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.023+0000 7f69e7fff700 1 -- 172.21.15.149:0/1569949858 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69f8004020 con 0x7f6a001028b0 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.023+0000 7f6a05722700 1 -- 172.21.15.149:0/1569949858 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0019fff0 con 0x7f6a001028b0 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.023+0000 7f69e7fff700 1 -- 172.21.15.149:0/1569949858 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8015660 con 0x7f6a001028b0 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.023+0000 7f6a05722700 1 -- 172.21.15.149:0/1569949858 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a001a04a0 con 0x7f6a001028b0 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.024+0000 7f69e7fff700 1 -- 172.21.15.149:0/1569949858 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f69f8015800 con 0x7f6a001028b0 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.024+0000 7f69e7fff700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69e8039200 0x7f69e803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.024+0000 7f69e7fff700 1 -- 172.21.15.149:0/1569949858 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f69f804bb40 con 0x7f6a001028b0 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.024+0000 7f6a05722700 1 -- 172.21.15.149:0/1569949858 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a00068aa0 con 0x7f6a001028b0 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.024+0000 7f69feffd700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69e8039200 0x7f69e803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:14:57.027 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.025+0000 7f69feffd700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69e8039200 0x7f69e803b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6a00101d70 tx=0x7f69f400b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:14:57.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.034+0000 7f69e7fff700 1 -- 172.21.15.149:0/1569949858 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f69f8010070 con 0x7f6a001028b0 2024-03-26T06:14:57.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:56 smithi066 ceph-mon[36504]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:57.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:56 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:57.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:56 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:57.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:56 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:57.246 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.246+0000 7f6a05722700 1 -- 172.21.15.149:0/1569949858 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi066=y;smithi149=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a0006a1a0 con 0x7f69e8039200 2024-03-26T06:14:57.254 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.254+0000 7f69e7fff700 1 -- 172.21.15.149:0/1569949858 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6a0006a1a0 con 0x7f69e8039200 2024-03-26T06:14:57.254 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mgr update... 2024-03-26T06:14:57.256 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.256+0000 7f6a05722700 1 -- 172.21.15.149:0/1569949858 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69e8039200 msgr2=0x7f69e803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:57.256 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.256+0000 7f6a05722700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69e8039200 0x7f69e803b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6a00101d70 tx=0x7f69f400b670 comp rx=0 tx=0).stop 2024-03-26T06:14:57.256 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.256+0000 7f6a05722700 1 -- 172.21.15.149:0/1569949858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a001028b0 msgr2=0x7f6a0019f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:14:57.256 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.256+0000 7f6a05722700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a001028b0 0x7f6a0019f660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f69f8007f10 tx=0x7f69f800d3b0 comp rx=0 tx=0).stop 2024-03-26T06:14:57.257 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.256+0000 7f6a05722700 1 -- 172.21.15.149:0/1569949858 shutdown_connections 2024-03-26T06:14:57.257 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.256+0000 7f6a05722700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a00100d10 0x7f6a001980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:57.257 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.256+0000 7f6a05722700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69e8039200 0x7f69e803b6c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:57.257 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.256+0000 7f6a05722700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a001028b0 0x7f6a0019f660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:57.257 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.256+0000 7f6a05722700 1 --2- 172.21.15.149:0/1569949858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a00101f10 0x7f6a001985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:14:57.257 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.256+0000 7f6a05722700 1 -- 172.21.15.149:0/1569949858 >> 172.21.15.149:0/1569949858 conn(0x7f6a000fc2b0 msgr2=0x7f6a00069a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:14:57.257 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.257+0000 7f6a05722700 1 -- 172.21.15.149:0/1569949858 shutdown_connections 2024-03-26T06:14:57.257 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:14:57.257+0000 7f6a05722700 1 -- 172.21.15.149:0/1569949858 wait complete. 2024-03-26T06:14:57.855 DEBUG:teuthology.orchestra.run.smithi066:mgr.y> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.y.service 2024-03-26T06:14:57.858 DEBUG:teuthology.orchestra.run.smithi149:mgr.z> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.z.service 2024-03-26T06:14:57.860 INFO:tasks.cephadm:Deploying OSDs... 2024-03-26T06:14:57.860 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:14:57.860 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T06:14:57.879 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T06:14:57.880 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-03-26T06:14:57.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T06:14:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T06:14:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T06:14:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:57.937 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T06:14:57.937 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:14:57.937 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-26T06:14:57.937 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:14:57.937 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:14:57.938 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-26 06:14:01.167084169 +0000 2024-03-26T06:14:57.938 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-26 06:07:55.398325307 +0000 2024-03-26T06:14:57.938 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-26 06:07:55.398325307 +0000 2024-03-26T06:14:57.938 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-26 06:07:55.398325307 +0000 2024-03-26T06:14:57.938 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T06:14:57.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:57.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:57.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T06:14:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T06:14:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T06:14:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-26T06:14:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-26T06:14:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000226496 s, 2.3 MB/s 2024-03-26T06:14:57.978 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T06:14:58.035 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-03-26T06:14:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T06:14:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:14:58.093 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-26T06:14:58.093 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:14:58.093 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:14:58.093 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-26 06:14:01.201083696 +0000 2024-03-26T06:14:58.093 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-26 06:07:55.745320587 +0000 2024-03-26T06:14:58.093 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-26 06:07:55.745320587 +0000 2024-03-26T06:14:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-26 06:07:55.745320587 +0000 2024-03-26T06:14:58.094 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T06:14:58.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:58.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:58.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:58.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:58.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:14:58.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:58.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:14:58.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:14:58.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T06:14:58.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T06:14:58.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T06:14:58.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:14:58.162 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-26T06:14:58.162 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-26T06:14:58.162 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000229567 s, 2.2 MB/s 2024-03-26T06:14:58.164 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T06:14:58.221 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-03-26T06:14:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T06:14:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:14:58.277 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-26T06:14:58.277 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:14:58.278 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:14:58.278 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-26 06:14:01.233083251 +0000 2024-03-26T06:14:58.278 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-26 06:07:56.081316017 +0000 2024-03-26T06:14:58.278 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-26 06:07:56.081316017 +0000 2024-03-26T06:14:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-26 06:07:56.081316017 +0000 2024-03-26T06:14:58.279 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T06:14:58.346 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-26T06:14:58.346 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-26T06:14:58.346 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.00019336 s, 2.6 MB/s 2024-03-26T06:14:58.347 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T06:14:58.405 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-03-26T06:14:58.461 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T06:14:58.461 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:14:58.461 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-26T06:14:58.461 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:14:58.461 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:14:58.461 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-26 06:14:01.264082820 +0000 2024-03-26T06:14:58.461 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-26 06:07:56.428311296 +0000 2024-03-26T06:14:58.461 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-26 06:07:56.428311296 +0000 2024-03-26T06:14:58.461 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-26 06:07:56.428311296 +0000 2024-03-26T06:14:58.461 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T06:14:58.528 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-26T06:14:58.528 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-26T06:14:58.529 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000320039 s, 1.6 MB/s 2024-03-26T06:14:58.530 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T06:14:58.587 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-03-26T06:14:58.587 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T06:14:58.606 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T06:14:58.607 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-03-26T06:14:58.665 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T06:14:58.665 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:14:58.665 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-03-26T06:14:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:14:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:14:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-26 06:14:42.239356827 +0000 2024-03-26T06:14:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-26 06:08:17.592116388 +0000 2024-03-26T06:14:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-26 06:08:17.592116388 +0000 2024-03-26T06:14:58.666 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-26 06:08:17.592116388 +0000 2024-03-26T06:14:58.666 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T06:14:58.740 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-26T06:14:58.740 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-26T06:14:58.740 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000228547 s, 2.2 MB/s 2024-03-26T06:14:58.741 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T06:14:58.799 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-03-26T06:14:58.856 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T06:14:58.856 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:14:58.856 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-26T06:14:58.856 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:14:58.856 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:14:58.856 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-26 06:14:42.239356827 +0000 2024-03-26T06:14:58.856 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-26 06:08:17.938114089 +0000 2024-03-26T06:14:58.856 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-26 06:08:17.938114089 +0000 2024-03-26T06:14:58.857 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-26 06:08:17.938114089 +0000 2024-03-26T06:14:58.857 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T06:14:58.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:58 smithi149 ceph-mon[36681]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:58.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:58 smithi149 ceph-mon[36681]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi066=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:14:58.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:58 smithi149 ceph-mon[36681]: Saving service mgr spec with placement smithi037=x;smithi066=y;smithi149=z;count:3 2024-03-26T06:14:58.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:14:58 smithi149 ceph-mon[36681]: Deploying daemon mgr.z on smithi149 2024-03-26T06:14:58.925 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:58 smithi066 ceph-mon[36504]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:58.925 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:58 smithi066 ceph-mon[36504]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi066=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:14:58.925 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:58 smithi066 ceph-mon[36504]: Saving service mgr spec with placement smithi037=x;smithi066=y;smithi149=z;count:3 2024-03-26T06:14:58.925 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:14:58 smithi066 ceph-mon[36504]: Deploying daemon mgr.z on smithi149 2024-03-26T06:14:58.927 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-26T06:14:58.928 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-26T06:14:58.928 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000207958 s, 2.5 MB/s 2024-03-26T06:14:58.929 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T06:14:58.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:58 smithi037 ceph-mon[31974]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:14:58.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:58 smithi037 ceph-mon[31974]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi066=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:14:58.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:58 smithi037 ceph-mon[31974]: Saving service mgr spec with placement smithi037=x;smithi066=y;smithi149=z;count:3 2024-03-26T06:14:58.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:14:58 smithi037 ceph-mon[31974]: Deploying daemon mgr.z on smithi149 2024-03-26T06:14:58.986 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-03-26T06:14:59.043 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T06:14:59.043 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:14:59.043 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-26T06:14:59.043 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:14:59.043 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:14:59.043 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-26 06:14:42.239356827 +0000 2024-03-26T06:14:59.043 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-26 06:08:18.290111750 +0000 2024-03-26T06:14:59.043 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-26 06:08:18.290111750 +0000 2024-03-26T06:14:59.043 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-26 06:08:18.290111750 +0000 2024-03-26T06:14:59.044 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T06:14:59.113 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-26T06:14:59.113 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-26T06:14:59.113 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000178162 s, 2.9 MB/s 2024-03-26T06:14:59.114 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T06:14:59.172 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-03-26T06:14:59.228 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T06:14:59.228 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:14:59.228 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-03-26T06:14:59.228 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:14:59.228 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:14:59.228 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-03-26 06:14:42.240356820 +0000 2024-03-26T06:14:59.228 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-03-26 06:08:18.625109525 +0000 2024-03-26T06:14:59.228 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-03-26 06:08:18.625109525 +0000 2024-03-26T06:14:59.228 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-03-26 06:08:18.625109525 +0000 2024-03-26T06:14:59.229 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T06:14:59.299 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-03-26T06:14:59.299 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-03-26T06:14:59.299 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000177198 s, 2.9 MB/s 2024-03-26T06:14:59.300 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T06:14:59.326 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:14:59 smithi149 systemd[1]: Starting Ceph mgr.z for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:14:59.356 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-26T06:14:59.356 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T06:14:59.372 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T06:14:59.373 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-03-26T06:14:59.429 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T06:14:59.430 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:14:59.430 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-03-26T06:14:59.430 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:14:59.430 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:14:59.430 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-26 06:14:59.264263523 +0000 2024-03-26T06:14:59.430 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-26 06:08:30.512302696 +0000 2024-03-26T06:14:59.430 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-26 06:08:30.512302696 +0000 2024-03-26T06:14:59.430 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-26 06:08:30.512302696 +0000 2024-03-26T06:14:59.430 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T06:14:59.497 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-26T06:14:59.498 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-26T06:14:59.498 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000253726 s, 2.0 MB/s 2024-03-26T06:14:59.499 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T06:14:59.554 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-03-26T06:14:59.609 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T06:14:59.609 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:14:59.609 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-26T06:14:59.609 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:14:59.610 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:14:59.610 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-26 06:14:59.264263523 +0000 2024-03-26T06:14:59.610 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-26 06:08:30.854295593 +0000 2024-03-26T06:14:59.610 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-26 06:08:30.854295593 +0000 2024-03-26T06:14:59.610 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-26 06:08:30.854295593 +0000 2024-03-26T06:14:59.610 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T06:14:59.658 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:14:59 smithi149 podman[38244]: 2024-03-26 06:14:59.446411591 +0000 UTC m=+0.017446297 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:14:59.659 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:14:59 smithi149 podman[38244]: 2024-03-26 06:14:59.570307774 +0000 UTC m=+0.141342506 container create 4847a6846fae7177027dc828841082d0c23adbeb9d6c98c7eb182aeca8f20df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T06:14:59.675 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-26T06:14:59.675 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-26T06:14:59.675 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000209348 s, 2.4 MB/s 2024-03-26T06:14:59.676 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T06:14:59.779 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-03-26T06:14:59.835 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T06:14:59.835 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:14:59.835 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-26T06:14:59.835 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:14:59.835 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:14:59.835 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-26 06:14:59.264263523 +0000 2024-03-26T06:14:59.835 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-26 06:08:31.213288137 +0000 2024-03-26T06:14:59.836 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-26 06:08:31.213288137 +0000 2024-03-26T06:14:59.836 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-26 06:08:31.213288137 +0000 2024-03-26T06:14:59.836 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T06:14:59.902 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-26T06:14:59.902 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-26T06:14:59.902 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.00019534 s, 2.6 MB/s 2024-03-26T06:14:59.903 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T06:14:59.959 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-03-26T06:15:00.014 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T06:15:00.015 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T06:15:00.015 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-26T06:15:00.015 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:15:00.015 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T06:15:00.015 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-26 06:14:59.264263523 +0000 2024-03-26T06:15:00.015 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-26 06:08:31.560280930 +0000 2024-03-26T06:15:00.015 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-26 06:08:31.560280930 +0000 2024-03-26T06:15:00.015 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-26 06:08:31.560280930 +0000 2024-03-26T06:15:00.016 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T06:15:00.079 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:14:59 smithi149 podman[38244]: 2024-03-26 06:14:59.806163863 +0000 UTC m=+0.377198546 container init 4847a6846fae7177027dc828841082d0c23adbeb9d6c98c7eb182aeca8f20df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-26T06:15:00.079 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:14:59 smithi149 podman[38244]: 2024-03-26 06:14:59.808684958 +0000 UTC m=+0.379719649 container start 4847a6846fae7177027dc828841082d0c23adbeb9d6c98c7eb182aeca8f20df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-26T06:15:00.079 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:14:59 smithi149 ceph-mgr[38361]: -- 172.21.15.149:0/3088629441 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5651a6d0c5a0 con 0x5651a6ce1000 2024-03-26T06:15:00.079 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:14:59 smithi149 bash[38244]: 4847a6846fae7177027dc828841082d0c23adbeb9d6c98c7eb182aeca8f20df3 2024-03-26T06:15:00.079 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:14:59 smithi149 systemd[1]: Started Ceph mgr.z for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:15:00.080 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-26T06:15:00.081 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-26T06:15:00.081 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000207899 s, 2.5 MB/s 2024-03-26T06:15:00.082 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T06:15:00.138 INFO:tasks.cephadm:Deploying osd.0 on smithi037 with /dev/vg_nvme/lv_4... 2024-03-26T06:15:00.138 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T06:15:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:15:00.659 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:00.379+0000 7f28af4fb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T06:15:00.659 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:00.472+0000 7f28af4fb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T06:15:00.792 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:00 smithi037 ceph-mon[31974]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:00.792 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:00.792 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:00.792 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:00.792 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T06:15:00.792 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T06:15:00.793 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T06:15:00.793 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:00.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:00 smithi066 ceph-mon[36504]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:00.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:00.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:00.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:00.831 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T06:15:00.832 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T06:15:00.832 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T06:15:00.832 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:01.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:00 smithi149 ceph-mon[36681]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:01.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:01.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:01.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:01.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T06:15:01.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T06:15:01.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T06:15:01.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:01.408 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:01.112+0000 7f28af4fb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T06:15:01.409 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:01.206+0000 7f28af4fb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T06:15:01.409 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:01.350+0000 7f28af4fb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T06:15:01.733 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:01 smithi066 ceph-mon[36504]: Deploying daemon mgr.y on smithi066 2024-03-26T06:15:01.789 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:01 smithi149 ceph-mon[36681]: Deploying daemon mgr.y on smithi066 2024-03-26T06:15:01.789 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:01.444+0000 7f28af4fb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T06:15:01.789 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:01.526+0000 7f28af4fb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T06:15:01.790 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:01.789+0000 7f28af4fb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T06:15:01.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:01 smithi037 ceph-mon[31974]: Deploying daemon mgr.y on smithi066 2024-03-26T06:15:02.076 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:01.898+0000 7f28af4fb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T06:15:02.077 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:01.984+0000 7f28af4fb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T06:15:02.077 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:02 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:02.076+0000 7f28af4fb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T06:15:02.096 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:01 smithi066 systemd[1]: Starting Ceph mgr.y for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:15:02.409 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:02 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:02.256+0000 7f28af4fb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T06:15:02.413 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:02 smithi066 podman[37914]: 2024-03-26 06:15:02.032481997 +0000 UTC m=+0.018015603 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:15:02.413 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:02 smithi066 podman[37914]: 2024-03-26 06:15:02.245072183 +0000 UTC m=+0.230605819 container create 9354924ed7bbe9e74a5d342d136f37acd8c0f30138c100b0c6fdfdcc6180aa46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:15:02.413 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:02 smithi066 podman[37914]: 2024-03-26 06:15:02.414184184 +0000 UTC m=+0.399717796 container init 9354924ed7bbe9e74a5d342d136f37acd8c0f30138c100b0c6fdfdcc6180aa46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:15:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T06:15:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T06:15:02.463 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-03-26T06:15:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-03-26T06:15:02.463 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110934 s, 945 MB/s 2024-03-26T06:15:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-03-26T06:15:02.668 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:02 smithi066 podman[37914]: 2024-03-26 06:15:02.416881695 +0000 UTC m=+0.402415292 container start 9354924ed7bbe9e74a5d342d136f37acd8c0f30138c100b0c6fdfdcc6180aa46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:15:02.669 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:02 smithi066 ceph-mgr[37933]: -- 172.21.15.66:0/364806274 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b6624a25a0 con 0x55b662477000 2024-03-26T06:15:02.669 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:02 smithi066 bash[37914]: 9354924ed7bbe9e74a5d342d136f37acd8c0f30138c100b0c6fdfdcc6180aa46 2024-03-26T06:15:02.669 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:02 smithi066 systemd[1]: Started Ceph mgr.y for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:15:02.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:02 smithi037 ceph-mon[31974]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:02.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:02 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:02.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:02 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:02.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:02 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:02.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:02 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:03.007 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:02 smithi066 ceph-mon[36504]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:03.007 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:02 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:03.007 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:02 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:03.008 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:02 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:03.008 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:02 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:03.157 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi037:vg_nvme/lv_4 2024-03-26T06:15:03.158 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:02 smithi149 ceph-mon[36681]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:03.158 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:02 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:03.158 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:02 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:03.158 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:02 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:03.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:02 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:03.346 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:03 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:03.007+0000 7fd6382b6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T06:15:03.346 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:03 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:03.105+0000 7fd6382b6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T06:15:03.394 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:15:03.649 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:03.272+0000 7f28af4fb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T06:15:03.650 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:03.411+0000 7f28af4fb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T06:15:03.791 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:03 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:03.804 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:03 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:03.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:03 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:03.910 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:03.769+0000 7f28af4fb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T06:15:03.910 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:03.859+0000 7f28af4fb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T06:15:04.096 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:03 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:03.804+0000 7fd6382b6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T06:15:04.096 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:03 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:03.899+0000 7fd6382b6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T06:15:04.096 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:04.051+0000 7fd6382b6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T06:15:04.408 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:04.065+0000 7f28af4fb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:15:04.409 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:04.249+0000 7f28af4fb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T06:15:04.510 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:04.152+0000 7fd6382b6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T06:15:04.510 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:04.240+0000 7fd6382b6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T06:15:04.510 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:04.511+0000 7fd6382b6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T06:15:04.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:04.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:04.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:04.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T06:15:04.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T06:15:04.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T06:15:04.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:04.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:04 smithi037 ceph-mon[31974]: Reconfiguring daemon mgr.x on smithi037 2024-03-26T06:15:04.798 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:04.530+0000 7f28af4fb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:15:04.798 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:04.798+0000 7f28af4fb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T06:15:04.798 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:04.798 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.798 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.798 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:04.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:04.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T06:15:04.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T06:15:04.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T06:15:04.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:04.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:04 smithi149 ceph-mon[36681]: Reconfiguring daemon mgr.x on smithi037 2024-03-26T06:15:04.809 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:04.623+0000 7fd6382b6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T06:15:04.809 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:04.713+0000 7fd6382b6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T06:15:04.810 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:04.810+0000 7fd6382b6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T06:15:04.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:04.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:04.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:04.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:04.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T06:15:04.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T06:15:04.811 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T06:15:04.811 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:04.811 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-mon[36504]: Reconfiguring daemon mgr.x on smithi037 2024-03-26T06:15:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.985+0000 7fa4d1b84700 1 -- 172.21.15.37:0/2835470111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4cc102920 msgr2=0x7fa4cc1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.985+0000 7fa4d1b84700 1 --2- 172.21.15.37:0/2835470111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4cc102920 0x7fa4cc1091d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400b0d0 tx=0x7fa4c400b3e0 comp rx=0 tx=0).stop 2024-03-26T06:15:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.985+0000 7fa4d1b84700 1 -- 172.21.15.37:0/2835470111 shutdown_connections 2024-03-26T06:15:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.985+0000 7fa4d1b84700 1 --2- 172.21.15.37:0/2835470111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4cc102920 0x7fa4cc1091d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.985+0000 7fa4d1b84700 1 --2- 172.21.15.37:0/2835470111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4cc100d80 0x7fa4cc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.985+0000 7fa4d1b84700 1 --2- 172.21.15.37:0/2835470111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc101f80 0x7fa4cc1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.985+0000 7fa4d1b84700 1 -- 172.21.15.37:0/2835470111 >> 172.21.15.37:0/2835470111 conn(0x7fa4cc0fc340 msgr2=0x7fa4cc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:15:04.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.986+0000 7fa4d1b84700 1 -- 172.21.15.37:0/2835470111 shutdown_connections 2024-03-26T06:15:04.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.986+0000 7fa4d1b84700 1 -- 172.21.15.37:0/2835470111 wait complete. 2024-03-26T06:15:04.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.986+0000 7fa4d1b84700 1 Processor -- start 2024-03-26T06:15:04.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.986+0000 7fa4d1b84700 1 -- start start 2024-03-26T06:15:04.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.986+0000 7fa4d1b84700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4cc100d80 0x7fa4cc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:04.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.986+0000 7fa4d1b84700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc101f80 0x7fa4cc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:04.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.986+0000 7fa4d1b84700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4cc102920 0x7fa4cc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:04.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.986+0000 7fa4d1b84700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc10c900 con 0x7fa4cc100d80 2024-03-26T06:15:04.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.986+0000 7fa4d1b84700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc10c780 con 0x7fa4cc102920 2024-03-26T06:15:04.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.986+0000 7fa4d1b84700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc10ca80 con 0x7fa4cc101f80 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4cb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4cc100d80 0x7fa4cc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4cb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4cc100d80 0x7fa4cc198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43862/0 (socket says 172.21.15.37:43862) 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4cb7fe700 1 -- 172.21.15.37:0/468295578 learned_addr learned my addr 172.21.15.37:0/468295578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4cbfff700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4cc102920 0x7fa4cc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4caffd700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc101f80 0x7fa4cc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4cb7fe700 1 -- 172.21.15.37:0/468295578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc101f80 msgr2=0x7fa4cc1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4cb7fe700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc101f80 0x7fa4cc1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4cb7fe700 1 -- 172.21.15.37:0/468295578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4cc102920 msgr2=0x7fa4cc19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4cb7fe700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4cc102920 0x7fa4cc19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4cb7fe700 1 -- 172.21.15.37:0/468295578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc19fce0 con 0x7fa4cc100d80 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4cb7fe700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4cc100d80 0x7fa4cc198080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00d9b0 tx=0x7fa4bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:15:04.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4c8ff9700 1 -- 172.21.15.37:0/468295578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc016070 con 0x7fa4cc100d80 2024-03-26T06:15:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4c8ff9700 1 -- 172.21.15.37:0/468295578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4bc00be30 con 0x7fa4cc100d80 2024-03-26T06:15:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4d1b84700 1 -- 172.21.15.37:0/468295578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc19ffd0 con 0x7fa4cc100d80 2024-03-26T06:15:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4c8ff9700 1 -- 172.21.15.37:0/468295578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc00f3c0 con 0x7fa4cc100d80 2024-03-26T06:15:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.987+0000 7fa4d1b84700 1 -- 172.21.15.37:0/468295578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4cc1a0420 con 0x7fa4cc100d80 2024-03-26T06:15:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.988+0000 7fa4c8ff9700 1 -- 172.21.15.37:0/468295578 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fa4bc019070 con 0x7fa4cc100d80 2024-03-26T06:15:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.988+0000 7fa4c8ff9700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa4b4039200 0x7fa4b403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.988+0000 7fa4d1b84700 1 -- 172.21.15.37:0/468295578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4cc068aa0 con 0x7fa4cc100d80 2024-03-26T06:15:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.988+0000 7fa4c8ff9700 1 -- 172.21.15.37:0/468295578 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa4bc04bd90 con 0x7fa4cc100d80 2024-03-26T06:15:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.988+0000 7fa4caffd700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa4b4039200 0x7fa4b403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:15:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.989+0000 7fa4caffd700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa4b4039200 0x7fa4b403b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00045b0 tx=0x7fa4c000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:15:04.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:04.994+0000 7fa4c8ff9700 1 -- 172.21.15.37:0/468295578 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa4bc020030 con 0x7fa4cc100d80 2024-03-26T06:15:05.096 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:04.991+0000 7fd6382b6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T06:15:05.158 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:05.031+0000 7f28af4fb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T06:15:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:05.210+0000 7fa4d1b84700 1 -- 172.21.15.37:0/468295578 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4cc0fe440 con 0x7fa4b4039200 2024-03-26T06:15:05.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:05 smithi037 ceph-mon[31974]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:05.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:05 smithi037 ceph-mon[31974]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:15:05.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:15:05.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:15:05.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:06.016 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:05 smithi066 ceph-mon[36504]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:06.017 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:05 smithi066 ceph-mon[36504]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:15:06.017 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:15:06.017 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:15:06.017 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:06.162 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:05.820+0000 7f28af4fb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T06:15:06.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:05 smithi149 ceph-mon[36681]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:06.163 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:05 smithi149 ceph-mon[36681]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:15:06.163 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:15:06.163 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:15:06.163 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:06.346 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:06.017+0000 7fd6382b6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T06:15:06.346 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:06.152+0000 7fd6382b6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T06:15:06.658 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:06 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:06.401+0000 7f28af4fb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T06:15:06.659 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:06 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:06.497+0000 7f28af4fb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T06:15:06.788 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:06.498+0000 7fd6382b6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T06:15:06.789 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:06.590+0000 7fd6382b6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T06:15:06.789 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:06.789+0000 7fd6382b6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:15:07.096 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:06.978+0000 7fd6382b6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T06:15:07.118 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:15:06 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:15:06.709+0000 7f28af4fb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T06:15:07.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:07.161+0000 7fa4c8ff9700 1 -- 172.21.15.37:0/468295578 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 14) v1 ==== 91698+0+0 (secure 0 0 0) 0x7fa4bc01b450 con 0x7fa4cc100d80 2024-03-26T06:15:07.378 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:07.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:07.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:07.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: Standby manager daemon z started 2024-03-26T06:15:07.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T06:15:07.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:15:07.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T06:15:07.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:15:07.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:07.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:07.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:07.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: Standby manager daemon z started 2024-03-26T06:15:07.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T06:15:07.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:15:07.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T06:15:07.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:15:07.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.537 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:07.269+0000 7fd6382b6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:15:07.537 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:07.537+0000 7fd6382b6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: Standby manager daemon z started 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.149:0/707372229' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:15:07.538 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:07.846 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:07 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:07.779+0000 7fd6382b6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T06:15:08.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:08 smithi037 ceph-mon[31974]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:08.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:08 smithi037 ceph-mon[31974]: mgrmap e14: x(active, since 80s), standbys: z 2024-03-26T06:15:08.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:08 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T06:15:08.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:08 smithi066 ceph-mon[36504]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:08.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:08 smithi066 ceph-mon[36504]: mgrmap e14: x(active, since 80s), standbys: z 2024-03-26T06:15:08.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:08 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T06:15:08.658 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:08 smithi149 ceph-mon[36681]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:08.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:08 smithi149 ceph-mon[36681]: mgrmap e14: x(active, since 80s), standbys: z 2024-03-26T06:15:08.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:08 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T06:15:09.096 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:08 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:08.609+0000 7fd6382b6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T06:15:09.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:09 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2632619274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16004b33-cf5b-43d7-8c6d-04223f061c9d"}]: dispatch 2024-03-26T06:15:09.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:09 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2632619274' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16004b33-cf5b-43d7-8c6d-04223f061c9d"}]': finished 2024-03-26T06:15:09.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:09 smithi037 ceph-mon[31974]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T06:15:09.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:09 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:09.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:09 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3601810245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:15:09.522 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:09 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2632619274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16004b33-cf5b-43d7-8c6d-04223f061c9d"}]: dispatch 2024-03-26T06:15:09.522 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:09 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2632619274' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16004b33-cf5b-43d7-8c6d-04223f061c9d"}]': finished 2024-03-26T06:15:09.523 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:09 smithi066 ceph-mon[36504]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T06:15:09.523 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:09 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:09.523 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:09 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3601810245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:15:09.523 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:09 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:09.202+0000 7fd6382b6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T06:15:09.523 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:09 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:09.302+0000 7fd6382b6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T06:15:09.523 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:15:09 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:15:09.523+0000 7fd6382b6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T06:15:09.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:09 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2632619274' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16004b33-cf5b-43d7-8c6d-04223f061c9d"}]: dispatch 2024-03-26T06:15:09.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:09 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2632619274' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16004b33-cf5b-43d7-8c6d-04223f061c9d"}]': finished 2024-03-26T06:15:09.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:09 smithi149 ceph-mon[36681]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T06:15:09.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:09 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:09.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:09 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3601810245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:15:10.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:10.346+0000 7fa4c8ff9700 1 -- 172.21.15.37:0/468295578 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7fa4bc00ec30 con 0x7fa4cc100d80 2024-03-26T06:15:10.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:10 smithi066 ceph-mon[36504]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:10.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:10 smithi066 ceph-mon[36504]: Standby manager daemon y started 2024-03-26T06:15:10.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:10 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T06:15:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:10 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:15:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:10 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T06:15:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:10 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:15:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:10 smithi149 ceph-mon[36681]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:10 smithi149 ceph-mon[36681]: Standby manager daemon y started 2024-03-26T06:15:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:10 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T06:15:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:10 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:15:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:10 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T06:15:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:10 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:15:10.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:10 smithi037 ceph-mon[31974]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:10.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:10 smithi037 ceph-mon[31974]: Standby manager daemon y started 2024-03-26T06:15:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:10 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T06:15:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:10 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:15:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:10 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T06:15:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:10 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.66:0/3588246835' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:15:11.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:11 smithi149 ceph-mon[36681]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-26T06:15:11.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:11 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T06:15:11.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:11 smithi037 ceph-mon[31974]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-26T06:15:11.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:11 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T06:15:11.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:11 smithi066 ceph-mon[36504]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-26T06:15:11.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:11 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T06:15:12.561 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:12 smithi037 ceph-mon[31974]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:12.658 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:12 smithi149 ceph-mon[36681]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:12.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:12 smithi066 ceph-mon[36504]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:14.634 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:14 smithi037 ceph-mon[31974]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:14.658 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:14 smithi149 ceph-mon[36681]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:14.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:14 smithi066 ceph-mon[36504]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:16.631 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:16 smithi037 ceph-mon[31974]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:16.632 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T06:15:16.632 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:16.658 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:16 smithi149 ceph-mon[36681]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:16.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T06:15:16.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:16.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:16 smithi066 ceph-mon[36504]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T06:15:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:17.637 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:17 smithi037 ceph-mon[31974]: Deploying daemon osd.0 on smithi037 2024-03-26T06:15:17.658 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:17 smithi149 ceph-mon[36681]: Deploying daemon osd.0 on smithi037 2024-03-26T06:15:17.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:17 smithi066 ceph-mon[36504]: Deploying daemon osd.0 on smithi037 2024-03-26T06:15:18.637 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:18 smithi037 ceph-mon[31974]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:18.658 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:18 smithi149 ceph-mon[36681]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:18.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:18 smithi066 ceph-mon[36504]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:20.658 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:20 smithi149 ceph-mon[36681]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:20.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:20 smithi037 ceph-mon[31974]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:20.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:20 smithi066 ceph-mon[36504]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:22.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:22 smithi066 ceph-mon[36504]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:22.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:22.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:22.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:22.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:22 smithi149 ceph-mon[36681]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:22.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:22.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:22.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:22.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:22 smithi037 ceph-mon[31974]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:22.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:22.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:22.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:24.691 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:24 smithi037 ceph-mon[31974]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:24.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.782+0000 7fa4c8ff9700 1 -- 172.21.15.37:0/468295578 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa4cc0fe440 con 0x7fa4b4039200 2024-03-26T06:15:24.783 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 0 on host 'smithi037' 2024-03-26T06:15:24.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 -- 172.21.15.37:0/468295578 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa4b4039200 msgr2=0x7fa4b403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:24.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa4b4039200 0x7fa4b403b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00045b0 tx=0x7fa4c000b3c0 comp rx=0 tx=0).stop 2024-03-26T06:15:24.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 -- 172.21.15.37:0/468295578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4cc100d80 msgr2=0x7fa4cc198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:24.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4cc100d80 0x7fa4cc198080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00d9b0 tx=0x7fa4bc00dcc0 comp rx=0 tx=0).stop 2024-03-26T06:15:24.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 -- 172.21.15.37:0/468295578 shutdown_connections 2024-03-26T06:15:24.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4cc100d80 0x7fa4cc198080 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:24.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa4b4039200 0x7fa4b403b6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:24.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4cc102920 0x7fa4cc19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:24.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 --2- 172.21.15.37:0/468295578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4cc101f80 0x7fa4cc1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:24.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 -- 172.21.15.37:0/468295578 >> 172.21.15.37:0/468295578 conn(0x7fa4cc0fc340 msgr2=0x7fa4cc0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:15:24.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 -- 172.21.15.37:0/468295578 shutdown_connections 2024-03-26T06:15:24.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:15:24.785+0000 7fa4d1b84700 1 -- 172.21.15.37:0/468295578 wait complete. 2024-03-26T06:15:24.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:24 smithi066 ceph-mon[36504]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:24.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:24 smithi149 ceph-mon[36681]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:25.441 DEBUG:teuthology.orchestra.run.smithi037:osd.0> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.0.service 2024-03-26T06:15:25.443 INFO:tasks.cephadm:Deploying osd.1 on smithi066 with /dev/vg_nvme/lv_4... 2024-03-26T06:15:25.443 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T06:15:25.703 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:15:26.018 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:25 smithi066 ceph-mon[36504]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:26.018 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:25 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:26.018 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:25 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:26.019 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:25 smithi066 ceph-mon[36504]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T06:15:26.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:25 smithi149 ceph-mon[36681]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:26.083 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:25 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:26.083 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:25 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:26.083 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:25 smithi149 ceph-mon[36681]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T06:15:26.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:25 smithi037 ceph-mon[31974]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:26.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:25 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:26.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:25 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:26.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:25 smithi037 ceph-mon[31974]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T06:15:27.042 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:26 smithi066 ceph-mon[36504]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T06:15:27.042 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:26 smithi066 ceph-mon[36504]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T06:15:27.042 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:26 smithi066 ceph-mon[36504]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:15:27.042 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:26 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:27.042 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:26 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:27.043 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:26 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:27.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:26 smithi149 ceph-mon[36681]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T06:15:27.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:26 smithi149 ceph-mon[36681]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T06:15:27.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:26 smithi149 ceph-mon[36681]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:15:27.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:26 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:27.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:26 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:27.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:26 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:27.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:26 smithi037 ceph-mon[31974]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T06:15:27.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:26 smithi037 ceph-mon[31974]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T06:15:27.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:26 smithi037 ceph-mon[31974]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:15:27.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:27.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:27.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:27.697 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:15:27 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0[46788]: 2024-03-26T06:15:27.586+0000 7ff88bfc8700 -1 osd.0 0 waiting for initial osdmap 2024-03-26T06:15:27.698 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:15:27 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0[46788]: 2024-03-26T06:15:27.590+0000 7ff8845b4700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:15:27.703 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T06:15:27.703 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T06:15:27.703 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-03-26T06:15:27.703 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-03-26T06:15:27.703 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111669 s, 939 MB/s 2024-03-26T06:15:27.703 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-03-26T06:15:28.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:27 smithi066 ceph-mon[36504]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:28.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:27 smithi066 ceph-mon[36504]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:15:28.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:27 smithi066 ceph-mon[36504]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T06:15:28.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:27 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:28.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:27 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:28.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:27 smithi066 ceph-mon[36504]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' 2024-03-26T06:15:28.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:27 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:28.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:27 smithi149 ceph-mon[36681]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:28.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:27 smithi149 ceph-mon[36681]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:15:28.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:27 smithi149 ceph-mon[36681]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T06:15:28.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:27 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:28.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:27 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:28.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:27 smithi149 ceph-mon[36681]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' 2024-03-26T06:15:28.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:27 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:28.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:27 smithi037 ceph-mon[31974]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:28.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:27 smithi037 ceph-mon[31974]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:15:28.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:27 smithi037 ceph-mon[31974]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T06:15:28.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:27 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:28.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:27 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:28.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:27 smithi037 ceph-mon[31974]: from='osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230]' entity='osd.0' 2024-03-26T06:15:28.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:27 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:28.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi066:vg_nvme/lv_4 2024-03-26T06:15:28.712 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:15:29.039 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:28 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:15:29.039 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:28 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:15:29.039 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:28 smithi066 ceph-mon[36504]: osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230] boot 2024-03-26T06:15:29.039 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:28 smithi066 ceph-mon[36504]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T06:15:29.039 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:28 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:29.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:28 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:15:29.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:28 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:15:29.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:28 smithi037 ceph-mon[31974]: osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230] boot 2024-03-26T06:15:29.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:28 smithi037 ceph-mon[31974]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T06:15:29.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:28 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:29.158 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:28 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:15:29.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:28 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:15:29.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:28 smithi149 ceph-mon[36681]: osd.0 [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230] boot 2024-03-26T06:15:29.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:28 smithi149 ceph-mon[36681]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T06:15:29.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:28 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:15:30.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.305+0000 7f99284bb700 1 -- 172.21.15.66:0/3589846532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99201031c0 msgr2=0x7f9920103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:30.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.305+0000 7f99284bb700 1 --2- 172.21.15.66:0/3589846532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99201031c0 0x7f9920103620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99100099c0 tx=0x7f9910009cd0 comp rx=0 tx=0).stop 2024-03-26T06:15:30.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.305+0000 7f99284bb700 1 -- 172.21.15.66:0/3589846532 shutdown_connections 2024-03-26T06:15:30.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.305+0000 7f99284bb700 1 --2- 172.21.15.66:0/3589846532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9920100140 0x7f9920100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:30.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.305+0000 7f99284bb700 1 --2- 172.21.15.66:0/3589846532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9920103b60 0x7f992010a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:30.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.305+0000 7f99284bb700 1 --2- 172.21.15.66:0/3589846532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99201031c0 0x7f9920103620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:30.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.305+0000 7f99284bb700 1 -- 172.21.15.66:0/3589846532 >> 172.21.15.66:0/3589846532 conn(0x7f99200fba80 msgr2=0x7f99200fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:15:30.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.305+0000 7f99284bb700 1 -- 172.21.15.66:0/3589846532 shutdown_connections 2024-03-26T06:15:30.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.305+0000 7f99284bb700 1 -- 172.21.15.66:0/3589846532 wait complete. 2024-03-26T06:15:30.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:30.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi037 2024-03-26T06:15:30.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:15:30.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi037 to 17093M 2024-03-26T06:15:30.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:30.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:30.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:30.307 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:30.307 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:30.307 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.305+0000 7f99284bb700 1 Processor -- start 2024-03-26T06:15:30.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.305+0000 7f99284bb700 1 -- start start 2024-03-26T06:15:30.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f99284bb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9920100140 0x7f9920102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:30.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f99284bb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99201031c0 0x7f9920101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:30.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f99284bb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9920103b60 0x7f99201016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:30.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f99284bb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992010da20 con 0x7f9920103b60 2024-03-26T06:15:30.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f99284bb700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992010d8a0 con 0x7f9920100140 2024-03-26T06:15:30.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f99284bb700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992010dba0 con 0x7f99201031c0 2024-03-26T06:15:30.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f9926257700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9920100140 0x7f9920102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:15:30.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f9926257700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9920100140 0x7f9920102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40190/0 (socket says 172.21.15.66:40190) 2024-03-26T06:15:30.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f9926257700 1 -- 172.21.15.66:0/805990982 learned_addr learned my addr 172.21.15.66:0/805990982 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f9925a56700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99201031c0 0x7f9920101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f9926257700 1 -- 172.21.15.66:0/805990982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99201031c0 msgr2=0x7f9920101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f9926257700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99201031c0 0x7f9920101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f9926257700 1 -- 172.21.15.66:0/805990982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9920103b60 msgr2=0x7f99201016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f9926257700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9920103b60 0x7f99201016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f9926257700 1 -- 172.21.15.66:0/805990982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9920101f60 con 0x7f9920100140 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.306+0000 7f9926257700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9920100140 0x7f9920102b20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f991c00d980 tx=0x7f991c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.307+0000 7f99177fe700 1 -- 172.21.15.66:0/805990982 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c017030 con 0x7f9920100140 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.307+0000 7f99284bb700 1 -- 172.21.15.66:0/805990982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99201a2170 con 0x7f9920100140 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.307+0000 7f99177fe700 1 -- 172.21.15.66:0/805990982 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f991c004500 con 0x7f9920100140 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.307+0000 7f99177fe700 1 -- 172.21.15.66:0/805990982 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c003aa0 con 0x7f9920100140 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.307+0000 7f99284bb700 1 -- 172.21.15.66:0/805990982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99201a26b0 con 0x7f9920100140 2024-03-26T06:15:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.308+0000 7f99284bb700 1 -- 172.21.15.66:0/805990982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9920068aa0 con 0x7f9920100140 2024-03-26T06:15:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.309+0000 7f99177fe700 1 -- 172.21.15.66:0/805990982 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f991c01a070 con 0x7f9920100140 2024-03-26T06:15:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.310+0000 7f99177fe700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f990c0a2800 0x7f990c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.310+0000 7f99177fe700 1 -- 172.21.15.66:0/805990982 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f991c0ccc90 con 0x7f9920100140 2024-03-26T06:15:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.310+0000 7f9925a56700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f990c0a2800 0x7f990c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:15:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.311+0000 7f9925a56700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f990c0a2800 0x7f990c0a4cc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9910000c00 tx=0x7f9910005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:15:30.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.314+0000 7f99177fe700 1 -- 172.21.15.66:0/805990982 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f991c09b780 con 0x7f9920100140 2024-03-26T06:15:30.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:30.527+0000 7f99284bb700 1 -- 172.21.15.66:0/805990982 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9920062220 con 0x7f990c0a2800 2024-03-26T06:15:30.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:30.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi037 2024-03-26T06:15:30.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:15:30.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi037 to 17093M 2024-03-26T06:15:30.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:30.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:30.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:30.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:30.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:30.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi037 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi037 to 17093M 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:30.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:30.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:31.457 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:31 smithi066 ceph-mon[36504]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T06:15:31.457 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:31 smithi066 ceph-mon[36504]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:15:31.457 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:15:31.457 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:15:31.457 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:31.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:31 smithi149 ceph-mon[36681]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T06:15:31.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:31 smithi149 ceph-mon[36681]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:15:31.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:15:31.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:15:31.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:31.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:31 smithi037 ceph-mon[31974]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T06:15:31.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:31 smithi037 ceph-mon[31974]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:15:31.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:15:31.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:15:31.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:32.530 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:32 smithi066 ceph-mon[36504]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:32.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:32 smithi037 ceph-mon[31974]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:32.658 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:32 smithi149 ceph-mon[36681]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:34.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:34 smithi066 ceph-mon[36504]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:34.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:34 smithi066 ceph-mon[36504]: from='client.? 172.21.15.66:0/3541792525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "802bebb3-7af9-45af-bd5e-b5b7e1156403"}]: dispatch 2024-03-26T06:15:34.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:34 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "802bebb3-7af9-45af-bd5e-b5b7e1156403"}]: dispatch 2024-03-26T06:15:34.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:34 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "802bebb3-7af9-45af-bd5e-b5b7e1156403"}]': finished 2024-03-26T06:15:34.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:34 smithi066 ceph-mon[36504]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T06:15:34.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:34 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:34.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:34 smithi066 ceph-mon[36504]: from='client.? 172.21.15.66:0/4197006614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:15:34.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:34 smithi149 ceph-mon[36681]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:34.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:34 smithi149 ceph-mon[36681]: from='client.? 172.21.15.66:0/3541792525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "802bebb3-7af9-45af-bd5e-b5b7e1156403"}]: dispatch 2024-03-26T06:15:34.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:34 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "802bebb3-7af9-45af-bd5e-b5b7e1156403"}]: dispatch 2024-03-26T06:15:34.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:34 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "802bebb3-7af9-45af-bd5e-b5b7e1156403"}]': finished 2024-03-26T06:15:34.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:34 smithi149 ceph-mon[36681]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T06:15:34.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:34 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:34.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:34 smithi149 ceph-mon[36681]: from='client.? 172.21.15.66:0/4197006614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:15:34.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:34 smithi037 ceph-mon[31974]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:34.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:34 smithi037 ceph-mon[31974]: from='client.? 172.21.15.66:0/3541792525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "802bebb3-7af9-45af-bd5e-b5b7e1156403"}]: dispatch 2024-03-26T06:15:34.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:34 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "802bebb3-7af9-45af-bd5e-b5b7e1156403"}]: dispatch 2024-03-26T06:15:34.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:34 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "802bebb3-7af9-45af-bd5e-b5b7e1156403"}]': finished 2024-03-26T06:15:34.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:34 smithi037 ceph-mon[31974]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T06:15:34.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:34 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:34.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:34 smithi037 ceph-mon[31974]: from='client.? 172.21.15.66:0/4197006614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:15:36.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:36 smithi066 ceph-mon[36504]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:36.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:36 smithi149 ceph-mon[36681]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:36.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:36 smithi037 ceph-mon[31974]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:38.553 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:38 smithi066 ceph-mon[36504]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:38.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:38 smithi149 ceph-mon[36681]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:38.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:38 smithi037 ceph-mon[31974]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:40.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:40 smithi066 ceph-mon[36504]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:40.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:40 smithi149 ceph-mon[36681]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:40.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:40 smithi037 ceph-mon[31974]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:41.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T06:15:41.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:41.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:41 smithi066 ceph-mon[36504]: Deploying daemon osd.1 on smithi066 2024-03-26T06:15:41.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T06:15:41.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:41.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:41 smithi149 ceph-mon[36681]: Deploying daemon osd.1 on smithi066 2024-03-26T06:15:41.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T06:15:41.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:41.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:41 smithi037 ceph-mon[31974]: Deploying daemon osd.1 on smithi066 2024-03-26T06:15:42.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:42 smithi066 ceph-mon[36504]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:42.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:42 smithi149 ceph-mon[36681]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:42.660 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:42 smithi037 ceph-mon[31974]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:44.346 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:44 smithi066 ceph-mon[36504]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:44.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:44 smithi149 ceph-mon[36681]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:44.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:44 smithi037 ceph-mon[31974]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:46.446 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:46 smithi066 ceph-mon[36504]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:46.447 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:46.447 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:46.447 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:46.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:46 smithi149 ceph-mon[36681]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:46.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:46.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:46.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:46.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:46 smithi037 ceph-mon[31974]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:46.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:46.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:46.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:48.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:48 smithi066 ceph-mon[36504]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:48.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:48 smithi149 ceph-mon[36681]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:48.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:48 smithi037 ceph-mon[31974]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:48.943 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 1 on host 'smithi066' 2024-03-26T06:15:48.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.942+0000 7f99177fe700 1 -- 172.21.15.66:0/805990982 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9920062220 con 0x7f990c0a2800 2024-03-26T06:15:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.944+0000 7f99284bb700 1 -- 172.21.15.66:0/805990982 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f990c0a2800 msgr2=0x7f990c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.945+0000 7f99284bb700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f990c0a2800 0x7f990c0a4cc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9910000c00 tx=0x7f9910005fb0 comp rx=0 tx=0).stop 2024-03-26T06:15:48.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.945+0000 7f99284bb700 1 -- 172.21.15.66:0/805990982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9920100140 msgr2=0x7f9920102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.945+0000 7f99284bb700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9920100140 0x7f9920102b20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f991c00d980 tx=0x7f991c00dc90 comp rx=0 tx=0).stop 2024-03-26T06:15:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.945+0000 7f99284bb700 1 -- 172.21.15.66:0/805990982 shutdown_connections 2024-03-26T06:15:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.945+0000 7f99284bb700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9920103b60 0x7f99201016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.945+0000 7f99284bb700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f990c0a2800 0x7f990c0a4cc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.945+0000 7f99284bb700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9920100140 0x7f9920102b20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.945+0000 7f99284bb700 1 --2- 172.21.15.66:0/805990982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99201031c0 0x7f9920101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.945+0000 7f99284bb700 1 -- 172.21.15.66:0/805990982 >> 172.21.15.66:0/805990982 conn(0x7f99200fba80 msgr2=0x7f9920106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:15:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.945+0000 7f99284bb700 1 -- 172.21.15.66:0/805990982 shutdown_connections 2024-03-26T06:15:48.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:15:48.945+0000 7f99284bb700 1 -- 172.21.15.66:0/805990982 wait complete. 2024-03-26T06:15:49.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:49.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:49.499 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.499 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.499 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.499 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:49.530 DEBUG:teuthology.orchestra.run.smithi066:osd.1> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.1.service 2024-03-26T06:15:49.533 INFO:tasks.cephadm:Deploying osd.2 on smithi149 with /dev/vg_nvme/lv_4... 2024-03-26T06:15:49.533 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T06:15:49.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:49 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:49 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:49 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:49.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:49 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:49.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:49 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:49 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:49 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:49 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:49.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:15:49.778 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:15:49.800 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:15:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1[41142]: 2024-03-26T06:15:49.707+0000 7ff2f267f7c0 -1 osd.1 0 log_to_monitors true 2024-03-26T06:15:50.642 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:50 smithi066 ceph-mon[36504]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:50.642 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:50 smithi066 ceph-mon[36504]: from='osd.1 [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T06:15:50.643 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:50 smithi066 ceph-mon[36504]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T06:15:50.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:50 smithi149 ceph-mon[36681]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:50.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:50 smithi149 ceph-mon[36681]: from='osd.1 [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T06:15:50.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:50 smithi149 ceph-mon[36681]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T06:15:50.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:50 smithi037 ceph-mon[31974]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:50.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:50 smithi037 ceph-mon[31974]: from='osd.1 [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T06:15:50.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:50 smithi037 ceph-mon[31974]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T06:15:51.580 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T06:15:51.581 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T06:15:51.581 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-26T06:15:51.581 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-26T06:15:51.581 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112576 s, 931 MB/s 2024-03-26T06:15:51.581 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-26T06:15:51.623 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:15:51 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1[41142]: 2024-03-26T06:15:51.371+0000 7ff2e74e3700 -1 osd.1 0 waiting for initial osdmap 2024-03-26T06:15:51.623 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:15:51 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1[41142]: 2024-03-26T06:15:51.376+0000 7ff2e3ad7700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:15:51.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:51 smithi066 ceph-mon[36504]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T06:15:51.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:51 smithi066 ceph-mon[36504]: from='osd.1 [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:15:51.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:51 smithi066 ceph-mon[36504]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T06:15:51.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:51 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:51.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:51 smithi066 ceph-mon[36504]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:15:51.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:51 smithi066 ceph-mon[36504]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:15:51.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:51 smithi149 ceph-mon[36681]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T06:15:51.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:51 smithi149 ceph-mon[36681]: from='osd.1 [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:15:51.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:51 smithi149 ceph-mon[36681]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T06:15:51.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:51 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:51.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:51 smithi149 ceph-mon[36681]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:15:51.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:51 smithi149 ceph-mon[36681]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:15:51.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:51 smithi037 ceph-mon[31974]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T06:15:51.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:51 smithi037 ceph-mon[31974]: from='osd.1 [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:15:51.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:51 smithi037 ceph-mon[31974]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T06:15:51.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:51 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:51.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:51 smithi037 ceph-mon[31974]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:15:51.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:51 smithi037 ceph-mon[31974]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:15:52.219 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-03-26T06:15:52.474 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:15:52.509 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:52.509 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T06:15:52.510 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:52.510 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:52.510 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.510 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.510 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:15:52.510 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.510 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:52.510 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:52.510 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.510 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T06:15:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:15:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:52.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T06:15:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:15:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:15:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:15:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:53.459 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:53 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:15:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:53 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:15:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:53 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi066 2024-03-26T06:15:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:53 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi066 to 17080M 2024-03-26T06:15:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:53 smithi149 ceph-mon[36681]: osd.1 [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667] boot 2024-03-26T06:15:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:53 smithi149 ceph-mon[36681]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T06:15:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:53.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:53 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:15:53.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:53 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:15:53.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:53 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi066 2024-03-26T06:15:53.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:53 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi066 to 17080M 2024-03-26T06:15:53.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:53 smithi037 ceph-mon[31974]: osd.1 [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667] boot 2024-03-26T06:15:53.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:53 smithi037 ceph-mon[31974]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T06:15:53.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:53.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:53 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:15:53.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:53 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:15:53.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:53 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi066 2024-03-26T06:15:53.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:53 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi066 to 17080M 2024-03-26T06:15:53.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:53 smithi066 ceph-mon[36504]: osd.1 [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667] boot 2024-03-26T06:15:53.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:53 smithi066 ceph-mon[36504]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T06:15:53.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:15:53.940 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.938+0000 7fb6939d5700 1 -- 172.21.15.149:0/1271925128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb68c100d10 msgr2=0x7fb68c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:53.940 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.938+0000 7fb6939d5700 1 --2- 172.21.15.149:0/1271925128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb68c100d10 0x7fb68c101130 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009990 tx=0x7fb67c009ca0 comp rx=0 tx=0).stop 2024-03-26T06:15:53.940 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.938+0000 7fb6939d5700 1 -- 172.21.15.149:0/1271925128 shutdown_connections 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.938+0000 7fb6939d5700 1 --2- 172.21.15.149:0/1271925128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb68c1028b0 0x7fb68c109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.938+0000 7fb6939d5700 1 --2- 172.21.15.149:0/1271925128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb68c100d10 0x7fb68c101130 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.938+0000 7fb6939d5700 1 --2- 172.21.15.149:0/1271925128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb68c101f10 0x7fb68c102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.938+0000 7fb6939d5700 1 -- 172.21.15.149:0/1271925128 >> 172.21.15.149:0/1271925128 conn(0x7fb68c0fc2d0 msgr2=0x7fb68c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.938+0000 7fb6939d5700 1 -- 172.21.15.149:0/1271925128 shutdown_connections 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.938+0000 7fb6939d5700 1 -- 172.21.15.149:0/1271925128 wait complete. 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb6939d5700 1 Processor -- start 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb6939d5700 1 -- start start 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb6939d5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb68c100d10 0x7fb68c104ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb6939d5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb68c101f10 0x7fb68c1032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb6939d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb68c1028b0 0x7fb68c103830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:53.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb6939d5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c10c8d0 con 0x7fb68c1028b0 2024-03-26T06:15:53.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb6939d5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c10c750 con 0x7fb68c101f10 2024-03-26T06:15:53.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb6939d5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c10ca50 con 0x7fb68c100d10 2024-03-26T06:15:53.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb691771700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb68c100d10 0x7fb68c104ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:15:53.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb691771700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb68c100d10 0x7fb68c104ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:34128/0 (socket says 172.21.15.149:34128) 2024-03-26T06:15:53.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb691771700 1 -- 172.21.15.149:0/2947147013 learned_addr learned my addr 172.21.15.149:0/2947147013 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-26T06:15:53.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb691f72700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb68c1028b0 0x7fb68c103830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:15:53.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb690f70700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb68c101f10 0x7fb68c1032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:15:53.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.939+0000 7fb691771700 1 -- 172.21.15.149:0/2947147013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb68c101f10 msgr2=0x7fb68c1032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:53.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.940+0000 7fb691771700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb68c101f10 0x7fb68c1032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:53.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.940+0000 7fb691771700 1 -- 172.21.15.149:0/2947147013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb68c1028b0 msgr2=0x7fb68c103830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:15:53.942 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.940+0000 7fb691771700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb68c1028b0 0x7fb68c103830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:15:53.943 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.940+0000 7fb691771700 1 -- 172.21.15.149:0/2947147013 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c1040e0 con 0x7fb68c100d10 2024-03-26T06:15:53.943 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.940+0000 7fb691f72700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb68c1028b0 0x7fb68c103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:15:53.943 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.940+0000 7fb691771700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb68c100d10 0x7fb68c104ca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0039c0 tx=0x7fb67c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:15:53.943 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.940+0000 7fb6827fc700 1 -- 172.21.15.149:0/2947147013 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c003d30 con 0x7fb68c100d10 2024-03-26T06:15:53.943 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.940+0000 7fb6827fc700 1 -- 172.21.15.149:0/2947147013 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb67c004830 con 0x7fb68c100d10 2024-03-26T06:15:53.943 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.940+0000 7fb6827fc700 1 -- 172.21.15.149:0/2947147013 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c005480 con 0x7fb68c100d10 2024-03-26T06:15:53.943 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.940+0000 7fb6939d5700 1 -- 172.21.15.149:0/2947147013 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c1a2140 con 0x7fb68c100d10 2024-03-26T06:15:53.943 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.940+0000 7fb6939d5700 1 -- 172.21.15.149:0/2947147013 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c1a2530 con 0x7fb68c100d10 2024-03-26T06:15:53.944 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.941+0000 7fb6939d5700 1 -- 172.21.15.149:0/2947147013 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68c068ad0 con 0x7fb68c100d10 2024-03-26T06:15:53.944 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.942+0000 7fb6827fc700 1 -- 172.21.15.149:0/2947147013 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7fb67c003ed0 con 0x7fb68c100d10 2024-03-26T06:15:53.944 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.942+0000 7fb6827fc700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb6780a2880 0x7fb6780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:15:53.944 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.943+0000 7fb6827fc700 1 -- 172.21.15.149:0/2947147013 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fb67c0ce2f0 con 0x7fb68c100d10 2024-03-26T06:15:53.944 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.943+0000 7fb690f70700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb6780a2880 0x7fb6780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:15:53.944 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.943+0000 7fb690f70700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb6780a2880 0x7fb6780a4d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb688005950 tx=0x7fb6880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:15:53.947 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:53.947+0000 7fb6827fc700 1 -- 172.21.15.149:0/2947147013 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb68c068ad0 con 0x7fb68c100d10 2024-03-26T06:15:54.152 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:15:54.152+0000 7fb6939d5700 1 -- 172.21.15.149:0/2947147013 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb68c0fe4e0 con 0x7fb6780a2880 2024-03-26T06:15:54.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:54 smithi149 ceph-mon[36681]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:54.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:54 smithi149 ceph-mon[36681]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T06:15:54.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:15:54.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:15:54.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:54 smithi037 ceph-mon[31974]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:54 smithi037 ceph-mon[31974]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T06:15:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:15:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:15:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:54.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:54 smithi066 ceph-mon[36504]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T06:15:54.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:54 smithi066 ceph-mon[36504]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T06:15:54.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:15:54.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:15:54.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:15:55.650 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:55 smithi149 ceph-mon[36681]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:15:55.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:55 smithi037 ceph-mon[31974]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:15:55.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:55 smithi066 ceph-mon[36504]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:15:56.658 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:56 smithi149 ceph-mon[36681]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:15:56.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:56 smithi037 ceph-mon[31974]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:15:56.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:56 smithi066 ceph-mon[36504]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:15:57.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:57 smithi149 ceph-mon[36681]: from='client.? 172.21.15.149:0/1689533553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8"}]: dispatch 2024-03-26T06:15:57.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:57 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8"}]: dispatch 2024-03-26T06:15:57.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:57 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8"}]': finished 2024-03-26T06:15:57.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:57 smithi149 ceph-mon[36681]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T06:15:57.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:15:57.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:57 smithi149 ceph-mon[36681]: from='client.? 172.21.15.149:0/121648984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:15:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:57 smithi037 ceph-mon[31974]: from='client.? 172.21.15.149:0/1689533553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8"}]: dispatch 2024-03-26T06:15:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:57 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8"}]: dispatch 2024-03-26T06:15:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:57 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8"}]': finished 2024-03-26T06:15:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:57 smithi037 ceph-mon[31974]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T06:15:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:15:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:57 smithi037 ceph-mon[31974]: from='client.? 172.21.15.149:0/121648984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:15:57.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:57 smithi066 ceph-mon[36504]: from='client.? 172.21.15.149:0/1689533553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8"}]: dispatch 2024-03-26T06:15:57.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:57 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8"}]: dispatch 2024-03-26T06:15:57.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:57 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8"}]': finished 2024-03-26T06:15:57.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:57 smithi066 ceph-mon[36504]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T06:15:57.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:15:57.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:57 smithi066 ceph-mon[36504]: from='client.? 172.21.15.149:0/121648984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:15:58.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:15:58 smithi149 ceph-mon[36681]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:15:58.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:15:58 smithi037 ceph-mon[31974]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:15:58.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:15:58 smithi066 ceph-mon[36504]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:00.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:00 smithi149 ceph-mon[36681]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:00.660 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:00 smithi066 ceph-mon[36504]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:00.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:00 smithi037 ceph-mon[31974]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:02.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:02 smithi149 ceph-mon[36681]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:02.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:02 smithi037 ceph-mon[31974]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:02.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:02 smithi066 ceph-mon[36504]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:04.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:04 smithi149 ceph-mon[36681]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:04.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T06:16:04.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:04.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:04 smithi037 ceph-mon[31974]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:04.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T06:16:04.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:04.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:04 smithi066 ceph-mon[36504]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:04.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T06:16:04.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:05.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:05 smithi037 ceph-mon[31974]: Deploying daemon osd.2 on smithi149 2024-03-26T06:16:05.733 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:05 smithi149 ceph-mon[36681]: Deploying daemon osd.2 on smithi149 2024-03-26T06:16:05.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:05 smithi066 ceph-mon[36504]: Deploying daemon osd.2 on smithi149 2024-03-26T06:16:06.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:06 smithi037 ceph-mon[31974]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:06.789 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:06 smithi149 ceph-mon[36681]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:06.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:06 smithi066 ceph-mon[36504]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:08.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:08 smithi149 ceph-mon[36681]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:08.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:08 smithi037 ceph-mon[31974]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:08.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:08 smithi066 ceph-mon[36504]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:10 smithi066 ceph-mon[36504]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:10 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:10 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:10 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:10 smithi149 ceph-mon[36681]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:10 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:10 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:10 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:10.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:10 smithi037 ceph-mon[31974]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:10 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:10 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:10 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:12.459 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:12 smithi149 ceph-mon[36681]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:12.524 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.523+0000 7fb6827fc700 1 -- 172.21.15.149:0/2947147013 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb68c0fe4e0 con 0x7fb6780a2880 2024-03-26T06:16:12.525 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 2 on host 'smithi149' 2024-03-26T06:16:12.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 -- 172.21.15.149:0/2947147013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb6780a2880 msgr2=0x7fb6780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:16:12.526 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb6780a2880 0x7fb6780a4d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb688005950 tx=0x7fb6880058e0 comp rx=0 tx=0).stop 2024-03-26T06:16:12.527 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 -- 172.21.15.149:0/2947147013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb68c100d10 msgr2=0x7fb68c104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:16:12.527 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb68c100d10 0x7fb68c104ca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0039c0 tx=0x7fb67c0047b0 comp rx=0 tx=0).stop 2024-03-26T06:16:12.527 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 -- 172.21.15.149:0/2947147013 shutdown_connections 2024-03-26T06:16:12.527 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb68c1028b0 0x7fb68c103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:12.527 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb6780a2880 0x7fb6780a4d40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:12.527 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb68c100d10 0x7fb68c104ca0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:12.527 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 --2- 172.21.15.149:0/2947147013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb68c101f10 0x7fb68c1032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:12.527 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 -- 172.21.15.149:0/2947147013 >> 172.21.15.149:0/2947147013 conn(0x7fb68c0fc2d0 msgr2=0x7fb68c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:16:12.527 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 -- 172.21.15.149:0/2947147013 shutdown_connections 2024-03-26T06:16:12.527 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:16:12.525+0000 7fb6939d5700 1 -- 172.21.15.149:0/2947147013 wait complete. 2024-03-26T06:16:12.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:12 smithi066 ceph-mon[36504]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:12.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:12 smithi037 ceph-mon[31974]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:13.240 DEBUG:teuthology.orchestra.run.smithi149:osd.2> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.2.service 2024-03-26T06:16:13.243 INFO:tasks.cephadm:Deploying osd.3 on smithi037 with /dev/vg_nvme/lv_3... 2024-03-26T06:16:13.243 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T06:16:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:16:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:13 smithi037 ceph-mon[31974]: from='osd.2 [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T06:16:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:13 smithi037 ceph-mon[31974]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T06:16:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:13.623 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:13.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:13.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:13.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:13 smithi066 ceph-mon[36504]: from='osd.2 [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T06:16:13.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:13 smithi066 ceph-mon[36504]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T06:16:13.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:13.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:13.901 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:13.901 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:13.901 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:13 smithi149 ceph-mon[36681]: from='osd.2 [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T06:16:13.901 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:13 smithi149 ceph-mon[36681]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T06:16:13.901 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:13.901 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:14.783 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:14 smithi037 ceph-mon[31974]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:14.784 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:14 smithi037 ceph-mon[31974]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T06:16:14.784 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:14 smithi037 ceph-mon[31974]: from='osd.2 [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:16:14.784 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:14 smithi037 ceph-mon[31974]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T06:16:14.784 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:14 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:14.784 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:14 smithi037 ceph-mon[31974]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:16:14.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:14 smithi066 ceph-mon[36504]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:14.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:14 smithi066 ceph-mon[36504]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T06:16:14.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:14 smithi066 ceph-mon[36504]: from='osd.2 [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:16:14.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:14 smithi066 ceph-mon[36504]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T06:16:14.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:14 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:14.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:14 smithi066 ceph-mon[36504]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:16:14.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:14 smithi149 ceph-mon[36681]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:14.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:14 smithi149 ceph-mon[36681]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T06:16:14.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:14 smithi149 ceph-mon[36681]: from='osd.2 [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:16:14.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:14 smithi149 ceph-mon[36681]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T06:16:14.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:14 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:14.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:14 smithi149 ceph-mon[36681]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:16:15.521 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:16:15 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2[41768]: 2024-03-26T06:16:15.313+0000 7f89fe2a8700 -1 osd.2 0 waiting for initial osdmap 2024-03-26T06:16:15.522 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:16:15 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2[41768]: 2024-03-26T06:16:15.321+0000 7f89f6894700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:16:15.673 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T06:16:15.673 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T06:16:15.673 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-03-26T06:16:15.674 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-03-26T06:16:15.674 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117664 s, 891 MB/s 2024-03-26T06:16:15.674 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-03-26T06:16:15.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:15 smithi037 ceph-mon[31974]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:16:15.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:15 smithi037 ceph-mon[31974]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T06:16:15.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:15 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:15.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:15 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:15.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:15 smithi037 ceph-mon[31974]: from='osd.2 ' entity='osd.2' 2024-03-26T06:16:15.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:15 smithi066 ceph-mon[36504]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:16:15.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:15 smithi066 ceph-mon[36504]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T06:16:15.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:15 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:15.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:15 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:15.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:15 smithi066 ceph-mon[36504]: from='osd.2 ' entity='osd.2' 2024-03-26T06:16:15.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:15 smithi149 ceph-mon[36681]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:16:15.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:15 smithi149 ceph-mon[36681]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T06:16:15.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:15 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:15.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:15 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:15.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:15 smithi149 ceph-mon[36681]: from='osd.2 ' entity='osd.2' 2024-03-26T06:16:16.219 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi037:vg_nvme/lv_3 2024-03-26T06:16:16.467 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:16:16.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:16:16.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:16:16.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:16.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:16.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: osd.2 [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377] boot 2024-03-26T06:16:16.835 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T06:16:16.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:16.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:16.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:16.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:16:16.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:16.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:16.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:16.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:16.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:16:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:16:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: osd.2 [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377] boot 2024-03-26T06:16:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T06:16:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:16:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:16.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:16.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:16.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:16.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:16:16.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:16:16.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T06:16:16.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:16.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: osd.2 [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377] boot 2024-03-26T06:16:16.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T06:16:16.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:16:16.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:16.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:16.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:16:16.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:16.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:16.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:16.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:17.605 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:17 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi149 2024-03-26T06:16:17.605 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:17 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi149 to 5807M 2024-03-26T06:16:17.605 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:17.606 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:17.606 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:17.606 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:17.606 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:17 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T06:16:17.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:17 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi149 2024-03-26T06:16:17.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:17 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi149 to 5807M 2024-03-26T06:16:17.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:17 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:17.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:17 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:17.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:17 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:17.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:17 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:17.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:17 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T06:16:17.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:17 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi149 2024-03-26T06:16:17.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:17 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi149 to 5807M 2024-03-26T06:16:17.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:17 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:17.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:17 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:17.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:17 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:17.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:17 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:17.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:17 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T06:16:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.088+0000 7f0ae3a72700 1 -- 172.21.15.37:0/237022313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc101e10 msgr2=0x7f0adc102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:16:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.088+0000 7f0ae3a72700 1 --2- 172.21.15.37:0/237022313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc101e10 0x7f0adc102230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009990 tx=0x7f0ad8009ca0 comp rx=0 tx=0).stop 2024-03-26T06:16:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.088+0000 7f0ae3a72700 1 -- 172.21.15.37:0/237022313 shutdown_connections 2024-03-26T06:16:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.088+0000 7f0ae3a72700 1 --2- 172.21.15.37:0/237022313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc101e10 0x7f0adc102230 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.088+0000 7f0ae3a72700 1 --2- 172.21.15.37:0/237022313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0adc103930 0x7f0adc10a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.088+0000 7f0ae3a72700 1 --2- 172.21.15.37:0/237022313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0adc102f70 0x7f0adc1033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.088+0000 7f0ae3a72700 1 -- 172.21.15.37:0/237022313 >> 172.21.15.37:0/237022313 conn(0x7f0adc0fd3f0 msgr2=0x7f0adc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:16:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.088+0000 7f0ae3a72700 1 -- 172.21.15.37:0/237022313 shutdown_connections 2024-03-26T06:16:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae3a72700 1 -- 172.21.15.37:0/237022313 wait complete. 2024-03-26T06:16:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae3a72700 1 Processor -- start 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae3a72700 1 -- start start 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae3a72700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc101e10 0x7f0adc075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae3a72700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0adc102f70 0x7f0adc0741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae3a72700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0adc103930 0x7f0adc074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae3a72700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc10da80 con 0x7f0adc101e10 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae3a72700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc10d900 con 0x7f0adc102f70 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae3a72700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc10dc00 con 0x7f0adc103930 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae180e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc101e10 0x7f0adc075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae180e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc101e10 0x7f0adc075b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40950/0 (socket says 172.21.15.37:40950) 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae180e700 1 -- 172.21.15.37:0/770748684 learned_addr learned my addr 172.21.15.37:0/770748684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.089+0000 7f0ae200f700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0adc103930 0x7f0adc074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ae100d700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0adc102f70 0x7f0adc0741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:16:18.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ae180e700 1 -- 172.21.15.37:0/770748684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0adc103930 msgr2=0x7f0adc074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ae180e700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0adc103930 0x7f0adc074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ae180e700 1 -- 172.21.15.37:0/770748684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0adc102f70 msgr2=0x7f0adc0741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ae180e700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0adc102f70 0x7f0adc0741d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ae180e700 1 -- 172.21.15.37:0/770748684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc074fc0 con 0x7f0adc101e10 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ae200f700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0adc103930 0x7f0adc074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ae100d700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0adc102f70 0x7f0adc0741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ae180e700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc101e10 0x7f0adc075b80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8005b30 tx=0x7f0ad8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ad2ffd700 1 -- 172.21.15.37:0/770748684 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8026070 con 0x7f0adc101e10 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ae3a72700 1 -- 172.21.15.37:0/770748684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0adc1a21c0 con 0x7f0adc101e10 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ad2ffd700 1 -- 172.21.15.37:0/770748684 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ad8004220 con 0x7f0adc101e10 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ad2ffd700 1 -- 172.21.15.37:0/770748684 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80053a0 con 0x7f0adc101e10 2024-03-26T06:16:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.090+0000 7f0ae3a72700 1 -- 172.21.15.37:0/770748684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0adc1a2690 con 0x7f0adc101e10 2024-03-26T06:16:18.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.091+0000 7f0ae3a72700 1 -- 172.21.15.37:0/770748684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0adc068aa0 con 0x7f0adc101e10 2024-03-26T06:16:18.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.092+0000 7f0ad2ffd700 1 -- 172.21.15.37:0/770748684 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f0ad8029070 con 0x7f0adc101e10 2024-03-26T06:16:18.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.093+0000 7f0ad2ffd700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ac80a2880 0x7f0ac80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:16:18.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.093+0000 7f0ae100d700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ac80a2880 0x7f0ac80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:16:18.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.093+0000 7f0ad2ffd700 1 -- 172.21.15.37:0/770748684 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f0ad80ce5c0 con 0x7f0adc101e10 2024-03-26T06:16:18.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.093+0000 7f0ae100d700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ac80a2880 0x7f0ac80a4d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0acc007900 tx=0x7f0acc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:16:18.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.097+0000 7f0ad2ffd700 1 -- 172.21.15.37:0/770748684 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0ad8030020 con 0x7f0adc101e10 2024-03-26T06:16:18.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:18.312+0000 7f0ae3a72700 1 -- 172.21.15.37:0/770748684 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0adc062220 con 0x7f0ac80a2880 2024-03-26T06:16:18.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:18 smithi037 ceph-mon[31974]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:18 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T06:16:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:18 smithi037 ceph-mon[31974]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T06:16:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:18 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T06:16:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:18 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:16:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:18 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:16:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:18 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:18.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:18 smithi066 ceph-mon[36504]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:18.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:18 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T06:16:18.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:18 smithi066 ceph-mon[36504]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T06:16:18.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:18 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T06:16:18.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:18 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:16:18.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:18 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:16:18.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:18 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:18.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:18 smithi149 ceph-mon[36681]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:18.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:18 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T06:16:18.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:18 smithi149 ceph-mon[36681]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T06:16:18.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:18 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T06:16:18.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:18 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:16:18.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:18 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:16:18.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:18 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:19.198 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:16:19 smithi037 sudo[51617]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T06:16:19.198 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:16:19 smithi037 sudo[51617]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:19.198 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:16:19 smithi037 sudo[51617]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:19.198 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:16:19 smithi037 sudo[51617]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:19.535 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:16:19 smithi037 sudo[51634]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T06:16:19.535 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:16:19 smithi037 sudo[51634]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:19.535 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:16:19 smithi037 sudo[51634]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:19.535 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:16:19 smithi037 sudo[51634]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:19 smithi066 ceph-mon[36504]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:16:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T06:16:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:19 smithi066 ceph-mon[36504]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T06:16:19.808 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:19.907 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:19 smithi037 ceph-mon[31974]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:16:19.907 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T06:16:19.907 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:19 smithi037 ceph-mon[31974]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T06:16:19.907 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:19.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:19 smithi149 ceph-mon[36681]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:16:19.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T06:16:19.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:19 smithi149 ceph-mon[36681]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T06:16:19.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:20.096 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:16:19 smithi066 sudo[45293]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T06:16:20.096 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:16:19 smithi066 sudo[45293]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:20.097 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:16:19 smithi066 sudo[45293]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:20.097 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:16:19 smithi066 sudo[45293]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:20.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:20.543+0000 7f0ad2ffd700 1 -- 172.21.15.37:0/770748684 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0ad8098a80 con 0x7f0adc101e10 2024-03-26T06:16:20.547 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:16:20 smithi066 sudo[45296]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T06:16:20.547 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:16:20 smithi066 sudo[45296]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:20.547 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:16:20 smithi066 sudo[45296]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:20.547 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:16:20 smithi066 sudo[45296]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:20.784 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:20 smithi149 ceph-mon[36681]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:20.784 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:20 smithi149 ceph-mon[36681]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T06:16:20.784 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:20 smithi149 ceph-mon[36681]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T06:16:20.784 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:20 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:20.784 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:16:20 smithi149 sudo[45840]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T06:16:20.784 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:16:20 smithi149 sudo[45840]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:20.784 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:16:20 smithi149 sudo[45840]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:20.785 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:16:20 smithi149 sudo[45840]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:20.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:20 smithi066 ceph-mon[36504]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:20.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:20 smithi066 ceph-mon[36504]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T06:16:20.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:20 smithi066 ceph-mon[36504]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T06:16:20.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:20 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:20.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:20 smithi037 ceph-mon[31974]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:20.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:20 smithi037 ceph-mon[31974]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T06:16:20.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:20 smithi037 ceph-mon[31974]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T06:16:20.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:20 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:21.159 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:16:20 smithi149 sudo[45843]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T06:16:21.159 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:16:20 smithi149 sudo[45843]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:21.159 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:16:20 smithi149 sudo[45843]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:21.159 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:16:20 smithi149 sudo[45843]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:21.336 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 sudo[51711]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T06:16:21.336 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 sudo[51711]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:21.336 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 sudo[51711]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:21.336 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 sudo[51711]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:21.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 sudo[51714]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T06:16:21.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 sudo[51714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:21.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 sudo[51714]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:21.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 sudo[51714]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:21.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 sudo[45846]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T06:16:21.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 sudo[45846]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:21.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 sudo[45846]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:21.837 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 sudo[45846]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:22.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 sudo[45849]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T06:16:22.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 sudo[45849]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:22.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 sudo[45849]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:22.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 sudo[45849]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:22.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T06:16:22.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:22.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T06:16:22.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/158440268' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dcbfb5-853c-4843-b19d-5018e9c6e268"}]: dispatch 2024-03-26T06:16:22.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T06:16:22.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:16:22.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dcbfb5-853c-4843-b19d-5018e9c6e268"}]: dispatch 2024-03-26T06:16:22.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:16:22.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:16:22.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68dcbfb5-853c-4843-b19d-5018e9c6e268"}]': finished 2024-03-26T06:16:22.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T06:16:22.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:22.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:16:22.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:16:22.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:21 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:16:22.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T06:16:22.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/158440268' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dcbfb5-853c-4843-b19d-5018e9c6e268"}]: dispatch 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dcbfb5-853c-4843-b19d-5018e9c6e268"}]: dispatch 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68dcbfb5-853c-4843-b19d-5018e9c6e268"}]': finished 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:16:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:16:22.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 sudo[45299]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T06:16:22.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 sudo[45299]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:22.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 sudo[45299]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:22.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 sudo[45299]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 sudo[45302]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T06:16:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 sudo[45302]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T06:16:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 sudo[45302]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T06:16:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 sudo[45302]: pam_unix(sudo:session): session closed for user root 2024-03-26T06:16:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T06:16:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T06:16:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/158440268' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dcbfb5-853c-4843-b19d-5018e9c6e268"}]: dispatch 2024-03-26T06:16:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T06:16:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dcbfb5-853c-4843-b19d-5018e9c6e268"}]: dispatch 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68dcbfb5-853c-4843-b19d-5018e9c6e268"}]': finished 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T06:16:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: Cluster is now healthy 2024-03-26T06:16:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T06:16:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T06:16:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:16:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:16:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:16:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2005403334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:16:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:22 smithi066 ceph-mon[36504]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T06:16:23.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:22 smithi149 ceph-mon[36681]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:23.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:22 smithi149 ceph-mon[36681]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T06:16:23.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:22 smithi149 ceph-mon[36681]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T06:16:23.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:22 smithi149 ceph-mon[36681]: Cluster is now healthy 2024-03-26T06:16:23.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:22 smithi149 ceph-mon[36681]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T06:16:23.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:22 smithi149 ceph-mon[36681]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T06:16:23.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:16:23.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:16:23.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:16:23.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:22 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2005403334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:16:23.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:22 smithi149 ceph-mon[36681]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T06:16:23.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:22 smithi037 ceph-mon[31974]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:23.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:22 smithi037 ceph-mon[31974]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T06:16:23.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:22 smithi037 ceph-mon[31974]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T06:16:23.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:22 smithi037 ceph-mon[31974]: Cluster is now healthy 2024-03-26T06:16:23.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:22 smithi037 ceph-mon[31974]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T06:16:23.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:22 smithi037 ceph-mon[31974]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T06:16:23.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:16:23.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:16:23.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:16:23.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:22 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2005403334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:16:23.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:22 smithi037 ceph-mon[31974]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T06:16:24.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:23 smithi149 ceph-mon[36681]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:24.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:23 smithi037 ceph-mon[31974]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:24.346 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:23 smithi066 ceph-mon[36504]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:25.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:25 smithi037 ceph-mon[31974]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:26.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:25 smithi149 ceph-mon[36681]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:26.346 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:25 smithi066 ceph-mon[36504]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:28.154 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:27 smithi149 ceph-mon[36681]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:28.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:27 smithi037 ceph-mon[31974]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:28.346 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:27 smithi066 ceph-mon[36504]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:30.149 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:29 smithi066 ceph-mon[36504]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:30.149 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:29 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T06:16:30.149 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:29 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:30.149 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:29 smithi066 ceph-mon[36504]: Deploying daemon osd.3 on smithi037 2024-03-26T06:16:30.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:29 smithi149 ceph-mon[36681]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:30.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:29 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T06:16:30.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:29 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:30.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:29 smithi149 ceph-mon[36681]: Deploying daemon osd.3 on smithi037 2024-03-26T06:16:30.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:29 smithi037 ceph-mon[31974]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:30.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:29 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T06:16:30.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:29 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:30.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:29 smithi037 ceph-mon[31974]: Deploying daemon osd.3 on smithi037 2024-03-26T06:16:32.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:31 smithi037 ceph-mon[31974]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:32.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:31 smithi149 ceph-mon[36681]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:32.346 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:31 smithi066 ceph-mon[36504]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:34.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:33 smithi037 ceph-mon[31974]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:34.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:33 smithi149 ceph-mon[36681]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:34.346 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:33 smithi066 ceph-mon[36504]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:35.829 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:35 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:35.830 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:35 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:35.830 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:35 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:35.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:35 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:35.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:35 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:35.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:35 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:35.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:35 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:35.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:35 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:35.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:35 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:36.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:36 smithi037 ceph-mon[31974]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:36.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:36 smithi066 ceph-mon[36504]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:36.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:36 smithi149 ceph-mon[36681]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:38.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.016+0000 7f0ad2ffd700 1 -- 172.21.15.37:0/770748684 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0adc062220 con 0x7f0ac80a2880 2024-03-26T06:16:38.018 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 3 on host 'smithi037' 2024-03-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.018+0000 7f0ae3a72700 1 -- 172.21.15.37:0/770748684 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ac80a2880 msgr2=0x7f0ac80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.018+0000 7f0ae3a72700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ac80a2880 0x7f0ac80a4d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0acc007900 tx=0x7f0acc008040 comp rx=0 tx=0).stop 2024-03-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.018+0000 7f0ae3a72700 1 -- 172.21.15.37:0/770748684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc101e10 msgr2=0x7f0adc075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.018+0000 7f0ae3a72700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc101e10 0x7f0adc075b80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8005b30 tx=0x7f0ad8009580 comp rx=0 tx=0).stop 2024-03-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.019+0000 7f0ae3a72700 1 -- 172.21.15.37:0/770748684 shutdown_connections 2024-03-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.019+0000 7f0ae3a72700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc101e10 0x7f0adc075b80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.019+0000 7f0ae3a72700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0ac80a2880 0x7f0ac80a4d40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.019+0000 7f0ae3a72700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0adc103930 0x7f0adc074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.019+0000 7f0ae3a72700 1 --2- 172.21.15.37:0/770748684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0adc102f70 0x7f0adc0741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.019+0000 7f0ae3a72700 1 -- 172.21.15.37:0/770748684 >> 172.21.15.37:0/770748684 conn(0x7f0adc0fd3f0 msgr2=0x7f0adc106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:16:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.019+0000 7f0ae3a72700 1 -- 172.21.15.37:0/770748684 shutdown_connections 2024-03-26T06:16:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:16:38.019+0000 7f0ae3a72700 1 -- 172.21.15.37:0/770748684 wait complete. 2024-03-26T06:16:38.685 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:38 smithi037 ceph-mon[31974]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:38.686 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.686 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.686 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.686 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.686 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:38 smithi037 ceph-mon[31974]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T06:16:38.717 DEBUG:teuthology.orchestra.run.smithi037:osd.3> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.3.service 2024-03-26T06:16:38.719 INFO:tasks.cephadm:Deploying osd.4 on smithi066 with /dev/vg_nvme/lv_3... 2024-03-26T06:16:38.719 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T06:16:38.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:38 smithi066 ceph-mon[36504]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:38.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:38 smithi066 ceph-mon[36504]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T06:16:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:38 smithi149 ceph-mon[36681]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:38 smithi149 ceph-mon[36681]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T06:16:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:16:40.132 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:40 smithi037 ceph-mon[31974]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:40.132 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:40 smithi037 ceph-mon[31974]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T06:16:40.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:40 smithi037 ceph-mon[31974]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T06:16:40.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:40 smithi037 ceph-mon[31974]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:16:40.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:40 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:40.190 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:40 smithi066 ceph-mon[36504]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:40.190 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:40 smithi066 ceph-mon[36504]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T06:16:40.190 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:40 smithi066 ceph-mon[36504]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T06:16:40.191 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:40 smithi066 ceph-mon[36504]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:16:40.191 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:40 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:40.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:40 smithi149 ceph-mon[36681]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:40.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:40 smithi149 ceph-mon[36681]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T06:16:40.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:40 smithi149 ceph-mon[36681]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T06:16:40.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:40 smithi149 ceph-mon[36681]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:16:40.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:40 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:41.034 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T06:16:41.034 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T06:16:41.035 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-03-26T06:16:41.035 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-03-26T06:16:41.035 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011244 s, 933 MB/s 2024-03-26T06:16:41.035 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-03-26T06:16:41.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:41 smithi037 ceph-mon[31974]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:16:41.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:41 smithi037 ceph-mon[31974]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T06:16:41.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:41.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:41.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:41 smithi037 ceph-mon[31974]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' 2024-03-26T06:16:41.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:41 smithi037 ceph-mon[31974]: osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060] boot 2024-03-26T06:16:41.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:41 smithi037 ceph-mon[31974]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T06:16:41.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:41.200 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:16:40 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3[53756]: 2024-03-26T06:16:40.894+0000 7f72bd207700 -1 osd.3 0 waiting for initial osdmap 2024-03-26T06:16:41.200 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:16:40 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3[53756]: 2024-03-26T06:16:40.902+0000 7f72b57f3700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:16:41.346 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:41 smithi066 ceph-mon[36504]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:16:41.346 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:41 smithi066 ceph-mon[36504]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T06:16:41.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:41.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:41.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:41 smithi066 ceph-mon[36504]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' 2024-03-26T06:16:41.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:41 smithi066 ceph-mon[36504]: osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060] boot 2024-03-26T06:16:41.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:41 smithi066 ceph-mon[36504]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T06:16:41.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:41.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:41 smithi149 ceph-mon[36681]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:16:41.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:41 smithi149 ceph-mon[36681]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T06:16:41.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:41.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:41.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:41 smithi149 ceph-mon[36681]: from='osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060]' entity='osd.3' 2024-03-26T06:16:41.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:41 smithi149 ceph-mon[36681]: osd.3 [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060] boot 2024-03-26T06:16:41.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:41 smithi149 ceph-mon[36681]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T06:16:41.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:16:41.683 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi066:vg_nvme/lv_3 2024-03-26T06:16:41.945 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:16:42.344 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:16:42.344 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:42 smithi066 ceph-mon[36504]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T06:16:42.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:16:42.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:16:42.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:42.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:16:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:16:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:42 smithi149 ceph-mon[36681]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T06:16:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:16:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:16:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-26T06:16:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:16:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:16:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:42.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:42.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:16:42.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:42.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:16:42.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:16:42.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:42 smithi037 ceph-mon[31974]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T06:16:43.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:43 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi037 2024-03-26T06:16:43.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:43 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi037 to 8546M 2024-03-26T06:16:43.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:43 smithi149 ceph-mon[36681]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T06:16:43.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:43 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi037 2024-03-26T06:16:43.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:43 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi037 to 8546M 2024-03-26T06:16:43.431 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:43 smithi066 ceph-mon[36504]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T06:16:43.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.438+0000 7f2255fc0700 1 -- 172.21.15.66:0/3092372900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2250101f10 msgr2=0x7f2250102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:16:43.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.438+0000 7f2255fc0700 1 --2- 172.21.15.66:0/3092372900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2250101f10 0x7f2250102370 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f22440099c0 tx=0x7f2244009cd0 comp rx=0 tx=0).stop 2024-03-26T06:16:43.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.438+0000 7f2255fc0700 1 -- 172.21.15.66:0/3092372900 shutdown_connections 2024-03-26T06:16:43.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.438+0000 7f2255fc0700 1 --2- 172.21.15.66:0/3092372900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2250100d10 0x7f2250101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:43.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.438+0000 7f2255fc0700 1 --2- 172.21.15.66:0/3092372900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22501028b0 0x7f2250109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:43.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.438+0000 7f2255fc0700 1 --2- 172.21.15.66:0/3092372900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2250101f10 0x7f2250102370 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:43.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.438+0000 7f2255fc0700 1 -- 172.21.15.66:0/3092372900 >> 172.21.15.66:0/3092372900 conn(0x7f22500fc2b0 msgr2=0x7f22500fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:16:43.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.438+0000 7f2255fc0700 1 -- 172.21.15.66:0/3092372900 shutdown_connections 2024-03-26T06:16:43.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.438+0000 7f2255fc0700 1 -- 172.21.15.66:0/3092372900 wait complete. 2024-03-26T06:16:43.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.439+0000 7f2255fc0700 1 Processor -- start 2024-03-26T06:16:43.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.439+0000 7f2255fc0700 1 -- start start 2024-03-26T06:16:43.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.439+0000 7f2255fc0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2250100d10 0x7f225006a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:16:43.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.439+0000 7f2255fc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2250101f10 0x7f225019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:16:43.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.439+0000 7f2255fc0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22501028b0 0x7f225019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:16:43.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.439+0000 7f2255fc0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225010c900 con 0x7f2250101f10 2024-03-26T06:16:43.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.439+0000 7f2255fc0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225010c780 con 0x7f22501028b0 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.439+0000 7f2255fc0700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225010ca80 con 0x7f2250100d10 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22501028b0 0x7f225019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22501028b0 0x7f225019f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46924/0 (socket says 172.21.15.66:46924) 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224ffff700 1 -- 172.21.15.66:0/2448197344 learned_addr learned my addr 172.21.15.66:0/2448197344 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224effd700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2250101f10 0x7f225019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224f7fe700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2250100d10 0x7f225006a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224ffff700 1 -- 172.21.15.66:0/2448197344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2250100d10 msgr2=0x7f225006a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224ffff700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2250100d10 0x7f225006a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224ffff700 1 -- 172.21.15.66:0/2448197344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2250101f10 msgr2=0x7f225019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224ffff700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2250101f10 0x7f225019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224ffff700 1 -- 172.21.15.66:0/2448197344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225019fca0 con 0x7f22501028b0 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224ffff700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22501028b0 0x7f225019f3d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f224800ada0 tx=0x7f224800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:16:43.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224cff9700 1 -- 172.21.15.66:0/2448197344 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248017020 con 0x7f22501028b0 2024-03-26T06:16:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f2255fc0700 1 -- 172.21.15.66:0/2448197344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225019ff90 con 0x7f22501028b0 2024-03-26T06:16:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224cff9700 1 -- 172.21.15.66:0/2448197344 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f224800cdb0 con 0x7f22501028b0 2024-03-26T06:16:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f224cff9700 1 -- 172.21.15.66:0/2448197344 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22480053e0 con 0x7f22501028b0 2024-03-26T06:16:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.440+0000 7f2255fc0700 1 -- 172.21.15.66:0/2448197344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22501a0440 con 0x7f22501028b0 2024-03-26T06:16:43.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.441+0000 7f2255fc0700 1 -- 172.21.15.66:0/2448197344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2250068aa0 con 0x7f22501028b0 2024-03-26T06:16:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.442+0000 7f224cff9700 1 -- 172.21.15.66:0/2448197344 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f224800c8d0 con 0x7f22501028b0 2024-03-26T06:16:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.443+0000 7f224cff9700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22380a2890 0x7f22380a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:16:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.444+0000 7f224cff9700 1 -- 172.21.15.66:0/2448197344 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f22480ccf30 con 0x7f22501028b0 2024-03-26T06:16:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.444+0000 7f224f7fe700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22380a2890 0x7f22380a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:16:43.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.444+0000 7f224f7fe700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22380a2890 0x7f22380a4d50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2250101d70 tx=0x7f2240009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:16:43.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:43 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi037 2024-03-26T06:16:43.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:43 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi037 to 8546M 2024-03-26T06:16:43.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:43 smithi037 ceph-mon[31974]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T06:16:43.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.448+0000 7f224cff9700 1 -- 172.21.15.66:0/2448197344 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2248021020 con 0x7f22501028b0 2024-03-26T06:16:43.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:16:43.663+0000 7f2255fc0700 1 -- 172.21.15.66:0/2448197344 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2250062220 con 0x7f22380a2890 2024-03-26T06:16:44.380 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:44 smithi066 ceph-mon[36504]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 517 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:16:44.380 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:44 smithi066 ceph-mon[36504]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:16:44.380 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:16:44.381 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:16:44.381 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:44.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:44 smithi149 ceph-mon[36681]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 517 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:16:44.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:44 smithi149 ceph-mon[36681]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:16:44.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:16:44.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:16:44.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:44.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:44 smithi037 ceph-mon[31974]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 517 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:16:44.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:44 smithi037 ceph-mon[31974]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:16:44.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:16:44.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:16:44.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:45.220 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:45 smithi066 ceph-mon[36504]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T06:16:45.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:45 smithi149 ceph-mon[36681]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T06:16:45.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:45 smithi037 ceph-mon[31974]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T06:16:46.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:46 smithi149 ceph-mon[36681]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T06:16:46.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:46 smithi149 ceph-mon[36681]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T06:16:46.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:46 smithi149 ceph-mon[36681]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T06:16:46.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:46 smithi037 ceph-mon[31974]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T06:16:46.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:46 smithi037 ceph-mon[31974]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T06:16:46.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:46 smithi037 ceph-mon[31974]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T06:16:46.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:46 smithi066 ceph-mon[36504]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T06:16:46.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:46 smithi066 ceph-mon[36504]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T06:16:46.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:46 smithi066 ceph-mon[36504]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T06:16:47.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:47 smithi149 ceph-mon[36681]: from='client.? 172.21.15.66:0/1532424173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce571a6-aa1d-40e6-a6d3-df9122391314"}]: dispatch 2024-03-26T06:16:47.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:47 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce571a6-aa1d-40e6-a6d3-df9122391314"}]: dispatch 2024-03-26T06:16:47.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:47 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ce571a6-aa1d-40e6-a6d3-df9122391314"}]': finished 2024-03-26T06:16:47.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:47 smithi149 ceph-mon[36681]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T06:16:47.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:16:47.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:47 smithi149 ceph-mon[36681]: from='client.? 172.21.15.66:0/641140057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:16:47.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:47 smithi037 ceph-mon[31974]: from='client.? 172.21.15.66:0/1532424173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce571a6-aa1d-40e6-a6d3-df9122391314"}]: dispatch 2024-03-26T06:16:47.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:47 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce571a6-aa1d-40e6-a6d3-df9122391314"}]: dispatch 2024-03-26T06:16:47.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:47 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ce571a6-aa1d-40e6-a6d3-df9122391314"}]': finished 2024-03-26T06:16:47.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:47 smithi037 ceph-mon[31974]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T06:16:47.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:16:47.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:47 smithi037 ceph-mon[31974]: from='client.? 172.21.15.66:0/641140057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:16:47.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:47 smithi066 ceph-mon[36504]: from='client.? 172.21.15.66:0/1532424173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce571a6-aa1d-40e6-a6d3-df9122391314"}]: dispatch 2024-03-26T06:16:47.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:47 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ce571a6-aa1d-40e6-a6d3-df9122391314"}]: dispatch 2024-03-26T06:16:47.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:47 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ce571a6-aa1d-40e6-a6d3-df9122391314"}]': finished 2024-03-26T06:16:47.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:47 smithi066 ceph-mon[36504]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T06:16:47.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:16:47.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:47 smithi066 ceph-mon[36504]: from='client.? 172.21.15.66:0/641140057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:16:48.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:48 smithi149 ceph-mon[36681]: pgmap v110: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T06:16:48.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:48 smithi037 ceph-mon[31974]: pgmap v110: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T06:16:48.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:48 smithi066 ceph-mon[36504]: pgmap v110: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T06:16:49.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:49 smithi037 ceph-mon[31974]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T06:16:49.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:49 smithi037 ceph-mon[31974]: Cluster is now healthy 2024-03-26T06:16:49.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:49 smithi066 ceph-mon[36504]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T06:16:49.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:49 smithi066 ceph-mon[36504]: Cluster is now healthy 2024-03-26T06:16:49.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:49 smithi149 ceph-mon[36681]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T06:16:49.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:49 smithi149 ceph-mon[36681]: Cluster is now healthy 2024-03-26T06:16:50.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:50 smithi066 ceph-mon[36504]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T06:16:50.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:50 smithi037 ceph-mon[31974]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T06:16:50.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:50 smithi149 ceph-mon[36681]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T06:16:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:52 smithi037 ceph-mon[31974]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-26T06:16:52.702 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:52 smithi066 ceph-mon[36504]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-26T06:16:52.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:52 smithi149 ceph-mon[36681]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-26T06:16:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:54 smithi037 ceph-mon[31974]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T06:16:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T06:16:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:54.711 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:54 smithi066 ceph-mon[36504]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T06:16:54.711 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T06:16:54.711 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:54.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:54 smithi149 ceph-mon[36681]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T06:16:54.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T06:16:54.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:16:55.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:55 smithi037 ceph-mon[31974]: Deploying daemon osd.4 on smithi066 2024-03-26T06:16:55.769 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:55 smithi066 ceph-mon[36504]: Deploying daemon osd.4 on smithi066 2024-03-26T06:16:55.810 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:55 smithi149 ceph-mon[36681]: Deploying daemon osd.4 on smithi066 2024-03-26T06:16:56.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:56 smithi037 ceph-mon[31974]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-26T06:16:56.746 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:56 smithi066 ceph-mon[36504]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-26T06:16:56.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:56 smithi149 ceph-mon[36681]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-26T06:16:58.533 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:16:58 smithi066 ceph-mon[36504]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-26T06:16:58.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:16:58 smithi037 ceph-mon[31974]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-26T06:16:58.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:16:58 smithi149 ceph-mon[36681]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-26T06:17:00.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:00 smithi066 ceph-mon[36504]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T06:17:00.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:00.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:00.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:00.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:00 smithi149 ceph-mon[36681]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T06:17:00.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:00.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:00.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:00.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:00 smithi037 ceph-mon[31974]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T06:17:00.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:00.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:00.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:02.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:02 smithi066 ceph-mon[36504]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:17:02.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:02 smithi149 ceph-mon[36681]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:17:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:02 smithi037 ceph-mon[31974]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:17:03.335 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 4 on host 'smithi066' 2024-03-26T06:17:03.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.334+0000 7f224cff9700 1 -- 172.21.15.66:0/2448197344 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2250062220 con 0x7f22380a2890 2024-03-26T06:17:03.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.336+0000 7f2255fc0700 1 -- 172.21.15.66:0/2448197344 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22380a2890 msgr2=0x7f22380a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:03.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.336+0000 7f2255fc0700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22380a2890 0x7f22380a4d50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2250101d70 tx=0x7f2240009260 comp rx=0 tx=0).stop 2024-03-26T06:17:03.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.336+0000 7f2255fc0700 1 -- 172.21.15.66:0/2448197344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22501028b0 msgr2=0x7f225019f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:03.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.336+0000 7f2255fc0700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22501028b0 0x7f225019f3d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f224800ada0 tx=0x7f224800c330 comp rx=0 tx=0).stop 2024-03-26T06:17:03.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.336+0000 7f2255fc0700 1 -- 172.21.15.66:0/2448197344 shutdown_connections 2024-03-26T06:17:03.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.336+0000 7f2255fc0700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22380a2890 0x7f22380a4d50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:03.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.337+0000 7f2255fc0700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22501028b0 0x7f225019f3d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:03.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.337+0000 7f2255fc0700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2250100d10 0x7f225006a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:03.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.337+0000 7f2255fc0700 1 --2- 172.21.15.66:0/2448197344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2250101f10 0x7f225019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:03.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.337+0000 7f2255fc0700 1 -- 172.21.15.66:0/2448197344 >> 172.21.15.66:0/2448197344 conn(0x7f22500fc2b0 msgr2=0x7f2250105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:17:03.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.337+0000 7f2255fc0700 1 -- 172.21.15.66:0/2448197344 shutdown_connections 2024-03-26T06:17:03.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:03.337+0000 7f2255fc0700 1 -- 172.21.15.66:0/2448197344 wait complete. 2024-03-26T06:17:03.575 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:03 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.576 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:03 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.576 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:03 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:03.576 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:03 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:03.576 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:03 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.576 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:03 smithi066 ceph-mon[36504]: from='osd.4 [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T06:17:03.576 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:03 smithi066 ceph-mon[36504]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T06:17:03.576 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:03 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.576 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:03 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.576 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:03 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:03.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:03 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:03 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:03 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:03.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:03 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:03.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:03 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:03 smithi037 ceph-mon[31974]: from='osd.4 [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T06:17:03.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:03 smithi037 ceph-mon[31974]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T06:17:03.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:03 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:03 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:03 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:03.885 DEBUG:teuthology.orchestra.run.smithi066:osd.4> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.4.service 2024-03-26T06:17:03.888 INFO:tasks.cephadm:Deploying osd.5 on smithi149 with /dev/vg_nvme/lv_3... 2024-03-26T06:17:03.888 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T06:17:03.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:03 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:03 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:03 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:03.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:03 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:03.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:03 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:03 smithi149 ceph-mon[36681]: from='osd.4 [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T06:17:03.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:03 smithi149 ceph-mon[36681]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T06:17:03.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:03 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:03 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:03.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:03 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:04.135 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:17:04.550 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:04 smithi149 ceph-mon[36681]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:17:04.550 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:04 smithi149 ceph-mon[36681]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T06:17:04.551 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:04 smithi149 ceph-mon[36681]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T06:17:04.551 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:04.551 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:04 smithi149 ceph-mon[36681]: from='osd.4 [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:17:04.551 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:04 smithi149 ceph-mon[36681]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:17:04.596 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:04 smithi066 ceph-mon[36504]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:17:04.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:04 smithi066 ceph-mon[36504]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T06:17:04.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:04 smithi066 ceph-mon[36504]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T06:17:04.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:04.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:04 smithi066 ceph-mon[36504]: from='osd.4 [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:17:04.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:04 smithi066 ceph-mon[36504]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:17:04.597 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:17:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4[48135]: 2024-03-26T06:17:04.436+0000 7f1688132700 -1 osd.4 0 waiting for initial osdmap 2024-03-26T06:17:04.597 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:17:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4[48135]: 2024-03-26T06:17:04.445+0000 7f1684726700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:17:04.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:04 smithi037 ceph-mon[31974]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:17:04.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:04 smithi037 ceph-mon[31974]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T06:17:04.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:04 smithi037 ceph-mon[31974]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T06:17:04.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:04.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:04 smithi037 ceph-mon[31974]: from='osd.4 [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:17:04.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:04 smithi037 ceph-mon[31974]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:17:05.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:05 smithi149 ceph-mon[36681]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:17:05.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:05 smithi149 ceph-mon[36681]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T06:17:05.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:05.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:05.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:05 smithi037 ceph-mon[31974]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:17:05.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:05 smithi037 ceph-mon[31974]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T06:17:05.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:05.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:05.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:05 smithi066 ceph-mon[36504]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:17:05.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:05 smithi066 ceph-mon[36504]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T06:17:05.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:05.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:05.870 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T06:17:05.870 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T06:17:05.870 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-26T06:17:05.871 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-26T06:17:05.871 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011377 s, 922 MB/s 2024-03-26T06:17:05.871 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-26T06:17:06.495 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-03-26T06:17:06.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:06 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:17:06.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:06 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:17:06.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:06 smithi149 ceph-mon[36681]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:17:06.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:06 smithi149 ceph-mon[36681]: osd.4 [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514] boot 2024-03-26T06:17:06.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:06 smithi149 ceph-mon[36681]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T06:17:06.562 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:06 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:06.656 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:06 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:17:06.656 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:06 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:17:06.656 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:06 smithi066 ceph-mon[36504]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:17:06.656 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:06 smithi066 ceph-mon[36504]: osd.4 [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514] boot 2024-03-26T06:17:06.656 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:06 smithi066 ceph-mon[36504]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T06:17:06.656 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:06 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:06.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:06 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:17:06.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:06 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:17:06.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:06 smithi037 ceph-mon[31974]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-26T06:17:06.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:06 smithi037 ceph-mon[31974]: osd.4 [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514] boot 2024-03-26T06:17:06.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:06 smithi037 ceph-mon[31974]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T06:17:06.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:06 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:17:06.782 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:17:07.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:07 smithi149 ceph-mon[36681]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T06:17:07.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:07.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:07.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:07.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:07.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:07.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:07.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:07.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:07 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:07.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:07 smithi066 ceph-mon[36504]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T06:17:07.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:07.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:07.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:07.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:07.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:07.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:07.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:07.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:07 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:07.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:07 smithi037 ceph-mon[31974]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T06:17:07.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:07.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:07.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:07.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:07.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:07.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:07.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:07.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:07 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:08.185 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 -- 172.21.15.149:0/1562931082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ed01022f0 msgr2=0x7f2ed0102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:08.185 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 --2- 172.21.15.149:0/1562931082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ed01022f0 0x7f2ed0102710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0099c0 tx=0x7f2ecc009cd0 comp rx=0 tx=0).stop 2024-03-26T06:17:08.185 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 -- 172.21.15.149:0/1562931082 shutdown_connections 2024-03-26T06:17:08.185 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 --2- 172.21.15.149:0/1562931082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed01019d0 0x7f2ed0109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:08.185 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 --2- 172.21.15.149:0/1562931082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ed01022f0 0x7f2ed0102710 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:08.185 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 --2- 172.21.15.149:0/1562931082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ed0101030 0x7f2ed0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:08.185 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 -- 172.21.15.149:0/1562931082 >> 172.21.15.149:0/1562931082 conn(0x7f2ed00fba80 msgr2=0x7f2ed00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 -- 172.21.15.149:0/1562931082 shutdown_connections 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 -- 172.21.15.149:0/1562931082 wait complete. 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 Processor -- start 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 -- start start 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ed0101030 0x7f2ed0198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.183+0000 7f2ed7742700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ed01019d0 0x7f2ed0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed7742700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed01022f0 0x7f2ed019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed7742700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed010da40 con 0x7f2ed01019d0 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed7742700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed010d8c0 con 0x7f2ed01022f0 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed7742700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed010dbc0 con 0x7f2ed0101030 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed54de700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ed0101030 0x7f2ed0198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:08.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed54de700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ed0101030 0x7f2ed0198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:46114/0 (socket says 172.21.15.149:46114) 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed54de700 1 -- 172.21.15.149:0/2206832754 learned_addr learned my addr 172.21.15.149:0/2206832754 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed4cdd700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ed01019d0 0x7f2ed0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed5cdf700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed01022f0 0x7f2ed019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed54de700 1 -- 172.21.15.149:0/2206832754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed01022f0 msgr2=0x7f2ed019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed54de700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed01022f0 0x7f2ed019f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed54de700 1 -- 172.21.15.149:0/2206832754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ed01019d0 msgr2=0x7f2ed0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed54de700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ed01019d0 0x7f2ed0198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed54de700 1 -- 172.21.15.149:0/2206832754 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed019fc80 con 0x7f2ed0101030 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed54de700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ed0101030 0x7f2ed0198020 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00f710 tx=0x7f2ecc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ec67fc700 1 -- 172.21.15.149:0/2206832754 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc009e60 con 0x7f2ed0101030 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed7742700 1 -- 172.21.15.149:0/2206832754 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed019ff10 con 0x7f2ed0101030 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ec67fc700 1 -- 172.21.15.149:0/2206832754 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc003e40 con 0x7f2ed0101030 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ec67fc700 1 -- 172.21.15.149:0/2206832754 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc005670 con 0x7f2ed0101030 2024-03-26T06:17:08.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.184+0000 7f2ed7742700 1 -- 172.21.15.149:0/2206832754 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed01a0360 con 0x7f2ed0101030 2024-03-26T06:17:08.188 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.185+0000 7f2ed7742700 1 -- 172.21.15.149:0/2206832754 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed0068aa0 con 0x7f2ed0101030 2024-03-26T06:17:08.188 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.186+0000 7f2ec67fc700 1 -- 172.21.15.149:0/2206832754 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2ecc00fcd0 con 0x7f2ed0101030 2024-03-26T06:17:08.188 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.187+0000 7f2ec67fc700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ebc0a2890 0x7f2ebc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:08.189 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.187+0000 7f2ec67fc700 1 -- 172.21.15.149:0/2206832754 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f2ecc0ce7b0 con 0x7f2ed0101030 2024-03-26T06:17:08.189 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.187+0000 7f2ed4cdd700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ebc0a2890 0x7f2ebc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:08.189 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.187+0000 7f2ed4cdd700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ebc0a2890 0x7f2ebc0a4d50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009c60 tx=0x7f2ec0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:17:08.192 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.192+0000 7f2ec67fc700 1 -- 172.21.15.149:0/2206832754 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2ecc013070 con 0x7f2ed0101030 2024-03-26T06:17:08.399 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:08.399+0000 7f2ed7742700 1 -- 172.21.15.149:0/2206832754 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ed0062220 con 0x7f2ebc0a2890 2024-03-26T06:17:08.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:08 smithi149 ceph-mon[36681]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:08.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:08 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi066 2024-03-26T06:17:08.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:08 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi066 to 8540M 2024-03-26T06:17:08.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:08 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:17:08.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:08 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:17:08.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:08 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:08 smithi066 ceph-mon[36504]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:08 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi066 2024-03-26T06:17:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:08 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi066 to 8540M 2024-03-26T06:17:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:08 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:17:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:08 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:17:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:08 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:08.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:08 smithi037 ceph-mon[31974]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:08.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:08 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi066 2024-03-26T06:17:08.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:08 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi066 to 8540M 2024-03-26T06:17:08.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:08 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:17:08.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:08 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:17:08.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:08 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:09.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:09 smithi149 ceph-mon[36681]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:17:09.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:09 smithi066 ceph-mon[36504]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:17:09.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:09 smithi037 ceph-mon[31974]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:17:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:10 smithi149 ceph-mon[36681]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:10.796 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:10 smithi066 ceph-mon[36504]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:10.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:10 smithi037 ceph-mon[31974]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:11.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:11 smithi149 ceph-mon[36681]: from='client.? 172.21.15.149:0/858220737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2ade669-ee83-41ad-957b-ea5b88768477"}]: dispatch 2024-03-26T06:17:11.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:11 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2ade669-ee83-41ad-957b-ea5b88768477"}]: dispatch 2024-03-26T06:17:11.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:11 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2ade669-ee83-41ad-957b-ea5b88768477"}]': finished 2024-03-26T06:17:11.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:11 smithi149 ceph-mon[36681]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T06:17:11.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:11 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:11.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:11 smithi066 ceph-mon[36504]: from='client.? 172.21.15.149:0/858220737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2ade669-ee83-41ad-957b-ea5b88768477"}]: dispatch 2024-03-26T06:17:11.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:11 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2ade669-ee83-41ad-957b-ea5b88768477"}]: dispatch 2024-03-26T06:17:11.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:11 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2ade669-ee83-41ad-957b-ea5b88768477"}]': finished 2024-03-26T06:17:11.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:11 smithi066 ceph-mon[36504]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T06:17:11.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:11 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:11.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:11 smithi037 ceph-mon[31974]: from='client.? 172.21.15.149:0/858220737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2ade669-ee83-41ad-957b-ea5b88768477"}]: dispatch 2024-03-26T06:17:11.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:11 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2ade669-ee83-41ad-957b-ea5b88768477"}]: dispatch 2024-03-26T06:17:11.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:11 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2ade669-ee83-41ad-957b-ea5b88768477"}]': finished 2024-03-26T06:17:11.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:11 smithi037 ceph-mon[31974]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T06:17:11.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:11 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:12.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:12 smithi066 ceph-mon[36504]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:12.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:12 smithi066 ceph-mon[36504]: from='client.? 172.21.15.149:0/3259125754' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:17:12.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:12 smithi149 ceph-mon[36681]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:12.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:12 smithi149 ceph-mon[36681]: from='client.? 172.21.15.149:0/3259125754' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:17:12.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:12 smithi037 ceph-mon[31974]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:12.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:12 smithi037 ceph-mon[31974]: from='client.? 172.21.15.149:0/3259125754' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:17:14.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:14 smithi037 ceph-mon[31974]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:14.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:14 smithi066 ceph-mon[36504]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:14.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:14 smithi149 ceph-mon[36681]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:16.621 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:16 smithi149 ceph-mon[36681]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:16.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:16 smithi066 ceph-mon[36504]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:16.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:16 smithi037 ceph-mon[31974]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:18.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:18 smithi149 ceph-mon[36681]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:18.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:18 smithi066 ceph-mon[36504]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:18 smithi037 ceph-mon[31974]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:19.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T06:17:19.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:19.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:19 smithi149 ceph-mon[36681]: Deploying daemon osd.5 on smithi149 2024-03-26T06:17:19.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T06:17:19.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:19.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:19 smithi066 ceph-mon[36504]: Deploying daemon osd.5 on smithi149 2024-03-26T06:17:19.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T06:17:19.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:19.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:19 smithi037 ceph-mon[31974]: Deploying daemon osd.5 on smithi149 2024-03-26T06:17:20.663 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:20 smithi149 ceph-mon[36681]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:20.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:20 smithi066 ceph-mon[36504]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:20.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:20 smithi037 ceph-mon[31974]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:22.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:22 smithi149 ceph-mon[36681]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:22.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:22 smithi066 ceph-mon[36504]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:22.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:22 smithi037 ceph-mon[31974]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:24.631 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:24 smithi149 ceph-mon[36681]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:24.631 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:24.631 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:24.632 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:24.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:24 smithi066 ceph-mon[36504]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:24.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:24.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:24.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:24.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:24 smithi037 ceph-mon[31974]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:26.753 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:26 smithi149 ceph-mon[36681]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:26.753 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:26 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:26.754 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:26 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:26.754 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:26 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:26.754 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:26 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:26.754 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:26 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:26.846 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:26 smithi066 ceph-mon[36504]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:26 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:26 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:26 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:26 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:26 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:26 smithi037 ceph-mon[31974]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:27.767 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 5 on host 'smithi149' 2024-03-26T06:17:27.767 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.766+0000 7f2ec67fc700 1 -- 172.21.15.149:0/2206832754 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2ed0062220 con 0x7f2ebc0a2890 2024-03-26T06:17:27.769 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 -- 172.21.15.149:0/2206832754 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ebc0a2890 msgr2=0x7f2ebc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:27.769 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ebc0a2890 0x7f2ebc0a4d50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009c60 tx=0x7f2ec0009260 comp rx=0 tx=0).stop 2024-03-26T06:17:27.769 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 -- 172.21.15.149:0/2206832754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ed0101030 msgr2=0x7f2ed0198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:27.770 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ed0101030 0x7f2ed0198020 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00f710 tx=0x7f2ecc009580 comp rx=0 tx=0).stop 2024-03-26T06:17:27.770 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 -- 172.21.15.149:0/2206832754 shutdown_connections 2024-03-26T06:17:27.770 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ebc0a2890 0x7f2ebc0a4d50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:27.770 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed01022f0 0x7f2ed019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:27.770 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ed0101030 0x7f2ed0198020 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:27.770 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 --2- 172.21.15.149:0/2206832754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ed01019d0 0x7f2ed0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:27.770 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 -- 172.21.15.149:0/2206832754 >> 172.21.15.149:0/2206832754 conn(0x7f2ed00fba80 msgr2=0x7f2ed01052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:17:27.770 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 -- 172.21.15.149:0/2206832754 shutdown_connections 2024-03-26T06:17:27.770 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:17:27.769+0000 7f2ed7742700 1 -- 172.21.15.149:0/2206832754 wait complete. 2024-03-26T06:17:28.441 DEBUG:teuthology.orchestra.run.smithi149:osd.5> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.5.service 2024-03-26T06:17:28.483 INFO:tasks.cephadm:Deploying osd.6 on smithi037 with /dev/vg_nvme/lv_2... 2024-03-26T06:17:28.484 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T06:17:28.682 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:28 smithi149 ceph-mon[36681]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:28.682 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:28 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:28.682 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:28 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:28.683 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:28 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:28.683 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:28 smithi149 ceph-mon[36681]: from='osd.5 [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T06:17:28.683 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:28 smithi149 ceph-mon[36681]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T06:17:28.736 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:17:28.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:28 smithi066 ceph-mon[36504]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:28.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:28 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:28.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:28 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:28.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:28 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:28.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:28 smithi066 ceph-mon[36504]: from='osd.5 [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T06:17:28.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:28 smithi066 ceph-mon[36504]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T06:17:28.858 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:28 smithi037 ceph-mon[31974]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:28.858 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:28 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:28.858 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:28 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:28.858 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:28 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:28.858 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:28 smithi037 ceph-mon[31974]: from='osd.5 [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T06:17:28.858 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:28 smithi037 ceph-mon[31974]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T06:17:29.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:29 smithi149 ceph-mon[36681]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:29.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:29 smithi149 ceph-mon[36681]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T06:17:29.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:29 smithi149 ceph-mon[36681]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T06:17:29.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:29 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:29.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:29 smithi149 ceph-mon[36681]: from='osd.5 [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:17:29.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:29 smithi149 ceph-mon[36681]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:17:29.910 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:17:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:17:29.739+0000 7fa5fc1bb700 -1 osd.5 0 waiting for initial osdmap 2024-03-26T06:17:29.910 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:17:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:17:29.745+0000 7fa5f87af700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:17:30.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:29 smithi066 ceph-mon[36504]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:30.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:29 smithi066 ceph-mon[36504]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T06:17:30.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:29 smithi066 ceph-mon[36504]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T06:17:30.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:29 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:30.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:29 smithi066 ceph-mon[36504]: from='osd.5 [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:17:30.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:29 smithi066 ceph-mon[36504]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:17:30.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:29 smithi037 ceph-mon[31974]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:30.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:29 smithi037 ceph-mon[31974]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T06:17:30.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:29 smithi037 ceph-mon[31974]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T06:17:30.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:29 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:30.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:29 smithi037 ceph-mon[31974]: from='osd.5 [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:17:30.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:29 smithi037 ceph-mon[31974]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:17:30.656 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T06:17:30.656 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T06:17:30.657 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-03-26T06:17:30.657 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-03-26T06:17:30.657 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112123 s, 935 MB/s 2024-03-26T06:17:30.657 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-03-26T06:17:30.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:30 smithi066 ceph-mon[36504]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:17:30.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:30 smithi066 ceph-mon[36504]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T06:17:30.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:30.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:31.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:30 smithi037 ceph-mon[31974]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:17:31.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:30 smithi037 ceph-mon[31974]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T06:17:31.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:31.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:31.128 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:30 smithi149 ceph-mon[36681]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:17:31.128 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:30 smithi149 ceph-mon[36681]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T06:17:31.128 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:31.128 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:31.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi037:vg_nvme/lv_2 2024-03-26T06:17:31.589 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:17:31.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:17:31.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:17:31.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:31.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:31.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: osd.5 [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638] boot 2024-03-26T06:17:31.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: osdmap e38: 6 total, 6 up, 6 in 2024-03-26T06:17:31.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:31.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi149 2024-03-26T06:17:31.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:31.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:31.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:31.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:31.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi149 to 2903M 2024-03-26T06:17:31.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:31.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:31.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:31.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:32.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:17:32.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:17:32.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:32.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:32.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: osd.5 [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638] boot 2024-03-26T06:17:32.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: osdmap e38: 6 total, 6 up, 6 in 2024-03-26T06:17:32.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:32.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi149 2024-03-26T06:17:32.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:32.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:32.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:32.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:32.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi149 to 2903M 2024-03-26T06:17:32.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:32.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:32.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:32.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:32.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:17:32.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:17:32.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-26T06:17:32.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:32.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: osd.5 [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638] boot 2024-03-26T06:17:32.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: osdmap e38: 6 total, 6 up, 6 in 2024-03-26T06:17:32.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:17:32.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi149 2024-03-26T06:17:32.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:32.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:32.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:32.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:32.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi149 to 2903M 2024-03-26T06:17:32.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:32.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:32.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:32.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:33.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:32 smithi037 ceph-mon[31974]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T06:17:33.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:32 smithi066 ceph-mon[36504]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T06:17:33.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:32 smithi149 ceph-mon[36681]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T06:17:33.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.313+0000 7fb5eee40700 1 -- 172.21.15.37:0/3525019506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8101f10 msgr2=0x7fb5e8102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:33.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.313+0000 7fb5eee40700 1 --2- 172.21.15.37:0/3525019506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8101f10 0x7fb5e8102370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0099c0 tx=0x7fb5dc009cd0 comp rx=0 tx=0).stop 2024-03-26T06:17:33.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.313+0000 7fb5eee40700 1 -- 172.21.15.37:0/3525019506 shutdown_connections 2024-03-26T06:17:33.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.313+0000 7fb5eee40700 1 --2- 172.21.15.37:0/3525019506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e81028b0 0x7fb5e8109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:33.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.313+0000 7fb5eee40700 1 --2- 172.21.15.37:0/3525019506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5e8100d10 0x7fb5e8101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:33.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.313+0000 7fb5eee40700 1 --2- 172.21.15.37:0/3525019506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8101f10 0x7fb5e8102370 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:33.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.313+0000 7fb5eee40700 1 -- 172.21.15.37:0/3525019506 >> 172.21.15.37:0/3525019506 conn(0x7fb5e80fc2d0 msgr2=0x7fb5e80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:17:33.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.313+0000 7fb5eee40700 1 -- 172.21.15.37:0/3525019506 shutdown_connections 2024-03-26T06:17:33.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.313+0000 7fb5eee40700 1 -- 172.21.15.37:0/3525019506 wait complete. 2024-03-26T06:17:33.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5eee40700 1 Processor -- start 2024-03-26T06:17:33.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5eee40700 1 -- start start 2024-03-26T06:17:33.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5eee40700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5e8100d10 0x7fb5e8198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5eee40700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e8101f10 0x7fb5e8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5eee40700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e81028b0 0x7fb5e819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5eee40700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e806af50 con 0x7fb5e81028b0 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5eee40700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e806add0 con 0x7fb5e8101f10 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5eee40700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e806b0d0 con 0x7fb5e8100d10 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5ed3dd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e81028b0 0x7fb5e819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5ed3dd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e81028b0 0x7fb5e819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57350/0 (socket says 172.21.15.37:57350) 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5ed3dd700 1 -- 172.21.15.37:0/3189489335 learned_addr learned my addr 172.21.15.37:0/3189489335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5e7fff700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e8101f10 0x7fb5e8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5ecbdc700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5e8100d10 0x7fb5e8198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5ed3dd700 1 -- 172.21.15.37:0/3189489335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5e8100d10 msgr2=0x7fb5e8198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:33.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5ed3dd700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5e8100d10 0x7fb5e8198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:33.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5ed3dd700 1 -- 172.21.15.37:0/3189489335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e8101f10 msgr2=0x7fb5e8198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:33.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5ed3dd700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e8101f10 0x7fb5e8198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:33.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5ed3dd700 1 -- 172.21.15.37:0/3189489335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e819fc70 con 0x7fb5e81028b0 2024-03-26T06:17:33.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5ecbdc700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5e8100d10 0x7fb5e8198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:17:33.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.314+0000 7fb5ed3dd700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e81028b0 0x7fb5e819f5d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0007f10 tx=0x7fb5e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:17:33.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.315+0000 7fb5e5ffb700 1 -- 172.21.15.37:0/3189489335 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0004bb0 con 0x7fb5e81028b0 2024-03-26T06:17:33.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.315+0000 7fb5eee40700 1 -- 172.21.15.37:0/3189489335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e819ff60 con 0x7fb5e81028b0 2024-03-26T06:17:33.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.315+0000 7fb5e5ffb700 1 -- 172.21.15.37:0/3189489335 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5e0004d50 con 0x7fb5e81028b0 2024-03-26T06:17:33.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.315+0000 7fb5e5ffb700 1 -- 172.21.15.37:0/3189489335 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0005a60 con 0x7fb5e81028b0 2024-03-26T06:17:33.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.315+0000 7fb5eee40700 1 -- 172.21.15.37:0/3189489335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e81a03c0 con 0x7fb5e81028b0 2024-03-26T06:17:33.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.316+0000 7fb5eee40700 1 -- 172.21.15.37:0/3189489335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e8068aa0 con 0x7fb5e81028b0 2024-03-26T06:17:33.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.317+0000 7fb5e5ffb700 1 -- 172.21.15.37:0/3189489335 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb5e001a020 con 0x7fb5e81028b0 2024-03-26T06:17:33.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.317+0000 7fb5e5ffb700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb5d00a2890 0x7fb5d00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:33.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.317+0000 7fb5ecbdc700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb5d00a2890 0x7fb5d00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:33.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.317+0000 7fb5e5ffb700 1 -- 172.21.15.37:0/3189489335 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fb5e00cd360 con 0x7fb5e81028b0 2024-03-26T06:17:33.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.318+0000 7fb5ecbdc700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb5d00a2890 0x7fb5d00a4d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8101d70 tx=0x7fb5d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:17:33.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.322+0000 7fb5e5ffb700 1 -- 172.21.15.37:0/3189489335 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb5e0010070 con 0x7fb5e81028b0 2024-03-26T06:17:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:33.537+0000 7fb5eee40700 1 -- 172.21.15.37:0/3189489335 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5e8062220 con 0x7fb5d00a2890 2024-03-26T06:17:34.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:33 smithi066 ceph-mon[36504]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:34.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:33 smithi066 ceph-mon[36504]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T06:17:34.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:33 smithi066 ceph-mon[36504]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:17:34.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:33 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:17:34.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:33 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:17:34.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:33 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:34.155 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:33 smithi037 ceph-mon[31974]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:34.155 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:33 smithi037 ceph-mon[31974]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T06:17:34.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:33 smithi037 ceph-mon[31974]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:17:34.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:33 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:17:34.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:33 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:17:34.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:33 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:34.163 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:33 smithi149 ceph-mon[36681]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:34.164 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:33 smithi149 ceph-mon[36681]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T06:17:34.164 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:33 smithi149 ceph-mon[36681]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:17:34.164 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:17:34.164 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:17:34.164 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:35.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:34 smithi066 ceph-mon[36504]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T06:17:35.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:34 smithi066 ceph-mon[36504]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T06:17:35.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:34 smithi037 ceph-mon[31974]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T06:17:35.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:34 smithi037 ceph-mon[31974]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T06:17:35.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:34 smithi149 ceph-mon[36681]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T06:17:35.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:34 smithi149 ceph-mon[36681]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T06:17:36.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:35 smithi066 ceph-mon[36504]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T06:17:36.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:35 smithi149 ceph-mon[36681]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T06:17:36.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:35 smithi037 ceph-mon[31974]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T06:17:36.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:36 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2554843411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dd601fe-399f-4ba5-87ab-d602a0930403"}]: dispatch 2024-03-26T06:17:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:36 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2554843411' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dd601fe-399f-4ba5-87ab-d602a0930403"}]': finished 2024-03-26T06:17:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:36 smithi037 ceph-mon[31974]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T06:17:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:36 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:37.033 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:36 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2554843411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dd601fe-399f-4ba5-87ab-d602a0930403"}]: dispatch 2024-03-26T06:17:37.033 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:36 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2554843411' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dd601fe-399f-4ba5-87ab-d602a0930403"}]': finished 2024-03-26T06:17:37.033 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:36 smithi149 ceph-mon[36681]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T06:17:37.033 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:36 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:37.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:36 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2554843411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dd601fe-399f-4ba5-87ab-d602a0930403"}]: dispatch 2024-03-26T06:17:37.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:36 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2554843411' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dd601fe-399f-4ba5-87ab-d602a0930403"}]': finished 2024-03-26T06:17:37.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:36 smithi066 ceph-mon[36504]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T06:17:37.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:36 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:37.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:37 smithi149 ceph-mon[36681]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T06:17:37.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:37 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/4031508661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:17:38.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:37 smithi066 ceph-mon[36504]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T06:17:38.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:37 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/4031508661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:17:38.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:37 smithi037 ceph-mon[31974]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T06:17:38.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:37 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/4031508661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:17:40.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:39 smithi066 ceph-mon[36504]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-26T06:17:40.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:39 smithi149 ceph-mon[36681]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-26T06:17:40.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:39 smithi037 ceph-mon[31974]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-26T06:17:41.843 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:41 smithi037 ceph-mon[31974]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:42.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:41 smithi066 ceph-mon[36504]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:42.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:41 smithi149 ceph-mon[36681]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:43.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:43 smithi037 ceph-mon[31974]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:44.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:43 smithi066 ceph-mon[36504]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:44.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:43 smithi149 ceph-mon[36681]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:45.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T06:17:45.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:45.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:44 smithi066 ceph-mon[36504]: Deploying daemon osd.6 on smithi037 2024-03-26T06:17:45.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T06:17:45.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:45.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:44 smithi149 ceph-mon[36681]: Deploying daemon osd.6 on smithi037 2024-03-26T06:17:45.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T06:17:45.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:45.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:44 smithi037 ceph-mon[31974]: Deploying daemon osd.6 on smithi037 2024-03-26T06:17:45.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:45 smithi037 ceph-mon[31974]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:46.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:45 smithi066 ceph-mon[36504]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:46.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:45 smithi149 ceph-mon[36681]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:48.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:47 smithi037 ceph-mon[31974]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:48.096 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:47 smithi066 ceph-mon[36504]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:48.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:47 smithi149 ceph-mon[36681]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:50.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:50 smithi037 ceph-mon[31974]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:50.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:50.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:50.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:50 smithi066 ceph-mon[36504]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:51.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:50 smithi149 ceph-mon[36681]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:51.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:51.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:51.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:52.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:52 smithi037 ceph-mon[31974]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:52.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:52.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:52.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:52.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:52.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:53.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:52 smithi066 ceph-mon[36504]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:53.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:53.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:53.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:53.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:53.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:53.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:52 smithi149 ceph-mon[36681]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:53.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:53.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:53.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:53.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:53.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.395+0000 7fb5e5ffb700 1 -- 172.21.15.37:0/3189489335 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb5e8062220 con 0x7fb5d00a2890 2024-03-26T06:17:53.396 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 6 on host 'smithi037' 2024-03-26T06:17:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 -- 172.21.15.37:0/3189489335 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb5d00a2890 msgr2=0x7fb5d00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb5d00a2890 0x7fb5d00a4d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8101d70 tx=0x7fb5d8009260 comp rx=0 tx=0).stop 2024-03-26T06:17:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 -- 172.21.15.37:0/3189489335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e81028b0 msgr2=0x7fb5e819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e81028b0 0x7fb5e819f5d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0007f10 tx=0x7fb5e000d3b0 comp rx=0 tx=0).stop 2024-03-26T06:17:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 -- 172.21.15.37:0/3189489335 shutdown_connections 2024-03-26T06:17:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e81028b0 0x7fb5e819f5d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb5d00a2890 0x7fb5d00a4d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5e8100d10 0x7fb5e8198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 --2- 172.21.15.37:0/3189489335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5e8101f10 0x7fb5e8198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 -- 172.21.15.37:0/3189489335 >> 172.21.15.37:0/3189489335 conn(0x7fb5e80fc2d0 msgr2=0x7fb5e8105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:17:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 -- 172.21.15.37:0/3189489335 shutdown_connections 2024-03-26T06:17:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:17:53.398+0000 7fb5eee40700 1 -- 172.21.15.37:0/3189489335 wait complete. 2024-03-26T06:17:54.062 DEBUG:teuthology.orchestra.run.smithi037:osd.6> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.6.service 2024-03-26T06:17:54.065 INFO:tasks.cephadm:Deploying osd.7 on smithi066 with /dev/vg_nvme/lv_2... 2024-03-26T06:17:54.066 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T06:17:54.326 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:17:54.649 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:54 smithi066 ceph-mon[36504]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:54.649 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:54.649 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:54.649 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:54 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:54.649 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:54 smithi066 ceph-mon[36504]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T06:17:54.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:54 smithi149 ceph-mon[36681]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:54.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:54.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:54.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:54 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:54.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:54 smithi149 ceph-mon[36681]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T06:17:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:54 smithi037 ceph-mon[31974]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:54 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:17:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:54 smithi037 ceph-mon[31974]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T06:17:55.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:55 smithi037 ceph-mon[31974]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T06:17:55.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:55 smithi037 ceph-mon[31974]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T06:17:55.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:55 smithi037 ceph-mon[31974]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:17:55.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:55 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:55.646 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:17:55 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:17:55.394+0000 7f75253e3700 -1 osd.6 0 waiting for initial osdmap 2024-03-26T06:17:55.647 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:17:55 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:17:55.405+0000 7f75219d7700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:17:55.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:55 smithi149 ceph-mon[36681]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T06:17:55.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:55 smithi149 ceph-mon[36681]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T06:17:55.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:55 smithi149 ceph-mon[36681]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:17:55.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:55 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:55.740 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:55 smithi066 ceph-mon[36504]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T06:17:55.740 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:55 smithi066 ceph-mon[36504]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T06:17:55.740 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:55 smithi066 ceph-mon[36504]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:17:55.740 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:55 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:56.526 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T06:17:56.526 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T06:17:56.526 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-03-26T06:17:56.526 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-03-26T06:17:56.526 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114914 s, 912 MB/s 2024-03-26T06:17:56.526 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-03-26T06:17:56.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:56 smithi066 ceph-mon[36504]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:56.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:56 smithi066 ceph-mon[36504]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:17:56.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:56 smithi066 ceph-mon[36504]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T06:17:56.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:56 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:56.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:56 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:56.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:56 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:56.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:56 smithi149 ceph-mon[36681]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:56.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:56 smithi149 ceph-mon[36681]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:17:56.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:56 smithi149 ceph-mon[36681]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T06:17:56.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:56 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:56.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:56 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:56.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:56 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:56.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:56 smithi037 ceph-mon[31974]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:56.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:56 smithi037 ceph-mon[31974]: from='osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:17:56.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:56 smithi037 ceph-mon[31974]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T06:17:56.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:56 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:56.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:56 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:56.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:56 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:57.213 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi066:vg_nvme/lv_2 2024-03-26T06:17:57.462 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:17:57.577 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:17:57.577 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:17:57.577 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917] boot 2024-03-26T06:17:57.577 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T06:17:57.577 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:57.577 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:57.577 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:57.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:57.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:57.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:57.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:57.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:57.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:57.578 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:57.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:17:57.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:17:57.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917] boot 2024-03-26T06:17:57.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T06:17:57.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:57.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:57.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:57.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:17:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:17:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: osd.6 [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917] boot 2024-03-26T06:17:57.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T06:17:57.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:17:57.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:57.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:57.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:57.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:57.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:17:57.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:57.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:57.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:17:57.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:17:58.670 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:58 smithi066 ceph-mon[36504]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:58.670 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:58 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi037 2024-03-26T06:17:58.670 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:58 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi037 to 5697M 2024-03-26T06:17:58.670 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:58 smithi066 ceph-mon[36504]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T06:17:58.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:58 smithi037 ceph-mon[31974]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:58.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:58 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi037 2024-03-26T06:17:58.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:58 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi037 to 5697M 2024-03-26T06:17:58.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:58 smithi037 ceph-mon[31974]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T06:17:58.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.895+0000 7fa0b5319700 1 -- 172.21.15.66:0/1866553850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b01000d0 msgr2=0x7fa0b01004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:58.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.895+0000 7fa0b5319700 1 --2- 172.21.15.66:0/1866553850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b01000d0 0x7fa0b01004f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800b0a0 tx=0x7fa0a800b3b0 comp rx=0 tx=0).stop 2024-03-26T06:17:58.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.895+0000 7fa0b5319700 1 -- 172.21.15.66:0/1866553850 shutdown_connections 2024-03-26T06:17:58.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.895+0000 7fa0b5319700 1 --2- 172.21.15.66:0/1866553850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b0103a50 0x7fa0b010a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:58.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.895+0000 7fa0b5319700 1 --2- 172.21.15.66:0/1866553850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b01000d0 0x7fa0b01004f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:58.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.895+0000 7fa0b5319700 1 --2- 172.21.15.66:0/1866553850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b01030b0 0x7fa0b0103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:58.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.895+0000 7fa0b5319700 1 -- 172.21.15.66:0/1866553850 >> 172.21.15.66:0/1866553850 conn(0x7fa0b00fba50 msgr2=0x7fa0b00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:17:58.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.895+0000 7fa0b5319700 1 -- 172.21.15.66:0/1866553850 shutdown_connections 2024-03-26T06:17:58.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.895+0000 7fa0b5319700 1 -- 172.21.15.66:0/1866553850 wait complete. 2024-03-26T06:17:58.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0b5319700 1 Processor -- start 2024-03-26T06:17:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0b5319700 1 -- start start 2024-03-26T06:17:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0b5319700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b01000d0 0x7fa0b0102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0b5319700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b01030b0 0x7fa0b0101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0b5319700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b0103a50 0x7fa0b01015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0b5319700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b010da40 con 0x7fa0b01030b0 2024-03-26T06:17:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0b5319700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b010d8c0 con 0x7fa0b0103a50 2024-03-26T06:17:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0b5319700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b010dbc0 con 0x7fa0b01000d0 2024-03-26T06:17:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0af7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b0103a50 0x7fa0b01015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0af7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b0103a50 0x7fa0b01015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43566/0 (socket says 172.21.15.66:43566) 2024-03-26T06:17:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0af7fe700 1 -- 172.21.15.66:0/4154076190 learned_addr learned my addr 172.21.15.66:0/4154076190 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-26T06:17:58.898 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0aeffd700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b01000d0 0x7fa0b0102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0ae7fc700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b01030b0 0x7fa0b0101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0af7fe700 1 -- 172.21.15.66:0/4154076190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b01000d0 msgr2=0x7fa0b0102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0af7fe700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b01000d0 0x7fa0b0102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0af7fe700 1 -- 172.21.15.66:0/4154076190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b01030b0 msgr2=0x7fa0b0101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0af7fe700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b01030b0 0x7fa0b0101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.896+0000 7fa0af7fe700 1 -- 172.21.15.66:0/4154076190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b0101e50 con 0x7fa0b0103a50 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.897+0000 7fa0af7fe700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b0103a50 0x7fa0b01015a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000b550 tx=0x7fa0a000b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.897+0000 7fa0a7fff700 1 -- 172.21.15.66:0/4154076190 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0015020 con 0x7fa0b0103a50 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.897+0000 7fa0b5319700 1 -- 172.21.15.66:0/4154076190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b01a2220 con 0x7fa0b0103a50 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.897+0000 7fa0a7fff700 1 -- 172.21.15.66:0/4154076190 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0a0011d10 con 0x7fa0b0103a50 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.897+0000 7fa0a7fff700 1 -- 172.21.15.66:0/4154076190 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000f2e0 con 0x7fa0b0103a50 2024-03-26T06:17:58.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.897+0000 7fa0b5319700 1 -- 172.21.15.66:0/4154076190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b01a2760 con 0x7fa0b0103a50 2024-03-26T06:17:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.898+0000 7fa0b5319700 1 -- 172.21.15.66:0/4154076190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b0068aa0 con 0x7fa0b0103a50 2024-03-26T06:17:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.899+0000 7fa0a7fff700 1 -- 172.21.15.66:0/4154076190 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa0a0018020 con 0x7fa0b0103a50 2024-03-26T06:17:58.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.900+0000 7fa0a7fff700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa09c0a2890 0x7fa09c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:17:58.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.900+0000 7fa0a7fff700 1 -- 172.21.15.66:0/4154076190 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fa0a00cd480 con 0x7fa0b0103a50 2024-03-26T06:17:58.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.900+0000 7fa0aeffd700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa09c0a2890 0x7fa09c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:17:58.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.901+0000 7fa0aeffd700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa09c0a2890 0x7fa09c0a4d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800b070 tx=0x7fa0a800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:17:58.904 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:58.905+0000 7fa0a7fff700 1 -- 172.21.15.66:0/4154076190 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa0a001f020 con 0x7fa0b0103a50 2024-03-26T06:17:58.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:58 smithi149 ceph-mon[36681]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-26T06:17:58.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:58 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi037 2024-03-26T06:17:58.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:58 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi037 to 5697M 2024-03-26T06:17:58.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:58 smithi149 ceph-mon[36681]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T06:17:59.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:17:59.120+0000 7fa0b5319700 1 -- 172.21.15.66:0/4154076190 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0b0062220 con 0x7fa09c0a2890 2024-03-26T06:17:59.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:59 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:17:59.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:59 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:17:59.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:17:59 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:59.751 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:59 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:17:59.751 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:59 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:17:59.751 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:17:59 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:17:59.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:59 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:17:59.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:59 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:17:59.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:17:59 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:00.626 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:00 smithi066 ceph-mon[36504]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:00.627 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:00 smithi066 ceph-mon[36504]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:18:00.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:00 smithi037 ceph-mon[31974]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:00.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:00 smithi037 ceph-mon[31974]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:18:00.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:00 smithi149 ceph-mon[36681]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:00.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:00 smithi149 ceph-mon[36681]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:18:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:02 smithi037 ceph-mon[31974]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:02 smithi037 ceph-mon[31974]: from='client.? 172.21.15.66:0/1843451326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f84e7511-f0e6-4cd0-b78f-cee1696df1b5"}]: dispatch 2024-03-26T06:18:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:02 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f84e7511-f0e6-4cd0-b78f-cee1696df1b5"}]: dispatch 2024-03-26T06:18:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:02 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f84e7511-f0e6-4cd0-b78f-cee1696df1b5"}]': finished 2024-03-26T06:18:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:02 smithi037 ceph-mon[31974]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T06:18:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:02 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:02 smithi037 ceph-mon[31974]: from='client.? 172.21.15.66:0/1395023587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:18:02.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:02 smithi066 ceph-mon[36504]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:02.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:02 smithi066 ceph-mon[36504]: from='client.? 172.21.15.66:0/1843451326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f84e7511-f0e6-4cd0-b78f-cee1696df1b5"}]: dispatch 2024-03-26T06:18:02.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:02 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f84e7511-f0e6-4cd0-b78f-cee1696df1b5"}]: dispatch 2024-03-26T06:18:02.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:02 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f84e7511-f0e6-4cd0-b78f-cee1696df1b5"}]': finished 2024-03-26T06:18:02.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:02 smithi066 ceph-mon[36504]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T06:18:02.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:02 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:02.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:02 smithi066 ceph-mon[36504]: from='client.? 172.21.15.66:0/1395023587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:18:02.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:02 smithi149 ceph-mon[36681]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:02.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:02 smithi149 ceph-mon[36681]: from='client.? 172.21.15.66:0/1843451326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f84e7511-f0e6-4cd0-b78f-cee1696df1b5"}]: dispatch 2024-03-26T06:18:02.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:02 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f84e7511-f0e6-4cd0-b78f-cee1696df1b5"}]: dispatch 2024-03-26T06:18:02.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:02 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f84e7511-f0e6-4cd0-b78f-cee1696df1b5"}]': finished 2024-03-26T06:18:02.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:02 smithi149 ceph-mon[36681]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T06:18:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:02 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:02 smithi149 ceph-mon[36681]: from='client.? 172.21.15.66:0/1395023587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:18:04.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:04 smithi037 ceph-mon[31974]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:04.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:04 smithi066 ceph-mon[36504]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:04.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:04 smithi149 ceph-mon[36681]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:06.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:06 smithi066 ceph-mon[36504]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:06.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:06 smithi149 ceph-mon[36681]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:06.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:06 smithi037 ceph-mon[31974]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:08.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:08 smithi037 ceph-mon[31974]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:08 smithi066 ceph-mon[36504]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:08.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:08 smithi149 ceph-mon[36681]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:09.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:09 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T06:18:09.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:09 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:09.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:09 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T06:18:09.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:09 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:09.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:09 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T06:18:09.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:09 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:10 smithi066 ceph-mon[36504]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:10 smithi066 ceph-mon[36504]: Deploying daemon osd.7 on smithi066 2024-03-26T06:18:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:10 smithi037 ceph-mon[31974]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:10 smithi037 ceph-mon[31974]: Deploying daemon osd.7 on smithi066 2024-03-26T06:18:10.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:10 smithi149 ceph-mon[36681]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:10.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:10 smithi149 ceph-mon[36681]: Deploying daemon osd.7 on smithi066 2024-03-26T06:18:12.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:12 smithi066 ceph-mon[36504]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:12.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:12 smithi037 ceph-mon[31974]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:12.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:12 smithi149 ceph-mon[36681]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:14.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:14 smithi066 ceph-mon[36504]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:14.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:14 smithi037 ceph-mon[31974]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:14.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:14 smithi149 ceph-mon[36681]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:16.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:15 smithi149 ceph-mon[36681]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:16.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:15 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:16.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:15 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:16.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:15 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:16.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:15 smithi037 ceph-mon[31974]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:16.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:15 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:16.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:15 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:16.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:15 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:16.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:15 smithi066 ceph-mon[36504]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:16.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:15 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:16.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:15 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:16.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:15 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:18.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:17 smithi149 ceph-mon[36681]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:18.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:17 smithi037 ceph-mon[31974]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:18.279 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:17 smithi066 ceph-mon[36504]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:18.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.336+0000 7fa0a7fff700 1 -- 172.21.15.66:0/4154076190 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa0b0062220 con 0x7fa09c0a2890 2024-03-26T06:18:18.337 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 7 on host 'smithi066' 2024-03-26T06:18:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 -- 172.21.15.66:0/4154076190 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa09c0a2890 msgr2=0x7fa09c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:18:18.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa09c0a2890 0x7fa09c0a4d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800b070 tx=0x7fa0a800b7b0 comp rx=0 tx=0).stop 2024-03-26T06:18:18.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 -- 172.21.15.66:0/4154076190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b0103a50 msgr2=0x7fa0b01015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:18:18.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b0103a50 0x7fa0b01015a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000b550 tx=0x7fa0a000b860 comp rx=0 tx=0).stop 2024-03-26T06:18:18.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 -- 172.21.15.66:0/4154076190 shutdown_connections 2024-03-26T06:18:18.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa09c0a2890 0x7fa09c0a4d50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:18.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0b0103a50 0x7fa0b01015a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:18.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b01000d0 0x7fa0b0102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:18.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 --2- 172.21.15.66:0/4154076190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b01030b0 0x7fa0b0101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:18.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 -- 172.21.15.66:0/4154076190 >> 172.21.15.66:0/4154076190 conn(0x7fa0b00fba50 msgr2=0x7fa0b0106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:18:18.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 -- 172.21.15.66:0/4154076190 shutdown_connections 2024-03-26T06:18:18.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:18:18.338+0000 7fa0b5319700 1 -- 172.21.15.66:0/4154076190 wait complete. 2024-03-26T06:18:18.918 DEBUG:teuthology.orchestra.run.smithi066:osd.7> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.7.service 2024-03-26T06:18:18.921 INFO:tasks.cephadm:Deploying osd.8 on smithi149 with /dev/vg_nvme/lv_2... 2024-03-26T06:18:18.921 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T06:18:19.169 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:18:19.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:19.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:19.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:19 smithi066 ceph-mon[36504]: from='osd.7 [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T06:18:19.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:19 smithi066 ceph-mon[36504]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T06:18:19.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:19.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:19.636 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:19.636 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:19.636 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:19 smithi149 ceph-mon[36681]: from='osd.7 [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T06:18:19.636 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:19 smithi149 ceph-mon[36681]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T06:18:19.636 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:19.636 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:19.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:19.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:19.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:19 smithi037 ceph-mon[31974]: from='osd.7 [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T06:18:19.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:19 smithi037 ceph-mon[31974]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T06:18:19.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:19.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:20.540 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:20 smithi149 ceph-mon[36681]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:20.540 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:20 smithi149 ceph-mon[36681]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T06:18:20.540 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:20 smithi149 ceph-mon[36681]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T06:18:20.540 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:20 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:20.540 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:20 smithi149 ceph-mon[36681]: from='osd.7 [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:18:20.541 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:20 smithi149 ceph-mon[36681]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:18:20.581 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:20 smithi066 ceph-mon[36504]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:20.581 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:20 smithi066 ceph-mon[36504]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T06:18:20.581 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:20 smithi066 ceph-mon[36504]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T06:18:20.581 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:20 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:20.581 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:20 smithi066 ceph-mon[36504]: from='osd.7 [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:18:20.581 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:20 smithi066 ceph-mon[36504]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:18:20.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:20 smithi037 ceph-mon[31974]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:20.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:20 smithi037 ceph-mon[31974]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T06:18:20.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:20 smithi037 ceph-mon[31974]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T06:18:20.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:20 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:20.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:20 smithi037 ceph-mon[31974]: from='osd.7 [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:18:20.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:20 smithi037 ceph-mon[31974]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:18:21.090 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T06:18:21.090 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T06:18:21.090 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-26T06:18:21.090 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-26T06:18:21.090 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129142 s, 812 MB/s 2024-03-26T06:18:21.091 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-26T06:18:21.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:21 smithi066 ceph-mon[36504]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:18:21.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:21 smithi066 ceph-mon[36504]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T06:18:21.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:21 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:21.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:21 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:21.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:21 smithi066 ceph-mon[36504]: from='osd.7 ' entity='osd.7' 2024-03-26T06:18:21.598 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:18:21 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:18:21.175+0000 7f00abd9c700 -1 osd.7 0 waiting for initial osdmap 2024-03-26T06:18:21.598 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:18:21 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:18:21.191+0000 7f00a4388700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:18:21.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:21 smithi149 ceph-mon[36681]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:18:21.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:21 smithi149 ceph-mon[36681]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T06:18:21.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:21 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:21.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:21 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:21.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:21 smithi149 ceph-mon[36681]: from='osd.7 ' entity='osd.7' 2024-03-26T06:18:21.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:21 smithi037 ceph-mon[31974]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:18:21.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:21 smithi037 ceph-mon[31974]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T06:18:21.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:21.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:21 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:21.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:21 smithi037 ceph-mon[31974]: from='osd.7 ' entity='osd.7' 2024-03-26T06:18:21.766 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-03-26T06:18:22.025 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:18:22.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:22 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:18:22.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:22 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:18:22.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:22 smithi149 ceph-mon[36681]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:22.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:22.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:22 smithi149 ceph-mon[36681]: osd.7 [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875] boot 2024-03-26T06:18:22.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:22 smithi149 ceph-mon[36681]: osdmap e51: 8 total, 8 up, 8 in 2024-03-26T06:18:22.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:22.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:22 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:18:22.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:22 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:18:22.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:22 smithi066 ceph-mon[36504]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:22.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:22.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:22 smithi066 ceph-mon[36504]: osd.7 [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875] boot 2024-03-26T06:18:22.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:22 smithi066 ceph-mon[36504]: osdmap e51: 8 total, 8 up, 8 in 2024-03-26T06:18:22.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:22.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:22 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:18:22.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:22 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:18:22.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:22 smithi037 ceph-mon[31974]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T06:18:22.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:22.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:22 smithi037 ceph-mon[31974]: osd.7 [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875] boot 2024-03-26T06:18:22.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:22 smithi037 ceph-mon[31974]: osdmap e51: 8 total, 8 up, 8 in 2024-03-26T06:18:22.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:18:23.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T06:18:23.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi066 2024-03-26T06:18:23.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:23.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:23.359 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:23.359 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi066 to 5693M 2024-03-26T06:18:23.359 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.359 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:23.359 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:23.359 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.359 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:23.359 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:23.359 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:23.359 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:23 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.557+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/1991112204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd0101f60 msgr2=0x7f1cd01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:18:23.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.557+0000 7f1cd6b6c700 1 --2- 172.21.15.149:0/1991112204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd0101f60 0x7f1cd01023c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009990 tx=0x7f1cb8009ca0 comp rx=0 tx=0).stop 2024-03-26T06:18:23.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.557+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/1991112204 shutdown_connections 2024-03-26T06:18:23.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.557+0000 7f1cd6b6c700 1 --2- 172.21.15.149:0/1991112204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cd0102900 0x7f1cd01091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:23.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.557+0000 7f1cd6b6c700 1 --2- 172.21.15.149:0/1991112204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd0100d60 0x7f1cd0101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:23.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.557+0000 7f1cd6b6c700 1 --2- 172.21.15.149:0/1991112204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd0101f60 0x7f1cd01023c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.557+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/1991112204 >> 172.21.15.149:0/1991112204 conn(0x7f1cd00fc300 msgr2=0x7f1cd00fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.557+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/1991112204 shutdown_connections 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.557+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/1991112204 wait complete. 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd6b6c700 1 Processor -- start 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd6b6c700 1 -- start start 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd6b6c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd0100d60 0x7f1cd0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd6b6c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd0101f60 0x7f1cd0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd6b6c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cd0102900 0x7f1cd019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd6b6c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd010da50 con 0x7f1cd0102900 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd6b6c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd010d8d0 con 0x7f1cd0101f60 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd6b6c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd010dbd0 con 0x7f1cd0100d60 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd4908700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd0100d60 0x7f1cd0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:18:23.564 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd4908700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd0100d60 0x7f1cd0197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:39370/0 (socket says 172.21.15.149:39370) 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd4908700 1 -- 172.21.15.149:0/3975868884 learned_addr learned my addr 172.21.15.149:0/3975868884 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1ccffff700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd0101f60 0x7f1cd0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd5109700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cd0102900 0x7f1cd019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd4908700 1 -- 172.21.15.149:0/3975868884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd0101f60 msgr2=0x7f1cd0198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd4908700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd0101f60 0x7f1cd0198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd4908700 1 -- 172.21.15.149:0/3975868884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cd0102900 msgr2=0x7f1cd019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd4908700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cd0102900 0x7f1cd019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.558+0000 7f1cd4908700 1 -- 172.21.15.149:0/3975868884 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd019fc40 con 0x7f1cd0100d60 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.559+0000 7f1cd4908700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd0100d60 0x7f1cd0197fe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800add0 tx=0x7f1cc800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.559+0000 7f1ccdffb700 1 -- 172.21.15.149:0/3975868884 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8017070 con 0x7f1cd0100d60 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.559+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/3975868884 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd019ff30 con 0x7f1cd0100d60 2024-03-26T06:18:23.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.559+0000 7f1ccdffb700 1 -- 172.21.15.149:0/3975868884 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cc8004bb0 con 0x7f1cd0100d60 2024-03-26T06:18:23.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.559+0000 7f1ccdffb700 1 -- 172.21.15.149:0/3975868884 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8004020 con 0x7f1cd0100d60 2024-03-26T06:18:23.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.559+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/3975868884 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd01a0380 con 0x7f1cd0100d60 2024-03-26T06:18:23.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.560+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/3975868884 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd002d830 con 0x7f1cd0100d60 2024-03-26T06:18:23.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.561+0000 7f1ccdffb700 1 -- 172.21.15.149:0/3975868884 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1cc801a070 con 0x7f1cd0100d60 2024-03-26T06:18:23.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.562+0000 7f1ccdffb700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1cbc0a2910 0x7f1cbc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:18:23.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.562+0000 7f1ccdffb700 1 -- 172.21.15.149:0/3975868884 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f1cc80cd190 con 0x7f1cd0100d60 2024-03-26T06:18:23.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.562+0000 7f1ccffff700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1cbc0a2910 0x7f1cbc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:18:23.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.563+0000 7f1ccffff700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1cbc0a2910 0x7f1cbc0a4dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800b5c0 tx=0x7f1cb8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:18:23.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.566+0000 7f1ccdffb700 1 -- 172.21.15.149:0/3975868884 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1cc809ad70 con 0x7f1cd0100d60 2024-03-26T06:18:23.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T06:18:23.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi066 2024-03-26T06:18:23.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:23.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:23.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:23.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi066 to 5693M 2024-03-26T06:18:23.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:23.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:23.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:23.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:23.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:23.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:23 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T06:18:23.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi066 2024-03-26T06:18:23.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:23.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:23.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:23.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi066 to 5693M 2024-03-26T06:18:23.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:23.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:23.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:23.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:23.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:23.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:23 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:23.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:23.771+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/3975868884 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cd0062220 con 0x7f1cbc0a2910 2024-03-26T06:18:24.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:24 smithi066 ceph-mon[36504]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:24.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:24 smithi066 ceph-mon[36504]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T06:18:24.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:18:24.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:18:24.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:24.658 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:24 smithi149 ceph-mon[36681]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:24.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:24 smithi149 ceph-mon[36681]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T06:18:24.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:18:24.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:18:24.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:24.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:24 smithi037 ceph-mon[31974]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:24.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:24 smithi037 ceph-mon[31974]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T06:18:24.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:18:24.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:18:24.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:25.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:25 smithi066 ceph-mon[36504]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:18:25.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:25 smithi066 ceph-mon[36504]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T06:18:25.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:25 smithi149 ceph-mon[36681]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:18:25.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:25 smithi149 ceph-mon[36681]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T06:18:25.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:25 smithi037 ceph-mon[31974]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:18:25.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:25 smithi037 ceph-mon[31974]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T06:18:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:26 smithi066 ceph-mon[36504]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:26 smithi066 ceph-mon[36504]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T06:18:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:26 smithi066 ceph-mon[36504]: from='client.? 172.21.15.149:0/313127264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "388ebc46-fd6a-48e1-a245-a8fe4c8d937e"}]: dispatch 2024-03-26T06:18:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:26 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "388ebc46-fd6a-48e1-a245-a8fe4c8d937e"}]: dispatch 2024-03-26T06:18:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:26 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "388ebc46-fd6a-48e1-a245-a8fe4c8d937e"}]': finished 2024-03-26T06:18:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:26 smithi066 ceph-mon[36504]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T06:18:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:26 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:26.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:26 smithi149 ceph-mon[36681]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:26.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:26 smithi149 ceph-mon[36681]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T06:18:26.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:26 smithi149 ceph-mon[36681]: from='client.? 172.21.15.149:0/313127264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "388ebc46-fd6a-48e1-a245-a8fe4c8d937e"}]: dispatch 2024-03-26T06:18:26.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:26 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "388ebc46-fd6a-48e1-a245-a8fe4c8d937e"}]: dispatch 2024-03-26T06:18:26.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:26 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "388ebc46-fd6a-48e1-a245-a8fe4c8d937e"}]': finished 2024-03-26T06:18:26.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:26 smithi149 ceph-mon[36681]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T06:18:26.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:26 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:26 smithi037 ceph-mon[31974]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:26 smithi037 ceph-mon[31974]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T06:18:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:26 smithi037 ceph-mon[31974]: from='client.? 172.21.15.149:0/313127264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "388ebc46-fd6a-48e1-a245-a8fe4c8d937e"}]: dispatch 2024-03-26T06:18:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:26 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "388ebc46-fd6a-48e1-a245-a8fe4c8d937e"}]: dispatch 2024-03-26T06:18:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:26 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "388ebc46-fd6a-48e1-a245-a8fe4c8d937e"}]': finished 2024-03-26T06:18:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:26 smithi037 ceph-mon[31974]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T06:18:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:27.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:27 smithi066 ceph-mon[36504]: from='client.? 172.21.15.149:0/3998588932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:18:27.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:27 smithi149 ceph-mon[36681]: from='client.? 172.21.15.149:0/3998588932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:18:27.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:27 smithi037 ceph-mon[31974]: from='client.? 172.21.15.149:0/3998588932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:18:28.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:28 smithi066 ceph-mon[36504]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:28.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:28 smithi149 ceph-mon[36681]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:28.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:28 smithi037 ceph-mon[31974]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:30.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:30 smithi066 ceph-mon[36504]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:30.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:30 smithi149 ceph-mon[36681]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:30.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:30 smithi037 ceph-mon[31974]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:32.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:32 smithi149 ceph-mon[36681]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:32.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:32 smithi066 ceph-mon[36504]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:32.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:32 smithi037 ceph-mon[31974]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:34.634 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:34 smithi149 ceph-mon[36681]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:34.634 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:34 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T06:18:34.635 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:34 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:34.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:34 smithi066 ceph-mon[36504]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:34.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:34 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T06:18:34.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:34 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:34.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:34 smithi037 ceph-mon[31974]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:34.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:34 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T06:18:34.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:34 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:35.791 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:35 smithi149 ceph-mon[36681]: Deploying daemon osd.8 on smithi149 2024-03-26T06:18:35.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:35 smithi066 ceph-mon[36504]: Deploying daemon osd.8 on smithi149 2024-03-26T06:18:35.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:35 smithi037 ceph-mon[31974]: Deploying daemon osd.8 on smithi149 2024-03-26T06:18:36.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:36 smithi066 ceph-mon[36504]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:36.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:36 smithi149 ceph-mon[36681]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:36 smithi037 ceph-mon[31974]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:38.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:38 smithi149 ceph-mon[36681]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:38.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:38 smithi066 ceph-mon[36504]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:38 smithi037 ceph-mon[31974]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:40.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:40 smithi149 ceph-mon[36681]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:40.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:40 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:40.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:40 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:40.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:40 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:40.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:40 smithi037 ceph-mon[31974]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:40.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:40 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:40.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:40 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:40.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:40 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:40.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:40 smithi066 ceph-mon[36504]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:40.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:40 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:40.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:40 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:40.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:40 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:42.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:42 smithi066 ceph-mon[36504]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:42.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:42 smithi149 ceph-mon[36681]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:42 smithi037 ceph-mon[31974]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:42.591 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.590+0000 7f1ccdffb700 1 -- 172.21.15.149:0/3975868884 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1cd0062220 con 0x7f1cbc0a2910 2024-03-26T06:18:42.591 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 8 on host 'smithi149' 2024-03-26T06:18:42.594 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.592+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/3975868884 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1cbc0a2910 msgr2=0x7f1cbc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:18:42.594 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.592+0000 7f1cd6b6c700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1cbc0a2910 0x7f1cbc0a4dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800b5c0 tx=0x7f1cb8005fb0 comp rx=0 tx=0).stop 2024-03-26T06:18:42.594 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.592+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/3975868884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd0100d60 msgr2=0x7f1cd0197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:18:42.594 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.592+0000 7f1cd6b6c700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd0100d60 0x7f1cd0197fe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800add0 tx=0x7f1cc800c330 comp rx=0 tx=0).stop 2024-03-26T06:18:42.594 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.593+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/3975868884 shutdown_connections 2024-03-26T06:18:42.595 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.593+0000 7f1cd6b6c700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cd0102900 0x7f1cd019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:42.595 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.593+0000 7f1cd6b6c700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1cbc0a2910 0x7f1cbc0a4dd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:42.595 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.593+0000 7f1cd6b6c700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd0100d60 0x7f1cd0197fe0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:42.595 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.593+0000 7f1cd6b6c700 1 --2- 172.21.15.149:0/3975868884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd0101f60 0x7f1cd0198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:42.595 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.593+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/3975868884 >> 172.21.15.149:0/3975868884 conn(0x7f1cd00fc300 msgr2=0x7f1cd0105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:18:42.595 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.593+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/3975868884 shutdown_connections 2024-03-26T06:18:42.595 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:18:42.593+0000 7f1cd6b6c700 1 -- 172.21.15.149:0/3975868884 wait complete. 2024-03-26T06:18:43.188 DEBUG:teuthology.orchestra.run.smithi149:osd.8> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.8.service 2024-03-26T06:18:43.191 INFO:tasks.cephadm:Deploying osd.9 on smithi037 with /dev/vg_nvme/lv_1... 2024-03-26T06:18:43.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T06:18:43.447 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:18:43.798 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:43 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:43.798 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:43 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:43.798 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:43 smithi037 ceph-mon[31974]: from='osd.8 [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T06:18:43.798 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:43 smithi037 ceph-mon[31974]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T06:18:43.798 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:43 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:43.798 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:43 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:43.838 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:43 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:43.838 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:43 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:43.838 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:43 smithi149 ceph-mon[36681]: from='osd.8 [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T06:18:43.838 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:43 smithi149 ceph-mon[36681]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T06:18:43.838 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:43 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:43.838 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:43 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:43.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:43 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:43.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:43 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:43.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:43 smithi066 ceph-mon[36504]: from='osd.8 [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T06:18:43.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:43 smithi066 ceph-mon[36504]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T06:18:43.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:43 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:43.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:43 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:44.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:44 smithi037 ceph-mon[31974]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:44.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:44 smithi037 ceph-mon[31974]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T06:18:44.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:44 smithi037 ceph-mon[31974]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T06:18:44.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:44.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:44 smithi037 ceph-mon[31974]: from='osd.8 [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:18:44.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:44 smithi037 ceph-mon[31974]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:18:44.724 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:44 smithi149 ceph-mon[36681]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:44.724 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:44 smithi149 ceph-mon[36681]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T06:18:44.724 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:44 smithi149 ceph-mon[36681]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T06:18:44.724 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:44.724 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:44 smithi149 ceph-mon[36681]: from='osd.8 [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:18:44.724 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:44 smithi149 ceph-mon[36681]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:18:44.725 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:18:44 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:18:44.569+0000 7ff16956c700 -1 osd.8 0 waiting for initial osdmap 2024-03-26T06:18:44.725 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:18:44 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:18:44.582+0000 7ff165b60700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:18:44.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:44 smithi066 ceph-mon[36504]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:44.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:44 smithi066 ceph-mon[36504]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T06:18:44.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:44 smithi066 ceph-mon[36504]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T06:18:44.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:44.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:44 smithi066 ceph-mon[36504]: from='osd.8 [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:18:44.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:44 smithi066 ceph-mon[36504]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:18:45.603 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T06:18:45.603 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T06:18:45.604 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-03-26T06:18:45.604 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-03-26T06:18:45.604 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113514 s, 924 MB/s 2024-03-26T06:18:45.604 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-03-26T06:18:45.725 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:45 smithi037 ceph-mon[31974]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:18:45.725 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:45 smithi037 ceph-mon[31974]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T06:18:45.725 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:45 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:45.726 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:45 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:45.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:45 smithi066 ceph-mon[36504]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:18:45.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:45 smithi066 ceph-mon[36504]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T06:18:45.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:45 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:45.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:45 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:45.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:45 smithi149 ceph-mon[36681]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:18:45.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:45 smithi149 ceph-mon[36681]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T06:18:45.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:45 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:45.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:45 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:46.327 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi037:vg_nvme/lv_1 2024-03-26T06:18:46.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:18:46.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:18:46.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:46.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: osd.8 [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283] boot 2024-03-26T06:18:46.580 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T06:18:46.582 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:18:46.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:18:46.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:18:46.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:46.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: osd.8 [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283] boot 2024-03-26T06:18:46.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T06:18:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:46 smithi066 ceph-mon[36504]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T06:18:46.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:18:46.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:18:46.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T06:18:46.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: osd.8 [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283] boot 2024-03-26T06:18:46.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T06:18:46.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:46.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:46.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:46.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:46.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:46.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:46.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:46 smithi037 ceph-mon[31974]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T06:18:46.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:18:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:18:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:46 smithi149 ceph-mon[36681]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T06:18:47.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:47 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi149 2024-03-26T06:18:47.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:47 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi149 to 1935M 2024-03-26T06:18:47.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:47.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:47.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:47.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:47.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:47 smithi066 ceph-mon[36504]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T06:18:47.906 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:47 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi149 2024-03-26T06:18:47.906 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:47 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi149 to 1935M 2024-03-26T06:18:47.906 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:47.906 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:47.907 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:47.907 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:47.907 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:47 smithi037 ceph-mon[31974]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T06:18:47.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:47 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi149 2024-03-26T06:18:47.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:47 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi149 to 1935M 2024-03-26T06:18:47.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:18:47.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:47.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:18:47.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:18:47.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:47 smithi149 ceph-mon[36681]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T06:18:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.528+0000 7f212f8da700 1 -- 172.21.15.37:0/1064828215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2128101e10 msgr2=0x7f2128102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:18:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.528+0000 7f212f8da700 1 --2- 172.21.15.37:0/1064828215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2128101e10 0x7f2128102230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2118009990 tx=0x7f2118009ca0 comp rx=0 tx=0).stop 2024-03-26T06:18:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.528+0000 7f212f8da700 1 -- 172.21.15.37:0/1064828215 shutdown_connections 2024-03-26T06:18:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.528+0000 7f212f8da700 1 --2- 172.21.15.37:0/1064828215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2128101e10 0x7f2128102230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.528+0000 7f212f8da700 1 --2- 172.21.15.37:0/1064828215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2128103930 0x7f212810a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.528+0000 7f212f8da700 1 --2- 172.21.15.37:0/1064828215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2128102f70 0x7f21281033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.528+0000 7f212f8da700 1 -- 172.21.15.37:0/1064828215 >> 172.21.15.37:0/1064828215 conn(0x7f21280fd3f0 msgr2=0x7f21280ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:18:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.529+0000 7f212f8da700 1 -- 172.21.15.37:0/1064828215 shutdown_connections 2024-03-26T06:18:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.529+0000 7f212f8da700 1 -- 172.21.15.37:0/1064828215 wait complete. 2024-03-26T06:18:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.529+0000 7f212f8da700 1 Processor -- start 2024-03-26T06:18:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.529+0000 7f212f8da700 1 -- start start 2024-03-26T06:18:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.529+0000 7f212f8da700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2128101e10 0x7f2128075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:18:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.529+0000 7f212f8da700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2128102f70 0x7f21280741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.529+0000 7f212f8da700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2128103930 0x7f2128074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.529+0000 7f212f8da700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212810d9e0 con 0x7f2128101e10 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.529+0000 7f212f8da700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212810d860 con 0x7f2128102f70 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.529+0000 7f212f8da700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212810db60 con 0x7f2128103930 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212d676700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2128101e10 0x7f2128075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212d676700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2128101e10 0x7f2128075b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37154/0 (socket says 172.21.15.37:37154) 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212d676700 1 -- 172.21.15.37:0/2599789117 learned_addr learned my addr 172.21.15.37:0/2599789117 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212de77700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2128103930 0x7f2128074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212ce75700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2128102f70 0x7f21280741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212d676700 1 -- 172.21.15.37:0/2599789117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2128103930 msgr2=0x7f2128074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212d676700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2128103930 0x7f2128074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212d676700 1 -- 172.21.15.37:0/2599789117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2128102f70 msgr2=0x7f21280741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:18:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212d676700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2128102f70 0x7f21280741d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:18:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212d676700 1 -- 172.21.15.37:0/2599789117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2128074fc0 con 0x7f2128101e10 2024-03-26T06:18:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212de77700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2128103930 0x7f2128074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:18:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212d676700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2128101e10 0x7f2128075b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f211800b5c0 tx=0x7f2118009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:18:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f211e7fc700 1 -- 172.21.15.37:0/2599789117 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118027070 con 0x7f2128101e10 2024-03-26T06:18:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f211e7fc700 1 -- 172.21.15.37:0/2599789117 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21180041e0 con 0x7f2128101e10 2024-03-26T06:18:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212f8da700 1 -- 172.21.15.37:0/2599789117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21281a20f0 con 0x7f2128101e10 2024-03-26T06:18:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f211e7fc700 1 -- 172.21.15.37:0/2599789117 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211800f460 con 0x7f2128101e10 2024-03-26T06:18:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.530+0000 7f212f8da700 1 -- 172.21.15.37:0/2599789117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21281a2540 con 0x7f2128101e10 2024-03-26T06:18:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.531+0000 7f212f8da700 1 -- 172.21.15.37:0/2599789117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2128068aa0 con 0x7f2128101e10 2024-03-26T06:18:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.533+0000 7f211e7fc700 1 -- 172.21.15.37:0/2599789117 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f211802a070 con 0x7f2128101e10 2024-03-26T06:18:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.534+0000 7f211e7fc700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f21140a2890 0x7f21140a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:18:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.534+0000 7f212ce75700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f21140a2890 0x7f21140a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:18:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.534+0000 7f211e7fc700 1 -- 172.21.15.37:0/2599789117 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f21180cf5d0 con 0x7f2128101e10 2024-03-26T06:18:48.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.534+0000 7f212ce75700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f21140a2890 0x7f21140a4d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2124005ea0 tx=0x7f2124005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:18:48.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.537+0000 7f211e7fc700 1 -- 172.21.15.37:0/2599789117 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2118013070 con 0x7f2128101e10 2024-03-26T06:18:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:18:48.755+0000 7f212f8da700 1 -- 172.21.15.37:0/2599789117 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2128062220 con 0x7f21140a2890 2024-03-26T06:18:48.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:48 smithi149 ceph-mon[36681]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:18:48.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:48 smithi037 ceph-mon[31974]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:18:49.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:48 smithi066 ceph-mon[36504]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:18:49.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:49 smithi149 ceph-mon[36681]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T06:18:49.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:49 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:18:49.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:49 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:18:49.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:49 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:49.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:49 smithi037 ceph-mon[31974]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T06:18:49.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:18:49.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:18:49.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:49 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:50.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:49 smithi066 ceph-mon[36504]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T06:18:50.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:18:50.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:18:50.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:49 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:18:50.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:50 smithi149 ceph-mon[36681]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:18:50.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:50 smithi149 ceph-mon[36681]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:18:50.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:50 smithi149 ceph-mon[36681]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T06:18:50.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:50 smithi037 ceph-mon[31974]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:18:50.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:50 smithi037 ceph-mon[31974]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:18:50.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:50 smithi037 ceph-mon[31974]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T06:18:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:50 smithi066 ceph-mon[36504]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:18:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:50 smithi066 ceph-mon[36504]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:18:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:50 smithi066 ceph-mon[36504]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T06:18:51.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:51 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3877948467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c030ee66-7e52-4cf8-a8c9-3987bd5e7328"}]: dispatch 2024-03-26T06:18:51.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:51 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3877948467' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c030ee66-7e52-4cf8-a8c9-3987bd5e7328"}]': finished 2024-03-26T06:18:51.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:51 smithi149 ceph-mon[36681]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T06:18:51.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:51 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:18:51.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:51 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3877948467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c030ee66-7e52-4cf8-a8c9-3987bd5e7328"}]: dispatch 2024-03-26T06:18:51.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:51 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3877948467' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c030ee66-7e52-4cf8-a8c9-3987bd5e7328"}]': finished 2024-03-26T06:18:51.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:51 smithi037 ceph-mon[31974]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T06:18:51.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:51 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:18:52.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:51 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3877948467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c030ee66-7e52-4cf8-a8c9-3987bd5e7328"}]: dispatch 2024-03-26T06:18:52.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:51 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3877948467' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c030ee66-7e52-4cf8-a8c9-3987bd5e7328"}]': finished 2024-03-26T06:18:52.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:51 smithi066 ceph-mon[36504]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T06:18:52.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:51 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:18:52.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:52 smithi037 ceph-mon[31974]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:18:52.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:52 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/924495834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:18:53.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:52 smithi066 ceph-mon[36504]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:18:53.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:52 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/924495834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:18:53.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:52 smithi149 ceph-mon[36681]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:18:53.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:52 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/924495834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:18:54.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:54 smithi037 ceph-mon[31974]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-26T06:18:55.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:54 smithi066 ceph-mon[36504]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-26T06:18:55.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:54 smithi149 ceph-mon[36681]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-26T06:18:56.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:56 smithi037 ceph-mon[31974]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-26T06:18:56.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:56 smithi149 ceph-mon[36681]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-26T06:18:57.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:56 smithi066 ceph-mon[36504]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-26T06:18:58.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:18:58 smithi037 ceph-mon[31974]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T06:18:59.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:18:58 smithi066 ceph-mon[36504]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T06:18:59.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:18:58 smithi149 ceph-mon[36681]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T06:19:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:00 smithi037 ceph-mon[31974]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-26T06:19:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T06:19:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:00 smithi037 ceph-mon[31974]: Deploying daemon osd.9 on smithi037 2024-03-26T06:19:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:00 smithi066 ceph-mon[36504]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-26T06:19:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T06:19:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:00 smithi066 ceph-mon[36504]: Deploying daemon osd.9 on smithi037 2024-03-26T06:19:01.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:00 smithi149 ceph-mon[36681]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-26T06:19:01.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T06:19:01.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:01.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:00 smithi149 ceph-mon[36681]: Deploying daemon osd.9 on smithi037 2024-03-26T06:19:02.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:02 smithi037 ceph-mon[31974]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T06:19:03.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:02 smithi066 ceph-mon[36504]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T06:19:03.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:02 smithi149 ceph-mon[36681]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T06:19:03.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:03 smithi037 ceph-mon[31974]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-26T06:19:04.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:03 smithi066 ceph-mon[36504]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-26T06:19:04.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:03 smithi149 ceph-mon[36681]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-26T06:19:06.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:06 smithi066 ceph-mon[36504]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:06.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:06 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:06.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:06 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:06.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:06 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:06.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:06 smithi149 ceph-mon[36681]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:06.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:06 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:06.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:06 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:06.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:06 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:06.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:06 smithi037 ceph-mon[31974]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:06.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:06 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:06.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:06 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:06.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:06 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:08.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:08 smithi037 ceph-mon[31974]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:08.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:08 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:08.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:08 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:08.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:08 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:08.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:08 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:08.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:08 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:08 smithi066 ceph-mon[36504]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:08 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:08 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:08 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:08 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:08 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:08.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:08 smithi149 ceph-mon[36681]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:08.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:08 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:08.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:08 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:08.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:08 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:08.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:08 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:08.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:08 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:09.363 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 9 on host 'smithi037' 2024-03-26T06:19:09.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.362+0000 7f211e7fc700 1 -- 172.21.15.37:0/2599789117 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2128062220 con 0x7f21140a2890 2024-03-26T06:19:09.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.364+0000 7f212f8da700 1 -- 172.21.15.37:0/2599789117 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f21140a2890 msgr2=0x7f21140a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:19:09.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.364+0000 7f212f8da700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f21140a2890 0x7f21140a4d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2124005ea0 tx=0x7f2124005e30 comp rx=0 tx=0).stop 2024-03-26T06:19:09.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.364+0000 7f212f8da700 1 -- 172.21.15.37:0/2599789117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2128101e10 msgr2=0x7f2128075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:19:09.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.364+0000 7f212f8da700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2128101e10 0x7f2128075b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f211800b5c0 tx=0x7f2118009ee0 comp rx=0 tx=0).stop 2024-03-26T06:19:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.364+0000 7f212f8da700 1 -- 172.21.15.37:0/2599789117 shutdown_connections 2024-03-26T06:19:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.364+0000 7f212f8da700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2128101e10 0x7f2128075b80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.364+0000 7f212f8da700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f21140a2890 0x7f21140a4d50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.364+0000 7f212f8da700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2128103930 0x7f2128074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.364+0000 7f212f8da700 1 --2- 172.21.15.37:0/2599789117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2128102f70 0x7f21280741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.364+0000 7f212f8da700 1 -- 172.21.15.37:0/2599789117 >> 172.21.15.37:0/2599789117 conn(0x7f21280fd3f0 msgr2=0x7f2128106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:19:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.365+0000 7f212f8da700 1 -- 172.21.15.37:0/2599789117 shutdown_connections 2024-03-26T06:19:09.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:19:09.365+0000 7f212f8da700 1 -- 172.21.15.37:0/2599789117 wait complete. 2024-03-26T06:19:10.013 DEBUG:teuthology.orchestra.run.smithi037:osd.9> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.9.service 2024-03-26T06:19:10.016 INFO:tasks.cephadm:Deploying osd.10 on smithi066 with /dev/vg_nvme/lv_1... 2024-03-26T06:19:10.016 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T06:19:10.276 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:19:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:10 smithi066 ceph-mon[36504]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:10 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:10 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:10 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:10 smithi066 ceph-mon[36504]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T06:19:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:10 smithi149 ceph-mon[36681]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:10 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:10 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:10 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:10.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:10 smithi149 ceph-mon[36681]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T06:19:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:10 smithi037 ceph-mon[31974]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:10 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:10.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:10 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:10.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:10 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:10.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:10 smithi037 ceph-mon[31974]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T06:19:11.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:11 smithi037 ceph-mon[31974]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T06:19:11.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:11 smithi037 ceph-mon[31974]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T06:19:11.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:11 smithi037 ceph-mon[31974]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:19:11.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:11 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:11.590 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:11 smithi066 ceph-mon[36504]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T06:19:11.590 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:11 smithi066 ceph-mon[36504]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T06:19:11.590 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:11 smithi066 ceph-mon[36504]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:19:11.590 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:11 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:11.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:11 smithi149 ceph-mon[36681]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T06:19:11.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:11 smithi149 ceph-mon[36681]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T06:19:11.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:11 smithi149 ceph-mon[36681]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-26T06:19:11.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:11 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:12.285 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T06:19:12.285 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T06:19:12.285 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-03-26T06:19:12.285 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-03-26T06:19:12.285 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111015 s, 945 MB/s 2024-03-26T06:19:12.285 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-03-26T06:19:12.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:12 smithi037 ceph-mon[31974]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:12.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:12 smithi037 ceph-mon[31974]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:19:12.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:12 smithi037 ceph-mon[31974]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T06:19:12.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:12 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:12.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:12 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:12.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:12 smithi037 ceph-mon[31974]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' 2024-03-26T06:19:12.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:12 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:12.425 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:19:12 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:19:12.151+0000 7f53052c3700 -1 osd.9 0 waiting for initial osdmap 2024-03-26T06:19:12.425 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:19:12 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:19:12.170+0000 7f52fd8af700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:19:12.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:12 smithi066 ceph-mon[36504]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:12.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:12 smithi066 ceph-mon[36504]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:19:12.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:12 smithi066 ceph-mon[36504]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T06:19:12.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:12 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:12.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:12 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:12.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:12 smithi066 ceph-mon[36504]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' 2024-03-26T06:19:12.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:12 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:12.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:12 smithi149 ceph-mon[36681]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:12.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:12 smithi149 ceph-mon[36681]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-26T06:19:12.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:12 smithi149 ceph-mon[36681]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T06:19:12.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:12 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:12.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:12 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:12.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:12 smithi149 ceph-mon[36681]: from='osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917]' entity='osd.9' 2024-03-26T06:19:12.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:12 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:12.894 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi066:vg_nvme/lv_1 2024-03-26T06:19:13.150 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:19:13.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:19:13.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:19:13.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917] boot 2024-03-26T06:19:13.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T06:19:13.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:13 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:13.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:19:13.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:19:13.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917] boot 2024-03-26T06:19:13.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T06:19:13.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:13.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:13.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:13.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.587 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:13.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:13.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:13.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:13 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:13.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: osd.9 [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917] boot 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:13.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:13 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:14.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:14 smithi066 ceph-mon[36504]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:14.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:14 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi037 2024-03-26T06:19:14.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:14 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi037 to 4273M 2024-03-26T06:19:14.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:14 smithi149 ceph-mon[36681]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:14.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:14 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi037 2024-03-26T06:19:14.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:14 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi037 to 4273M 2024-03-26T06:19:14.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:14 smithi037 ceph-mon[31974]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T06:19:14.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:14 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi037 2024-03-26T06:19:14.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:14 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi037 to 4273M 2024-03-26T06:19:14.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.867+0000 7f2286d58700 1 -- 172.21.15.66:0/349851675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2280101e10 msgr2=0x7f2280102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:19:14.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 --2- 172.21.15.66:0/349851675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2280101e10 0x7f2280102230 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2274009990 tx=0x7f2274009ca0 comp rx=0 tx=0).stop 2024-03-26T06:19:14.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 -- 172.21.15.66:0/349851675 shutdown_connections 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 --2- 172.21.15.66:0/349851675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2280103930 0x7f228010a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 --2- 172.21.15.66:0/349851675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2280101e10 0x7f2280102230 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 --2- 172.21.15.66:0/349851675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2280102f70 0x7f22801033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 -- 172.21.15.66:0/349851675 >> 172.21.15.66:0/349851675 conn(0x7f22800fd3f0 msgr2=0x7f22800ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 -- 172.21.15.66:0/349851675 shutdown_connections 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 -- 172.21.15.66:0/349851675 wait complete. 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 Processor -- start 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 -- start start 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2280101e10 0x7f2280197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2280102f70 0x7f2280198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2280103930 0x7f228019f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:19:14.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228010da00 con 0x7f2280103930 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228010d880 con 0x7f2280101e10 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.868+0000 7f2286d58700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228010db80 con 0x7f2280102f70 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f2284af4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2280101e10 0x7f2280197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f2284af4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2280101e10 0x7f2280197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45882/0 (socket says 172.21.15.66:45882) 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f2284af4700 1 -- 172.21.15.66:0/65224539 learned_addr learned my addr 172.21.15.66:0/65224539 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f227ffff700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2280102f70 0x7f2280198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f22852f5700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2280103930 0x7f228019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f2284af4700 1 -- 172.21.15.66:0/65224539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2280102f70 msgr2=0x7f2280198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f2284af4700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2280102f70 0x7f2280198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f2284af4700 1 -- 172.21.15.66:0/65224539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2280103930 msgr2=0x7f228019f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f2284af4700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2280103930 0x7f228019f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f2284af4700 1 -- 172.21.15.66:0/65224539 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228019fbb0 con 0x7f2280101e10 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f22852f5700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2280103930 0x7f228019f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:19:14.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f2284af4700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2280101e10 0x7f2280197f50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2274005b30 tx=0x7f2274009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:19:14.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f227dffb700 1 -- 172.21.15.66:0/65224539 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274003cc0 con 0x7f2280101e10 2024-03-26T06:19:14.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f2286d58700 1 -- 172.21.15.66:0/65224539 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228019fe40 con 0x7f2280101e10 2024-03-26T06:19:14.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f227dffb700 1 -- 172.21.15.66:0/65224539 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2274004830 con 0x7f2280101e10 2024-03-26T06:19:14.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f227dffb700 1 -- 172.21.15.66:0/65224539 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274005400 con 0x7f2280101e10 2024-03-26T06:19:14.872 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.869+0000 7f2286d58700 1 -- 172.21.15.66:0/65224539 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22801a01e0 con 0x7f2280101e10 2024-03-26T06:19:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.870+0000 7f2286d58700 1 -- 172.21.15.66:0/65224539 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228002d830 con 0x7f2280101e10 2024-03-26T06:19:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.872+0000 7f227dffb700 1 -- 172.21.15.66:0/65224539 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2274003e60 con 0x7f2280101e10 2024-03-26T06:19:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.872+0000 7f227dffb700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22700a2910 0x7f22700a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:19:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.872+0000 7f227dffb700 1 -- 172.21.15.66:0/65224539 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f22740cf680 con 0x7f2280101e10 2024-03-26T06:19:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.872+0000 7f227ffff700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22700a2910 0x7f22700a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:19:14.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.873+0000 7f227ffff700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22700a2910 0x7f22700a4dd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f226c00ac50 tx=0x7f226c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:19:14.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:14.876+0000 7f227dffb700 1 -- 172.21.15.66:0/65224539 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2274013070 con 0x7f2280101e10 2024-03-26T06:19:15.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:15.091+0000 7f2286d58700 1 -- 172.21.15.66:0/65224539 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2280062220 con 0x7f22700a2910 2024-03-26T06:19:15.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:15 smithi149 ceph-mon[36681]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T06:19:15.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:15 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:19:15.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:15 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:19:15.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:15 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:15.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:15 smithi037 ceph-mon[31974]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T06:19:15.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:15 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:19:15.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:15 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:19:15.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:15 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:15.720 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:15 smithi066 ceph-mon[36504]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T06:19:15.720 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:15 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:19:15.720 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:15 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:19:15.720 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:15 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:16.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:16 smithi149 ceph-mon[36681]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:16.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:16 smithi149 ceph-mon[36681]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:19:16.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:16 smithi037 ceph-mon[31974]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:16.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:16 smithi037 ceph-mon[31974]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:19:16.738 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:16 smithi066 ceph-mon[36504]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:16.738 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:16 smithi066 ceph-mon[36504]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:19:18.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:18 smithi066 ceph-mon[36504]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:18.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:18 smithi066 ceph-mon[36504]: from='client.? 172.21.15.66:0/1421008631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7ea7170-f2ed-4d72-afee-7b2828f3a774"}]: dispatch 2024-03-26T06:19:18.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:18 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7ea7170-f2ed-4d72-afee-7b2828f3a774"}]: dispatch 2024-03-26T06:19:18.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:18 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7ea7170-f2ed-4d72-afee-7b2828f3a774"}]': finished 2024-03-26T06:19:18.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:18 smithi066 ceph-mon[36504]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T06:19:18.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:18 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:18.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:18 smithi149 ceph-mon[36681]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:18.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:18 smithi149 ceph-mon[36681]: from='client.? 172.21.15.66:0/1421008631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7ea7170-f2ed-4d72-afee-7b2828f3a774"}]: dispatch 2024-03-26T06:19:18.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:18 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7ea7170-f2ed-4d72-afee-7b2828f3a774"}]: dispatch 2024-03-26T06:19:18.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:18 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7ea7170-f2ed-4d72-afee-7b2828f3a774"}]': finished 2024-03-26T06:19:18.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:18 smithi149 ceph-mon[36681]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T06:19:18.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:18 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:18 smithi037 ceph-mon[31974]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:18 smithi037 ceph-mon[31974]: from='client.? 172.21.15.66:0/1421008631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7ea7170-f2ed-4d72-afee-7b2828f3a774"}]: dispatch 2024-03-26T06:19:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:18 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7ea7170-f2ed-4d72-afee-7b2828f3a774"}]: dispatch 2024-03-26T06:19:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:18 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7ea7170-f2ed-4d72-afee-7b2828f3a774"}]': finished 2024-03-26T06:19:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:18 smithi037 ceph-mon[31974]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T06:19:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:18 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:19.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:19 smithi149 ceph-mon[36681]: from='client.? 172.21.15.66:0/337473814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:19:19.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:19 smithi037 ceph-mon[31974]: from='client.? 172.21.15.66:0/337473814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:19:19.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:19 smithi066 ceph-mon[36504]: from='client.? 172.21.15.66:0/337473814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:19:20.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:20 smithi149 ceph-mon[36681]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:20.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:20 smithi037 ceph-mon[31974]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:20.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:20 smithi066 ceph-mon[36504]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:22.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:22 smithi066 ceph-mon[36504]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:22.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:22 smithi149 ceph-mon[36681]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:22.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:22 smithi037 ceph-mon[31974]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:24.544 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:24 smithi066 ceph-mon[36504]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:24.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:24 smithi149 ceph-mon[36681]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:24.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:24 smithi037 ceph-mon[31974]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:26.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:26 smithi149 ceph-mon[36681]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:26.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:26 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T06:19:26.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:26 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:26.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:26 smithi037 ceph-mon[31974]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:26.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T06:19:26.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:26 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:26.700 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:26 smithi066 ceph-mon[36504]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:26.700 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:26 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T06:19:26.700 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:26 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:27.637 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:27 smithi066 ceph-mon[36504]: Deploying daemon osd.10 on smithi066 2024-03-26T06:19:27.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:27 smithi149 ceph-mon[36681]: Deploying daemon osd.10 on smithi066 2024-03-26T06:19:27.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:27 smithi037 ceph-mon[31974]: Deploying daemon osd.10 on smithi066 2024-03-26T06:19:28.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:28 smithi149 ceph-mon[36681]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:28.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:28 smithi037 ceph-mon[31974]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:28.726 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:28 smithi066 ceph-mon[36504]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:30.527 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:30 smithi066 ceph-mon[36504]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:30.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:30 smithi149 ceph-mon[36681]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:30 smithi037 ceph-mon[31974]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:32.617 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:32 smithi066 ceph-mon[36504]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:32.617 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:32 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:32.618 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:32 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:32.618 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:32 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:32.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:32 smithi149 ceph-mon[36681]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:32.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:32 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:32.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:32 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:32.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:32 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:32.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:32 smithi037 ceph-mon[31974]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:32.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:32 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:32.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:32 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:32.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:32 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:34.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:34 smithi149 ceph-mon[36681]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:34.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:34 smithi037 ceph-mon[31974]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:34.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:34 smithi066 ceph-mon[36504]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:35.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.160+0000 7f227dffb700 1 -- 172.21.15.66:0/65224539 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f2280062220 con 0x7f22700a2910 2024-03-26T06:19:35.161 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 10 on host 'smithi066' 2024-03-26T06:19:35.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 -- 172.21.15.66:0/65224539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22700a2910 msgr2=0x7f22700a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:19:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22700a2910 0x7f22700a4dd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f226c00ac50 tx=0x7f226c00a380 comp rx=0 tx=0).stop 2024-03-26T06:19:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 -- 172.21.15.66:0/65224539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2280101e10 msgr2=0x7f2280197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:19:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2280101e10 0x7f2280197f50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2274005b30 tx=0x7f2274009580 comp rx=0 tx=0).stop 2024-03-26T06:19:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 -- 172.21.15.66:0/65224539 shutdown_connections 2024-03-26T06:19:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2280103930 0x7f228019f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f22700a2910 0x7f22700a4dd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2280101e10 0x7f2280197f50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 --2- 172.21.15.66:0/65224539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2280102f70 0x7f2280198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 -- 172.21.15.66:0/65224539 >> 172.21.15.66:0/65224539 conn(0x7f22800fd3f0 msgr2=0x7f2280106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:19:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 -- 172.21.15.66:0/65224539 shutdown_connections 2024-03-26T06:19:35.163 INFO:teuthology.orchestra.run.smithi066.stderr:2024-03-26T06:19:35.162+0000 7f2286d58700 1 -- 172.21.15.66:0/65224539 wait complete. 2024-03-26T06:19:35.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:35 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:35 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:35 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:35.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:35 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:35.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:35 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:35 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:35 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:35 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:35.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:35 smithi037 ceph-mon[31974]: from='osd.10 [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T06:19:35.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:35 smithi037 ceph-mon[31974]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T06:19:35.784 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:35 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.784 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:35 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.784 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:35 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:35.784 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:35 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:35.784 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:35 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.784 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:35 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.784 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:35 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.784 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:35 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:35.784 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:35 smithi066 ceph-mon[36504]: from='osd.10 [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T06:19:35.784 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:35 smithi066 ceph-mon[36504]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T06:19:35.814 DEBUG:teuthology.orchestra.run.smithi066:osd.10> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.10.service 2024-03-26T06:19:35.816 INFO:tasks.cephadm:Deploying osd.11 on smithi149 with /dev/vg_nvme/lv_1... 2024-03-26T06:19:35.816 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T06:19:35.846 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:35 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:35 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:35 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:35.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:35 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:35.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:35 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:35 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:35 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:35.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:35 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:35.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:35 smithi149 ceph-mon[36681]: from='osd.10 [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T06:19:35.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:35 smithi149 ceph-mon[36681]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T06:19:36.063 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:19:36.756 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:36 smithi066 ceph-mon[36504]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:36.756 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:36 smithi066 ceph-mon[36504]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T06:19:36.757 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:36 smithi066 ceph-mon[36504]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T06:19:36.757 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:36 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:36.757 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:36 smithi066 ceph-mon[36504]: from='osd.10 [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:19:36.757 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:36 smithi066 ceph-mon[36504]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:19:36.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:36 smithi149 ceph-mon[36681]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:36.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:36 smithi149 ceph-mon[36681]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T06:19:36.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:36 smithi149 ceph-mon[36681]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T06:19:36.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:36 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:36.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:36 smithi149 ceph-mon[36681]: from='osd.10 [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:19:36.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:36 smithi149 ceph-mon[36681]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:19:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:36 smithi037 ceph-mon[31974]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:36 smithi037 ceph-mon[31974]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T06:19:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:36 smithi037 ceph-mon[31974]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T06:19:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:36 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:36 smithi037 ceph-mon[31974]: from='osd.10 [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:19:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:36 smithi037 ceph-mon[31974]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-03-26T06:19:38.115 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T06:19:38.115 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T06:19:38.116 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-26T06:19:38.116 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-26T06:19:38.116 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112877 s, 929 MB/s 2024-03-26T06:19:38.116 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-26T06:19:38.164 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:38 smithi149 ceph-mon[36681]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:38.164 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:38 smithi149 ceph-mon[36681]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:19:38.164 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:38 smithi149 ceph-mon[36681]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T06:19:38.164 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:38.164 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:38.164 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:38 smithi149 ceph-mon[36681]: from='osd.10 ' entity='osd.10' 2024-03-26T06:19:38.305 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:38 smithi066 ceph-mon[36504]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:38.305 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:38 smithi066 ceph-mon[36504]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:19:38.305 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:38 smithi066 ceph-mon[36504]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T06:19:38.305 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:38.305 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:38.306 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:38 smithi066 ceph-mon[36504]: from='osd.10 ' entity='osd.10' 2024-03-26T06:19:38.306 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:19:38 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:19:38.013+0000 7ff12f166700 -1 osd.10 0 waiting for initial osdmap 2024-03-26T06:19:38.306 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:19:38 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:19:38.030+0000 7ff127752700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:19:38.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:38 smithi037 ceph-mon[31974]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T06:19:38.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:38 smithi037 ceph-mon[31974]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-03-26T06:19:38.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:38 smithi037 ceph-mon[31974]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T06:19:38.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:38.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:38.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:38 smithi037 ceph-mon[31974]: from='osd.10 ' entity='osd.10' 2024-03-26T06:19:38.861 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2024-03-26T06:19:39.151 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:19:39.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: osd.10 [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155] boot 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:39.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:39.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:39.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:39.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:19:39.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:19:39.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: osd.10 [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155] boot 2024-03-26T06:19:39.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T06:19:39.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:39.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:39.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:39.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:39.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:39.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:39.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:39.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:19:39.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:19:39.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: osd.10 [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155] boot 2024-03-26T06:19:39.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T06:19:39.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:19:39.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:39.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:39.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:19:39.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:39.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:39.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:19:39.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:40.296 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:40 smithi149 ceph-mon[36681]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 511 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:40.296 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:40 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi066 2024-03-26T06:19:40.296 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:40 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi066 to 4270M 2024-03-26T06:19:40.296 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:40 smithi149 ceph-mon[36681]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T06:19:40.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:40 smithi037 ceph-mon[31974]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 511 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:40.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:40 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi066 2024-03-26T06:19:40.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:40 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi066 to 4270M 2024-03-26T06:19:40.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:40 smithi037 ceph-mon[31974]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T06:19:40.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:40 smithi066 ceph-mon[36504]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 511 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:40.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:40 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi066 2024-03-26T06:19:40.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:40 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi066 to 4270M 2024-03-26T06:19:40.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:40 smithi066 ceph-mon[36504]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T06:19:40.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.809+0000 7f1a875cd700 1 -- 172.21.15.149:0/879967027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a80101f40 msgr2=0x7f1a801023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:19:40.811 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.809+0000 7f1a875cd700 1 --2- 172.21.15.149:0/879967027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a80101f40 0x7f1a801023a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009990 tx=0x7f1a70009ca0 comp rx=0 tx=0).stop 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.809+0000 7f1a875cd700 1 -- 172.21.15.149:0/879967027 shutdown_connections 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.809+0000 7f1a875cd700 1 --2- 172.21.15.149:0/879967027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a801028e0 0x7f1a80109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.809+0000 7f1a875cd700 1 --2- 172.21.15.149:0/879967027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a80100d40 0x7f1a80101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.809+0000 7f1a875cd700 1 --2- 172.21.15.149:0/879967027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a80101f40 0x7f1a801023a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.809+0000 7f1a875cd700 1 -- 172.21.15.149:0/879967027 >> 172.21.15.149:0/879967027 conn(0x7f1a800fc300 msgr2=0x7f1a800fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.809+0000 7f1a875cd700 1 -- 172.21.15.149:0/879967027 shutdown_connections 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.809+0000 7f1a875cd700 1 -- 172.21.15.149:0/879967027 wait complete. 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.809+0000 7f1a875cd700 1 Processor -- start 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.809+0000 7f1a875cd700 1 -- start start 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a875cd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a80100d40 0x7f1a801980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a875cd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a80101f40 0x7f1a80198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:19:40.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a875cd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a801028e0 0x7f1a8019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:19:40.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a875cd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8010c920 con 0x7f1a80100d40 2024-03-26T06:19:40.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a875cd700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8010c7a0 con 0x7f1a80101f40 2024-03-26T06:19:40.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a875cd700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8010caa0 con 0x7f1a801028e0 2024-03-26T06:19:40.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a85b6a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a801028e0 0x7f1a8019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:19:40.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a85b6a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a801028e0 0x7f1a8019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:54588/0 (socket says 172.21.15.149:54588) 2024-03-26T06:19:40.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a85b6a700 1 -- 172.21.15.149:0/336924314 learned_addr learned my addr 172.21.15.149:0/336924314 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-26T06:19:40.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a84b68700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a80101f40 0x7f1a80198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:19:40.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a85369700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a80100d40 0x7f1a801980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:19:40.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a85b6a700 1 -- 172.21.15.149:0/336924314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a80101f40 msgr2=0x7f1a80198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:19:40.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a85b6a700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a80101f40 0x7f1a80198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:40.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a85b6a700 1 -- 172.21.15.149:0/336924314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a80100d40 msgr2=0x7f1a801980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:19:40.814 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a85b6a700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a80100d40 0x7f1a801980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:19:40.814 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a85b6a700 1 -- 172.21.15.149:0/336924314 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8019fce0 con 0x7f1a801028e0 2024-03-26T06:19:40.814 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a85b6a700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a801028e0 0x7f1a8019f640 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a78007f10 tx=0x7f1a7800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:19:40.814 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.810+0000 7f1a767fc700 1 -- 172.21.15.149:0/336924314 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78004bb0 con 0x7f1a801028e0 2024-03-26T06:19:40.814 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.811+0000 7f1a767fc700 1 -- 172.21.15.149:0/336924314 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a78004d50 con 0x7f1a801028e0 2024-03-26T06:19:40.814 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.811+0000 7f1a875cd700 1 -- 172.21.15.149:0/336924314 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8019ffd0 con 0x7f1a801028e0 2024-03-26T06:19:40.814 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.811+0000 7f1a767fc700 1 -- 172.21.15.149:0/336924314 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a780059f0 con 0x7f1a801028e0 2024-03-26T06:19:40.814 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.811+0000 7f1a875cd700 1 -- 172.21.15.149:0/336924314 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a801a0430 con 0x7f1a801028e0 2024-03-26T06:19:40.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.811+0000 7f1a875cd700 1 -- 172.21.15.149:0/336924314 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a80068ad0 con 0x7f1a801028e0 2024-03-26T06:19:40.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.813+0000 7f1a767fc700 1 -- 172.21.15.149:0/336924314 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1a7801a020 con 0x7f1a801028e0 2024-03-26T06:19:40.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.813+0000 7f1a767fc700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a6c0a2890 0x7f1a6c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:19:40.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.813+0000 7f1a767fc700 1 -- 172.21.15.149:0/336924314 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f1a780ce170 con 0x7f1a801028e0 2024-03-26T06:19:40.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.813+0000 7f1a85369700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a6c0a2890 0x7f1a6c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:19:40.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.814+0000 7f1a85369700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a6c0a2890 0x7f1a6c0a4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c005950 tx=0x7f1a7c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:19:40.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:40.817+0000 7f1a767fc700 1 -- 172.21.15.149:0/336924314 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1a80068ad0 con 0x7f1a801028e0 2024-03-26T06:19:41.025 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:19:41.024+0000 7f1a875cd700 1 -- 172.21.15.149:0/336924314 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a800fe480 con 0x7f1a6c0a2890 2024-03-26T06:19:41.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:41 smithi149 ceph-mon[36681]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T06:19:41.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:19:41.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:19:41.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:41.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:41 smithi037 ceph-mon[31974]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T06:19:41.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:19:41.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:19:41.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:41 smithi066 ceph-mon[36504]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T06:19:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T06:19:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T06:19:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:42.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:42 smithi149 ceph-mon[36681]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 511 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:42 smithi149 ceph-mon[36681]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:19:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:42 smithi149 ceph-mon[36681]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T06:19:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:42 smithi037 ceph-mon[31974]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 511 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:42 smithi037 ceph-mon[31974]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:19:42.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:42 smithi037 ceph-mon[31974]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T06:19:42.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:42 smithi066 ceph-mon[36504]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 511 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:42.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:42 smithi066 ceph-mon[36504]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:19:42.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:42 smithi066 ceph-mon[36504]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T06:19:43.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:43 smithi037 ceph-mon[31974]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T06:19:43.533 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:43 smithi066 ceph-mon[36504]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T06:19:43.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:43 smithi149 ceph-mon[36681]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T06:19:44.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:44 smithi149 ceph-mon[36681]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 512 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T06:19:44.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:44 smithi149 ceph-mon[36681]: from='client.? 172.21.15.149:0/2363478125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "780b3500-9d7a-48c1-8e39-ea0abb98dd5c"}]: dispatch 2024-03-26T06:19:44.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:44 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "780b3500-9d7a-48c1-8e39-ea0abb98dd5c"}]: dispatch 2024-03-26T06:19:44.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:44 smithi149 ceph-mon[36681]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "780b3500-9d7a-48c1-8e39-ea0abb98dd5c"}]': finished 2024-03-26T06:19:44.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:44 smithi149 ceph-mon[36681]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T06:19:44.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:19:44.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:44 smithi037 ceph-mon[31974]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 512 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T06:19:44.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:44 smithi037 ceph-mon[31974]: from='client.? 172.21.15.149:0/2363478125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "780b3500-9d7a-48c1-8e39-ea0abb98dd5c"}]: dispatch 2024-03-26T06:19:44.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:44 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "780b3500-9d7a-48c1-8e39-ea0abb98dd5c"}]: dispatch 2024-03-26T06:19:44.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:44 smithi037 ceph-mon[31974]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "780b3500-9d7a-48c1-8e39-ea0abb98dd5c"}]': finished 2024-03-26T06:19:44.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:44 smithi037 ceph-mon[31974]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T06:19:44.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:19:44.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:44 smithi066 ceph-mon[36504]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 512 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T06:19:44.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:44 smithi066 ceph-mon[36504]: from='client.? 172.21.15.149:0/2363478125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "780b3500-9d7a-48c1-8e39-ea0abb98dd5c"}]: dispatch 2024-03-26T06:19:44.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:44 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "780b3500-9d7a-48c1-8e39-ea0abb98dd5c"}]: dispatch 2024-03-26T06:19:44.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:44 smithi066 ceph-mon[36504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "780b3500-9d7a-48c1-8e39-ea0abb98dd5c"}]': finished 2024-03-26T06:19:44.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:44 smithi066 ceph-mon[36504]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T06:19:44.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:19:45.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:45 smithi037 ceph-mon[31974]: from='client.? 172.21.15.149:0/181192601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:19:45.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:45 smithi066 ceph-mon[36504]: from='client.? 172.21.15.149:0/181192601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:19:45.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:45 smithi149 ceph-mon[36681]: from='client.? 172.21.15.149:0/181192601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T06:19:46.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:46 smithi037 ceph-mon[31974]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-03-26T06:19:46.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:46 smithi066 ceph-mon[36504]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-03-26T06:19:46.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:46 smithi149 ceph-mon[36681]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-03-26T06:19:48.298 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:48 smithi149 ceph-mon[36681]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T06:19:48.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:48 smithi037 ceph-mon[31974]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T06:19:48.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:48 smithi066 ceph-mon[36504]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T06:19:50.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:50 smithi149 ceph-mon[36681]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-26T06:19:50.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:50 smithi066 ceph-mon[36504]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-26T06:19:50.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:50 smithi037 ceph-mon[31974]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-26T06:19:52.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:52 smithi149 ceph-mon[36681]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-26T06:19:52.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T06:19:52.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:52.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:52 smithi149 ceph-mon[36681]: Deploying daemon osd.11 on smithi149 2024-03-26T06:19:52.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:52 smithi066 ceph-mon[36504]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-26T06:19:52.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T06:19:52.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:52.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:52 smithi066 ceph-mon[36504]: Deploying daemon osd.11 on smithi149 2024-03-26T06:19:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:52 smithi037 ceph-mon[31974]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-26T06:19:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T06:19:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:19:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:52 smithi037 ceph-mon[31974]: Deploying daemon osd.11 on smithi149 2024-03-26T06:19:54.530 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:54 smithi149 ceph-mon[36681]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:54.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:54 smithi066 ceph-mon[36504]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:54 smithi037 ceph-mon[31974]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:56.332 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:56 smithi149 ceph-mon[36681]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:56.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:56 smithi066 ceph-mon[36504]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:56.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:56 smithi037 ceph-mon[31974]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:57.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:57.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:57.799 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:57 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:57 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:58.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:58.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:19:58.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:57 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:19:58.882 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:19:58 smithi149 ceph-mon[36681]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:58.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:19:58 smithi037 ceph-mon[31974]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:19:59.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:19:58 smithi066 ceph-mon[36504]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:20:00.093 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 11 on host 'smithi149' 2024-03-26T06:20:00.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.091+0000 7f1a767fc700 1 -- 172.21.15.149:0/336924314 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1a800fe480 con 0x7f1a6c0a2890 2024-03-26T06:20:00.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.094+0000 7f1a875cd700 1 -- 172.21.15.149:0/336924314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a6c0a2890 msgr2=0x7f1a6c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:00.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.094+0000 7f1a875cd700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a6c0a2890 0x7f1a6c0a4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c005950 tx=0x7f1a7c00a300 comp rx=0 tx=0).stop 2024-03-26T06:20:00.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.094+0000 7f1a875cd700 1 -- 172.21.15.149:0/336924314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a801028e0 msgr2=0x7f1a8019f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:00.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.094+0000 7f1a875cd700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a801028e0 0x7f1a8019f640 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a78007f10 tx=0x7f1a7800d3b0 comp rx=0 tx=0).stop 2024-03-26T06:20:00.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.094+0000 7f1a875cd700 1 -- 172.21.15.149:0/336924314 shutdown_connections 2024-03-26T06:20:00.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.094+0000 7f1a875cd700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a80100d40 0x7f1a801980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:00.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.094+0000 7f1a875cd700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a6c0a2890 0x7f1a6c0a4d50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:00.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.094+0000 7f1a875cd700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a801028e0 0x7f1a8019f640 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:00.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.094+0000 7f1a875cd700 1 --2- 172.21.15.149:0/336924314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a80101f40 0x7f1a80198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:00.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.094+0000 7f1a875cd700 1 -- 172.21.15.149:0/336924314 >> 172.21.15.149:0/336924314 conn(0x7f1a800fc300 msgr2=0x7f1a800fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:00.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.095+0000 7f1a875cd700 1 -- 172.21.15.149:0/336924314 shutdown_connections 2024-03-26T06:20:00.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:00.095+0000 7f1a875cd700 1 -- 172.21.15.149:0/336924314 wait complete. 2024-03-26T06:20:00.652 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:00 smithi149 ceph-mon[36681]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:20:00.652 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:00 smithi149 ceph-mon[36681]: overall HEALTH_OK 2024-03-26T06:20:00.652 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:00.652 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:00 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:00.653 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:00 smithi149 ceph-mon[36681]: from='osd.11 [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T06:20:00.653 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:00 smithi149 ceph-mon[36681]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T06:20:00.679 DEBUG:teuthology.orchestra.run.smithi149:osd.11> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.11.service 2024-03-26T06:20:00.682 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-26T06:20:00.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-26T06:20:00.899 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:00 smithi066 ceph-mon[36504]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:20:00.900 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:00 smithi066 ceph-mon[36504]: overall HEALTH_OK 2024-03-26T06:20:00.900 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:00.900 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:00 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:00.900 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:00 smithi066 ceph-mon[36504]: from='osd.11 [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T06:20:00.900 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:00 smithi066 ceph-mon[36504]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T06:20:00.940 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:20:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:00 smithi037 ceph-mon[31974]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:20:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:00 smithi037 ceph-mon[31974]: overall HEALTH_OK 2024-03-26T06:20:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:00 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:00 smithi037 ceph-mon[31974]: from='osd.11 [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T06:20:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:00 smithi037 ceph-mon[31974]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T06:20:02.055 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:01 smithi149 ceph-mon[36681]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:20:02.055 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:01 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:02.055 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:01 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:02.055 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:01 smithi149 ceph-mon[36681]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T06:20:02.055 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:01 smithi149 ceph-mon[36681]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T06:20:02.055 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:01 smithi149 ceph-mon[36681]: from='osd.11 [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:20:02.055 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:01 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:02.055 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:01 smithi149 ceph-mon[36681]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:20:02.067 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:01 smithi037 ceph-mon[31974]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:20:02.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:01 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:02.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:01 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:02.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:01 smithi037 ceph-mon[31974]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T06:20:02.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:01 smithi037 ceph-mon[31974]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T06:20:02.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:01 smithi037 ceph-mon[31974]: from='osd.11 [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:20:02.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:01 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:02.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:01 smithi037 ceph-mon[31974]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:20:02.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:01 smithi066 ceph-mon[36504]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:20:02.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:01 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:02.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:01 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:02.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:01 smithi066 ceph-mon[36504]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T06:20:02.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:01 smithi066 ceph-mon[36504]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T06:20:02.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:01 smithi066 ceph-mon[36504]: from='osd.11 [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:20:02.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:01 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:02.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:01 smithi066 ceph-mon[36504]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-26T06:20:02.409 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:20:02 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:20:02.054+0000 7fa23db37700 -1 osd.11 0 waiting for initial osdmap 2024-03-26T06:20:02.410 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:20:02 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:20:02.071+0000 7fa23a12b700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T06:20:02.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.691+0000 7f8893d7e700 1 -- 172.21.15.37:0/1205492679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f888c102f70 msgr2=0x7f888c1033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:02.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.691+0000 7f8893d7e700 1 --2- 172.21.15.37:0/1205492679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f888c102f70 0x7f888c1033f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8880009990 tx=0x7f8880009ca0 comp rx=0 tx=0).stop 2024-03-26T06:20:02.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.692+0000 7f8893d7e700 1 -- 172.21.15.37:0/1205492679 shutdown_connections 2024-03-26T06:20:02.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.692+0000 7f8893d7e700 1 --2- 172.21.15.37:0/1205492679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f888c103930 0x7f888c10a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:02.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.692+0000 7f8893d7e700 1 --2- 172.21.15.37:0/1205492679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888c101e10 0x7f888c102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:02.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.692+0000 7f8893d7e700 1 --2- 172.21.15.37:0/1205492679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f888c102f70 0x7f888c1033f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:02.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.692+0000 7f8893d7e700 1 -- 172.21.15.37:0/1205492679 >> 172.21.15.37:0/1205492679 conn(0x7f888c0fd3f0 msgr2=0x7f888c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:02.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.692+0000 7f8893d7e700 1 -- 172.21.15.37:0/1205492679 shutdown_connections 2024-03-26T06:20:02.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.692+0000 7f8893d7e700 1 -- 172.21.15.37:0/1205492679 wait complete. 2024-03-26T06:20:02.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.692+0000 7f8893d7e700 1 Processor -- start 2024-03-26T06:20:02.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.692+0000 7f8893d7e700 1 -- start start 2024-03-26T06:20:02.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8893d7e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f888c101e10 0x7f888c075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:02.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8893d7e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f888c102f70 0x7f888c0741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:02.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8893d7e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888c103930 0x7f888c074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:02.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8893d7e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c10d9e0 con 0x7f888c101e10 2024-03-26T06:20:02.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8893d7e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c10d860 con 0x7f888c102f70 2024-03-26T06:20:02.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8893d7e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c10db60 con 0x7f888c103930 2024-03-26T06:20:02.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8891b1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f888c101e10 0x7f888c075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:02.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8891b1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f888c101e10 0x7f888c075b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60476/0 (socket says 172.21.15.37:60476) 2024-03-26T06:20:02.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8891b1a700 1 -- 172.21.15.37:0/1783080393 learned_addr learned my addr 172.21.15.37:0/1783080393 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:02.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8891319700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f888c102f70 0x7f888c0741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:02.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f889231b700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888c103930 0x7f888c074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:02.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8891b1a700 1 -- 172.21.15.37:0/1783080393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888c103930 msgr2=0x7f888c074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:02.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8891b1a700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888c103930 0x7f888c074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:02.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8891b1a700 1 -- 172.21.15.37:0/1783080393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f888c102f70 msgr2=0x7f888c0741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:02.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8891b1a700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f888c102f70 0x7f888c0741d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:02.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8891b1a700 1 -- 172.21.15.37:0/1783080393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c074fc0 con 0x7f888c101e10 2024-03-26T06:20:02.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8891b1a700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f888c101e10 0x7f888c075b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f888800d980 tx=0x7f888800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:02.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f887effd700 1 -- 172.21.15.37:0/1783080393 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888016070 con 0x7f888c101e10 2024-03-26T06:20:02.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8893d7e700 1 -- 172.21.15.37:0/1783080393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888c1a2150 con 0x7f888c101e10 2024-03-26T06:20:02.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f887effd700 1 -- 172.21.15.37:0/1783080393 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8888004d10 con 0x7f888c101e10 2024-03-26T06:20:02.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f887effd700 1 -- 172.21.15.37:0/1783080393 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888005670 con 0x7f888c101e10 2024-03-26T06:20:02.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.693+0000 7f8893d7e700 1 -- 172.21.15.37:0/1783080393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888c1a2690 con 0x7f888c101e10 2024-03-26T06:20:02.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.694+0000 7f8893d7e700 1 -- 172.21.15.37:0/1783080393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888c068aa0 con 0x7f888c101e10 2024-03-26T06:20:02.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.696+0000 7f887effd700 1 -- 172.21.15.37:0/1783080393 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8888004750 con 0x7f888c101e10 2024-03-26T06:20:02.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.696+0000 7f887effd700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f88780a2890 0x7f88780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:02.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.697+0000 7f8891319700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f88780a2890 0x7f88780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:02.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.697+0000 7f887effd700 1 -- 172.21.15.37:0/1783080393 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f88880ce7c0 con 0x7f888c101e10 2024-03-26T06:20:02.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.697+0000 7f8891319700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f88780a2890 0x7f88780a4d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f888000b2e0 tx=0x7f888000b820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:02.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.700+0000 7f887effd700 1 -- 172.21.15.37:0/1783080393 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f888801f020 con 0x7f888c101e10 2024-03-26T06:20:02.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.904+0000 7f8893d7e700 1 -- 172.21.15.37:0/1783080393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f888c02dc60 con 0x7f888c101e10 2024-03-26T06:20:02.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.904+0000 7f887effd700 1 -- 172.21.15.37:0/1783080393 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f888809be20 con 0x7f888c101e10 2024-03-26T06:20:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:02.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.906+0000 7f8893d7e700 1 -- 172.21.15.37:0/1783080393 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f88780a2890 msgr2=0x7f88780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:02.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.906+0000 7f8893d7e700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f88780a2890 0x7f88780a4d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f888000b2e0 tx=0x7f888000b820 comp rx=0 tx=0).stop 2024-03-26T06:20:02.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.907+0000 7f8893d7e700 1 -- 172.21.15.37:0/1783080393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f888c101e10 msgr2=0x7f888c075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:02.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.907+0000 7f8893d7e700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f888c101e10 0x7f888c075b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f888800d980 tx=0x7f888800dc90 comp rx=0 tx=0).stop 2024-03-26T06:20:02.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.907+0000 7f8893d7e700 1 -- 172.21.15.37:0/1783080393 shutdown_connections 2024-03-26T06:20:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.907+0000 7f8893d7e700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f888c101e10 0x7f888c075b80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.907+0000 7f8893d7e700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f88780a2890 0x7f88780a4d50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.907+0000 7f8893d7e700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888c103930 0x7f888c074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.907+0000 7f8893d7e700 1 --2- 172.21.15.37:0/1783080393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f888c102f70 0x7f888c0741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.907+0000 7f8893d7e700 1 -- 172.21.15.37:0/1783080393 >> 172.21.15.37:0/1783080393 conn(0x7f888c0fd3f0 msgr2=0x7f888c106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.907+0000 7f8893d7e700 1 -- 172.21.15.37:0/1783080393 shutdown_connections 2024-03-26T06:20:02.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:02.907+0000 7f8893d7e700 1 -- 172.21.15.37:0/1783080393 wait complete. 2024-03-26T06:20:03.328 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:03 smithi149 ceph-mon[36681]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:20:03.328 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:03 smithi149 ceph-mon[36681]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T06:20:03.328 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:03 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:03.328 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:03 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:03.328 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:03 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1783080393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T06:20:03.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:03 smithi066 ceph-mon[36504]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:20:03.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:03 smithi066 ceph-mon[36504]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T06:20:03.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:03 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:03.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:03 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:03.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:03 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1783080393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T06:20:03.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:03 smithi037 ceph-mon[31974]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-26T06:20:03.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:03 smithi037 ceph-mon[31974]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T06:20:03.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:03 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:03.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:03 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:03.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:03 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1783080393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T06:20:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1711433978,"num_in_osds":12,"osd_in_since":1711433983,"num_remapped_pgs":0} 2024-03-26T06:20:04.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: purged_snaps scrub starts 2024-03-26T06:20:04.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: purged_snaps scrub ok 2024-03-26T06:20:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:20:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: osd.11 [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449] boot 2024-03-26T06:20:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T06:20:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:04 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:04.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: purged_snaps scrub starts 2024-03-26T06:20:04.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: purged_snaps scrub ok 2024-03-26T06:20:04.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:20:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: osd.11 [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449] boot 2024-03-26T06:20:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T06:20:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:04 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:04.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: purged_snaps scrub starts 2024-03-26T06:20:04.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: purged_snaps scrub ok 2024-03-26T06:20:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T06:20:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: osd.11 [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449] boot 2024-03-26T06:20:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T06:20:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:20:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:20:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:04 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:04.713 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-26T06:20:04.957 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:20:05.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:05 smithi037 ceph-mon[31974]: Detected new or changed devices on smithi149 2024-03-26T06:20:05.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:05 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi149 to 1451M 2024-03-26T06:20:05.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:05.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:05.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:05 smithi037 ceph-mon[31974]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T06:20:05.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:05.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:05.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:05.295 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:05.295 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:05 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:05.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:05 smithi066 ceph-mon[36504]: Detected new or changed devices on smithi149 2024-03-26T06:20:05.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:05 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi149 to 1451M 2024-03-26T06:20:05.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:05.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:05.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:05 smithi066 ceph-mon[36504]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T06:20:05.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:05.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:05.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:05.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:05.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:05 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:05.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:05 smithi149 ceph-mon[36681]: Detected new or changed devices on smithi149 2024-03-26T06:20:05.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:05 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi149 to 1451M 2024-03-26T06:20:05.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:05.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:05.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:05 smithi149 ceph-mon[36681]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T06:20:05.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:05.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:05.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:05.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:05.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:05 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:06.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:06 smithi037 ceph-mon[31974]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:06.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:06 smithi066 ceph-mon[36504]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:06.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:06 smithi149 ceph-mon[36681]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.682+0000 7f4b918b0700 1 -- 172.21.15.37:0/2832007921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c103430 msgr2=0x7f4b8c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.682+0000 7f4b918b0700 1 --2- 172.21.15.37:0/2832007921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c103430 0x7f4b8c103850 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00b0a0 tx=0x7f4b7c00b3b0 comp rx=0 tx=0).stop 2024-03-26T06:20:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.682+0000 7f4b918b0700 1 -- 172.21.15.37:0/2832007921 shutdown_connections 2024-03-26T06:20:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.682+0000 7f4b918b0700 1 --2- 172.21.15.37:0/2832007921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c103430 0x7f4b8c103850 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.682+0000 7f4b918b0700 1 --2- 172.21.15.37:0/2832007921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c100850 0x7f4b8c10a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:06.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.682+0000 7f4b918b0700 1 --2- 172.21.15.37:0/2832007921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b8c0ffeb0 0x7f4b8c100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.682+0000 7f4b918b0700 1 -- 172.21.15.37:0/2832007921 >> 172.21.15.37:0/2832007921 conn(0x7f4b8c0fba70 msgr2=0x7f4b8c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.682+0000 7f4b918b0700 1 -- 172.21.15.37:0/2832007921 shutdown_connections 2024-03-26T06:20:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.682+0000 7f4b918b0700 1 -- 172.21.15.37:0/2832007921 wait complete. 2024-03-26T06:20:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.682+0000 7f4b918b0700 1 Processor -- start 2024-03-26T06:20:06.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.682+0000 7f4b918b0700 1 -- start start 2024-03-26T06:20:06.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b918b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c0ffeb0 0x7f4b8c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:06.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b918b0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b8c100850 0x7f4b8c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:06.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b918b0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c103430 0x7f4b8c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:06.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b918b0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c10dae0 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b918b0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c10d960 con 0x7f4b8c100850 2024-03-26T06:20:06.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b918b0700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c10dc60 con 0x7f4b8c103430 2024-03-26T06:20:06.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c0ffeb0 0x7f4b8c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:06.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c0ffeb0 0x7f4b8c1980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60500/0 (socket says 172.21.15.37:60500) 2024-03-26T06:20:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8affd700 1 -- 172.21.15.37:0/2206790063 learned_addr learned my addr 172.21.15.37:0/2206790063 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8a7fc700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b8c100850 0x7f4b8c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8b7fe700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c103430 0x7f4b8c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8affd700 1 -- 172.21.15.37:0/2206790063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c103430 msgr2=0x7f4b8c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8affd700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c103430 0x7f4b8c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8affd700 1 -- 172.21.15.37:0/2206790063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b8c100850 msgr2=0x7f4b8c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8affd700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b8c100850 0x7f4b8c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8affd700 1 -- 172.21.15.37:0/2206790063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c19fd40 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8a7fc700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b8c100850 0x7f4b8c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:20:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8b7fe700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c103430 0x7f4b8c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:20:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b8affd700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c0ffeb0 0x7f4b8c1980e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c007dc0 tx=0x7f4b7c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.683+0000 7f4b908ae700 1 -- 172.21.15.37:0/2206790063 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c021070 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.684+0000 7f4b908ae700 1 -- 172.21.15.37:0/2206790063 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b7c01dd40 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.684+0000 7f4b918b0700 1 -- 172.21.15.37:0/2206790063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8c19ffd0 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.684+0000 7f4b908ae700 1 -- 172.21.15.37:0/2206790063 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0123f0 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.684+0000 7f4b918b0700 1 -- 172.21.15.37:0/2206790063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c1a03c0 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.685+0000 7f4b918b0700 1 -- 172.21.15.37:0/2206790063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c068aa0 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.686+0000 7f4b908ae700 1 -- 172.21.15.37:0/2206790063 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4b7c024070 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.686+0000 7f4b908ae700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4b780a2890 0x7f4b780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.686+0000 7f4b8a7fc700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4b780a2890 0x7f4b780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.686+0000 7f4b908ae700 1 -- 172.21.15.37:0/2206790063 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4b7c00f020 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.687+0000 7f4b8a7fc700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4b780a2890 0x7f4b780a4d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b740099d0 tx=0x7f4b74008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.690+0000 7f4b908ae700 1 -- 172.21.15.37:0/2206790063 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4b7c02b020 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.896+0000 7f4b918b0700 1 -- 172.21.15.37:0/2206790063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f4b8c0629f0 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.896+0000 7f4b908ae700 1 -- 172.21.15.37:0/2206790063 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f4b7c097e20 con 0x7f4b8c0ffeb0 2024-03-26T06:20:06.896 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:06.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 -- 172.21.15.37:0/2206790063 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4b780a2890 msgr2=0x7f4b780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:06.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4b780a2890 0x7f4b780a4d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b740099d0 tx=0x7f4b74008040 comp rx=0 tx=0).stop 2024-03-26T06:20:06.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 -- 172.21.15.37:0/2206790063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c0ffeb0 msgr2=0x7f4b8c1980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:06.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c0ffeb0 0x7f4b8c1980e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c007dc0 tx=0x7f4b7c007ea0 comp rx=0 tx=0).stop 2024-03-26T06:20:06.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 -- 172.21.15.37:0/2206790063 shutdown_connections 2024-03-26T06:20:06.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c0ffeb0 0x7f4b8c1980e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:06.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4b780a2890 0x7f4b780a4d50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:06.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c103430 0x7f4b8c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 --2- 172.21.15.37:0/2206790063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b8c100850 0x7f4b8c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 -- 172.21.15.37:0/2206790063 >> 172.21.15.37:0/2206790063 conn(0x7f4b8c0fba70 msgr2=0x7f4b8c106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 -- 172.21.15.37:0/2206790063 shutdown_connections 2024-03-26T06:20:06.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:06.898+0000 7f4b918b0700 1 -- 172.21.15.37:0/2206790063 wait complete. 2024-03-26T06:20:07.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:07 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2206790063' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T06:20:07.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:07 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2206790063' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T06:20:07.615 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1711434003,"num_in_osds":12,"osd_in_since":1711433983,"num_remapped_pgs":0} 2024-03-26T06:20:07.616 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T06:20:07.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:07 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2206790063' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T06:20:07.859 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:20:08.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:08 smithi149 ceph-mon[36681]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:08.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:08 smithi066 ceph-mon[36504]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:08.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:08 smithi037 ceph-mon[31974]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:09.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.560+0000 7f1cb80b2700 1 -- 172.21.15.37:0/2418266250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cb0069a90 msgr2=0x7f1cb0069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:09.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.560+0000 7f1cb80b2700 1 --2- 172.21.15.37:0/2418266250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cb0069a90 0x7f1cb0069ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40099c0 tx=0x7f1ca4009cd0 comp rx=0 tx=0).stop 2024-03-26T06:20:09.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.560+0000 7f1cb80b2700 1 -- 172.21.15.37:0/2418266250 shutdown_connections 2024-03-26T06:20:09.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.560+0000 7f1cb80b2700 1 --2- 172.21.15.37:0/2418266250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb006a430 0x7f1cb010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:09.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.560+0000 7f1cb80b2700 1 --2- 172.21.15.37:0/2418266250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cb0107e30 0x7f1cb0108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:09.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.560+0000 7f1cb80b2700 1 --2- 172.21.15.37:0/2418266250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cb0069a90 0x7f1cb0069ef0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:09.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.560+0000 7f1cb80b2700 1 -- 172.21.15.37:0/2418266250 >> 172.21.15.37:0/2418266250 conn(0x7f1cb0077c10 msgr2=0x7f1cb0078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:09.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.560+0000 7f1cb80b2700 1 -- 172.21.15.37:0/2418266250 shutdown_connections 2024-03-26T06:20:09.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.560+0000 7f1cb80b2700 1 -- 172.21.15.37:0/2418266250 wait complete. 2024-03-26T06:20:09.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb80b2700 1 Processor -- start 2024-03-26T06:20:09.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb80b2700 1 -- start start 2024-03-26T06:20:09.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb80b2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb0069a90 0x7f1cb01a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:09.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb80b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cb006a430 0x7f1cb01a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:09.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb80b2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cb0107e30 0x7f1cb019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:09.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb80b2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb0112170 con 0x7f1cb006a430 2024-03-26T06:20:09.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb80b2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb0111ff0 con 0x7f1cb0107e30 2024-03-26T06:20:09.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb80b2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb01122f0 con 0x7f1cb0069a90 2024-03-26T06:20:09.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb564d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cb006a430 0x7f1cb01a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:09.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb564d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cb006a430 0x7f1cb01a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46184/0 (socket says 172.21.15.37:46184) 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb564d700 1 -- 172.21.15.37:0/920350399 learned_addr learned my addr 172.21.15.37:0/920350399 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb5e4e700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb0069a90 0x7f1cb01a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb664f700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cb0107e30 0x7f1cb019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb564d700 1 -- 172.21.15.37:0/920350399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb0069a90 msgr2=0x7f1cb01a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb564d700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb0069a90 0x7f1cb01a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb564d700 1 -- 172.21.15.37:0/920350399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cb0107e30 msgr2=0x7f1cb019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb564d700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cb0107e30 0x7f1cb019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.561+0000 7f1cb564d700 1 -- 172.21.15.37:0/920350399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb019af10 con 0x7f1cb006a430 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.562+0000 7f1cb564d700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cb006a430 0x7f1cb01a0aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40098e0 tx=0x7f1ca40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.562+0000 7f1caeffd700 1 -- 172.21.15.37:0/920350399 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4027070 con 0x7f1cb006a430 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.562+0000 7f1caeffd700 1 -- 172.21.15.37:0/920350399 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ca40041f0 con 0x7f1cb006a430 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.562+0000 7f1caeffd700 1 -- 172.21.15.37:0/920350399 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca400f460 con 0x7f1cb006a430 2024-03-26T06:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.562+0000 7f1cb80b2700 1 -- 172.21.15.37:0/920350399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb019b1a0 con 0x7f1cb006a430 2024-03-26T06:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.562+0000 7f1cb80b2700 1 -- 172.21.15.37:0/920350399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb01a7310 con 0x7f1cb006a430 2024-03-26T06:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.563+0000 7f1cb80b2700 1 -- 172.21.15.37:0/920350399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb01034e0 con 0x7f1cb006a430 2024-03-26T06:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.565+0000 7f1caeffd700 1 -- 172.21.15.37:0/920350399 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1ca402a070 con 0x7f1cb006a430 2024-03-26T06:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.565+0000 7f1caeffd700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1c980a2890 0x7f1c980a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.565+0000 7f1cb5e4e700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1c980a2890 0x7f1c980a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:09.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.565+0000 7f1caeffd700 1 -- 172.21.15.37:0/920350399 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1ca40cf6e0 con 0x7f1cb006a430 2024-03-26T06:20:09.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.566+0000 7f1cb5e4e700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1c980a2890 0x7f1c980a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000a400 tx=0x7f1ca000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:09.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.569+0000 7f1caeffd700 1 -- 172.21.15.37:0/920350399 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1ca4013070 con 0x7f1cb006a430 2024-03-26T06:20:09.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.777+0000 7f1cb80b2700 1 -- 172.21.15.37:0/920350399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cb0068aa0 con 0x7f1cb006a430 2024-03-26T06:20:09.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.778+0000 7f1caeffd700 1 -- 172.21.15.37:0/920350399 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18382 (secure 0 0 0) 0x7f1cb0068aa0 con 0x7f1cb006a430 2024-03-26T06:20:09.779 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:09.779 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:20:04.052278+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:20:09.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 -- 172.21.15.37:0/920350399 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1c980a2890 msgr2=0x7f1c980a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1c980a2890 0x7f1c980a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000a400 tx=0x7f1ca000d040 comp rx=0 tx=0).stop 2024-03-26T06:20:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 -- 172.21.15.37:0/920350399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cb006a430 msgr2=0x7f1cb01a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cb006a430 0x7f1cb01a0aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40098e0 tx=0x7f1ca40047b0 comp rx=0 tx=0).stop 2024-03-26T06:20:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 -- 172.21.15.37:0/920350399 shutdown_connections 2024-03-26T06:20:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1c980a2890 0x7f1c980a4d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cb0107e30 0x7f1cb019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb0069a90 0x7f1cb01a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 --2- 172.21.15.37:0/920350399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cb006a430 0x7f1cb01a0aa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 -- 172.21.15.37:0/920350399 >> 172.21.15.37:0/920350399 conn(0x7f1cb0077c10 msgr2=0x7f1cb010cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 -- 172.21.15.37:0/920350399 shutdown_connections 2024-03-26T06:20:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:09.780+0000 7f1cb80b2700 1 -- 172.21.15.37:0/920350399 wait complete. 2024-03-26T06:20:10.386 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:10 smithi037 ceph-mon[31974]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:10.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:10 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/920350399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:10.414 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T06:16:17.005303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T06:20:10.415 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-26T06:20:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:10 smithi066 ceph-mon[36504]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:10 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/920350399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:10.656 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:20:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:10 smithi149 ceph-mon[36681]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:10.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:10 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/920350399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.227+0000 7f0c672c7700 1 -- 172.21.15.37:0/2897076396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c60107e10 msgr2=0x7f0c601081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.227+0000 7f0c672c7700 1 --2- 172.21.15.37:0/2897076396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c60107e10 0x7f0c601081f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0099c0 tx=0x7f0c5c009cd0 comp rx=0 tx=0).stop 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 -- 172.21.15.37:0/2897076396 shutdown_connections 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 --2- 172.21.15.37:0/2897076396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c60107e10 0x7f0c601081f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 --2- 172.21.15.37:0/2897076396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c601027f0 0x7f0c6010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 --2- 172.21.15.37:0/2897076396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c60101e30 0x7f0c601022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 -- 172.21.15.37:0/2897076396 >> 172.21.15.37:0/2897076396 conn(0x7f0c60078f10 msgr2=0x7f0c60079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 -- 172.21.15.37:0/2897076396 shutdown_connections 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 -- 172.21.15.37:0/2897076396 wait complete. 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 Processor -- start 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 -- start start 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c60101e30 0x7f0c600740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c601027f0 0x7f0c60074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c60107e10 0x7f0c60074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60112110 con 0x7f0c60101e30 2024-03-26T06:20:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60111f90 con 0x7f0c60107e10 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.228+0000 7f0c672c7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60112290 con 0x7f0c601027f0 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c65063700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c60101e30 0x7f0c600740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c65063700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c60101e30 0x7f0c600740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46206/0 (socket says 172.21.15.37:46206) 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c65063700 1 -- 172.21.15.37:0/96059449 learned_addr learned my addr 172.21.15.37:0/96059449 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c64862700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c601027f0 0x7f0c60074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c65864700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c60107e10 0x7f0c60074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c65063700 1 -- 172.21.15.37:0/96059449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c601027f0 msgr2=0x7f0c60074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c65063700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c601027f0 0x7f0c60074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c65063700 1 -- 172.21.15.37:0/96059449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c60107e10 msgr2=0x7f0c60074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c65063700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c60107e10 0x7f0c60074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c65063700 1 -- 172.21.15.37:0/96059449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c601a6a30 con 0x7f0c60101e30 2024-03-26T06:20:12.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c64862700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c601027f0 0x7f0c60074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:20:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c65063700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c60101e30 0x7f0c600740e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009fd0 tx=0x7f0c5c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c567fc700 1 -- 172.21.15.37:0/96059449 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c027070 con 0x7f0c60101e30 2024-03-26T06:20:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c567fc700 1 -- 172.21.15.37:0/96059449 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c5c004150 con 0x7f0c60101e30 2024-03-26T06:20:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c672c7700 1 -- 172.21.15.37:0/96059449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c601a6cc0 con 0x7f0c60101e30 2024-03-26T06:20:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c567fc700 1 -- 172.21.15.37:0/96059449 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00f460 con 0x7f0c60101e30 2024-03-26T06:20:12.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.229+0000 7f0c672c7700 1 -- 172.21.15.37:0/96059449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c601a7110 con 0x7f0c60101e30 2024-03-26T06:20:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.230+0000 7f0c672c7700 1 -- 172.21.15.37:0/96059449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c60068aa0 con 0x7f0c60101e30 2024-03-26T06:20:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.231+0000 7f0c567fc700 1 -- 172.21.15.37:0/96059449 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0c5c02a070 con 0x7f0c60101e30 2024-03-26T06:20:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.232+0000 7f0c567fc700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0c4c0a2890 0x7f0c4c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.232+0000 7f0c64862700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0c4c0a2890 0x7f0c4c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.232+0000 7f0c567fc700 1 -- 172.21.15.37:0/96059449 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0c5c0cf5a0 con 0x7f0c60101e30 2024-03-26T06:20:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.232+0000 7f0c64862700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0c4c0a2890 0x7f0c4c0a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0c600794a0 tx=0x7f0c50005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:12.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.236+0000 7f0c567fc700 1 -- 172.21.15.37:0/96059449 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0c5c05e260 con 0x7f0c60101e30 2024-03-26T06:20:12.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.440+0000 7f0c672c7700 1 -- 172.21.15.37:0/96059449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c6002d830 con 0x7f0c60101e30 2024-03-26T06:20:12.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.441+0000 7f0c567fc700 1 -- 172.21.15.37:0/96059449 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0c5c013070 con 0x7f0c60101e30 2024-03-26T06:20:12.442 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-26T06:20:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 -- 172.21.15.37:0/96059449 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0c4c0a2890 msgr2=0x7f0c4c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0c4c0a2890 0x7f0c4c0a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0c600794a0 tx=0x7f0c50005f50 comp rx=0 tx=0).stop 2024-03-26T06:20:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 -- 172.21.15.37:0/96059449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c60101e30 msgr2=0x7f0c600740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c60101e30 0x7f0c600740e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009fd0 tx=0x7f0c5c003730 comp rx=0 tx=0).stop 2024-03-26T06:20:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 -- 172.21.15.37:0/96059449 shutdown_connections 2024-03-26T06:20:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c60101e30 0x7f0c600740e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0c4c0a2890 0x7f0c4c0a4d50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c60107e10 0x7f0c60074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 --2- 172.21.15.37:0/96059449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c601027f0 0x7f0c60074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 -- 172.21.15.37:0/96059449 >> 172.21.15.37:0/96059449 conn(0x7f0c60078f10 msgr2=0x7f0c601009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 -- 172.21.15.37:0/96059449 shutdown_connections 2024-03-26T06:20:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:12.443+0000 7f0c672c7700 1 -- 172.21.15.37:0/96059449 wait complete. 2024-03-26T06:20:12.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:12 smithi066 ceph-mon[36504]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:12.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:12 smithi149 ceph-mon[36681]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:12.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:12 smithi037 ceph-mon[31974]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:13.286 INFO:tasks.cephadm:Adding mds.a on smithi037 2024-03-26T06:20:13.287 INFO:tasks.cephadm:Adding mds.d on smithi037 2024-03-26T06:20:13.287 INFO:tasks.cephadm:Adding mds.g on smithi037 2024-03-26T06:20:13.287 INFO:tasks.cephadm:Adding mds.j on smithi037 2024-03-26T06:20:13.287 INFO:tasks.cephadm:Adding mds.b on smithi066 2024-03-26T06:20:13.287 INFO:tasks.cephadm:Adding mds.e on smithi066 2024-03-26T06:20:13.287 INFO:tasks.cephadm:Adding mds.h on smithi066 2024-03-26T06:20:13.287 INFO:tasks.cephadm:Adding mds.k on smithi066 2024-03-26T06:20:13.287 INFO:tasks.cephadm:Adding mds.c on smithi149 2024-03-26T06:20:13.287 INFO:tasks.cephadm:Adding mds.f on smithi149 2024-03-26T06:20:13.287 INFO:tasks.cephadm:Adding mds.i on smithi149 2024-03-26T06:20:13.287 INFO:tasks.cephadm:Adding mds.l on smithi149 2024-03-26T06:20:13.287 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph orch apply mds all '12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l' 2024-03-26T06:20:13.538 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:20:13.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:13 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/96059449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:20:13.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:13 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/96059449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:20:13.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:13 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/96059449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:20:14.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:14 smithi066 ceph-mon[36504]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:14.637 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:14 smithi149 ceph-mon[36681]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:14.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:14 smithi037 ceph-mon[31974]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:14.972 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 -- 172.21.15.149:0/821463304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9128102900 msgr2=0x7f91281091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:14.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 --2- 172.21.15.149:0/821463304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9128102900 0x7f91281091b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f912000b0a0 tx=0x7f912000b3b0 comp rx=0 tx=0).stop 2024-03-26T06:20:14.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 -- 172.21.15.149:0/821463304 shutdown_connections 2024-03-26T06:20:14.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 --2- 172.21.15.149:0/821463304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9128100d60 0x7f9128101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:14.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 --2- 172.21.15.149:0/821463304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9128102900 0x7f91281091b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:14.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 --2- 172.21.15.149:0/821463304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9128101f60 0x7f91281023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:14.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 -- 172.21.15.149:0/821463304 >> 172.21.15.149:0/821463304 conn(0x7f91280fc300 msgr2=0x7f91280fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:14.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 -- 172.21.15.149:0/821463304 shutdown_connections 2024-03-26T06:20:14.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 -- 172.21.15.149:0/821463304 wait complete. 2024-03-26T06:20:14.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 Processor -- start 2024-03-26T06:20:14.973 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 -- start start 2024-03-26T06:20:14.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9128100d60 0x7f9128197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:14.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9128101f60 0x7f9128198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:14.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9128102900 0x7f912819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:14.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912810da90 con 0x7f9128102900 2024-03-26T06:20:14.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912810d910 con 0x7f9128100d60 2024-03-26T06:20:14.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.969+0000 7f91304cf700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912810dc10 con 0x7f9128101f60 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f912da6a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9128101f60 0x7f9128198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f912da6a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9128101f60 0x7f9128198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:33264/0 (socket says 172.21.15.149:33264) 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f912da6a700 1 -- 172.21.15.149:0/2579298110 learned_addr learned my addr 172.21.15.149:0/2579298110 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f912e26b700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9128100d60 0x7f9128197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f912ea6c700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9128102900 0x7f912819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f912da6a700 1 -- 172.21.15.149:0/2579298110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9128100d60 msgr2=0x7f9128197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f912da6a700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9128100d60 0x7f9128197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f912da6a700 1 -- 172.21.15.149:0/2579298110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9128102900 msgr2=0x7f912819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f912da6a700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9128102900 0x7f912819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f912da6a700 1 -- 172.21.15.149:0/2579298110 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912819fc40 con 0x7f9128101f60 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f912da6a700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9128101f60 0x7f9128198520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f911800b560 tx=0x7f911800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f911f7fe700 1 -- 172.21.15.149:0/2579298110 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118015020 con 0x7f9128101f60 2024-03-26T06:20:14.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f911f7fe700 1 -- 172.21.15.149:0/2579298110 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9118011d10 con 0x7f9128101f60 2024-03-26T06:20:14.976 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f91304cf700 1 -- 172.21.15.149:0/2579298110 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912819ff30 con 0x7f9128101f60 2024-03-26T06:20:14.976 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f911f7fe700 1 -- 172.21.15.149:0/2579298110 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118010e50 con 0x7f9128101f60 2024-03-26T06:20:14.976 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.970+0000 7f91304cf700 1 -- 172.21.15.149:0/2579298110 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91281a03e0 con 0x7f9128101f60 2024-03-26T06:20:14.976 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.971+0000 7f91304cf700 1 -- 172.21.15.149:0/2579298110 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9128068aa0 con 0x7f9128101f60 2024-03-26T06:20:14.977 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.972+0000 7f911f7fe700 1 -- 172.21.15.149:0/2579298110 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9118018020 con 0x7f9128101f60 2024-03-26T06:20:14.977 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.973+0000 7f911f7fe700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f91140a2a30 0x7f91140a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:14.977 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.973+0000 7f911f7fe700 1 -- 172.21.15.149:0/2579298110 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f91180ce260 con 0x7f9128101f60 2024-03-26T06:20:14.977 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.974+0000 7f912e26b700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f91140a2a30 0x7f91140a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:14.977 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.974+0000 7f912e26b700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f91140a2a30 0x7f91140a4ef0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9128101dc0 tx=0x7f912400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:14.978 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:14.977+0000 7f911f7fe700 1 -- 172.21.15.149:0/2579298110 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f911801f020 con 0x7f9128101f60 2024-03-26T06:20:15.182 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.181+0000 7f91304cf700 1 -- 172.21.15.149:0/2579298110 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f9128062420 con 0x7f91140a2a30 2024-03-26T06:20:15.190 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.188+0000 7f911f7fe700 1 -- 172.21.15.149:0/2579298110 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f9128062420 con 0x7f91140a2a30 2024-03-26T06:20:15.190 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mds.all update... 2024-03-26T06:20:15.193 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.191+0000 7f91304cf700 1 -- 172.21.15.149:0/2579298110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f91140a2a30 msgr2=0x7f91140a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:15.193 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.191+0000 7f91304cf700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f91140a2a30 0x7f91140a4ef0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9128101dc0 tx=0x7f912400b410 comp rx=0 tx=0).stop 2024-03-26T06:20:15.193 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.191+0000 7f91304cf700 1 -- 172.21.15.149:0/2579298110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9128101f60 msgr2=0x7f9128198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:15.193 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.191+0000 7f91304cf700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9128101f60 0x7f9128198520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f911800b560 tx=0x7f911800b870 comp rx=0 tx=0).stop 2024-03-26T06:20:15.193 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.191+0000 7f91304cf700 1 -- 172.21.15.149:0/2579298110 shutdown_connections 2024-03-26T06:20:15.193 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.191+0000 7f91304cf700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9128102900 0x7f912819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:15.193 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.191+0000 7f91304cf700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f91140a2a30 0x7f91140a4ef0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:15.193 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.191+0000 7f91304cf700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9128100d60 0x7f9128197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:15.194 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.191+0000 7f91304cf700 1 --2- 172.21.15.149:0/2579298110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9128101f60 0x7f9128198520 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:15.194 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.191+0000 7f91304cf700 1 -- 172.21.15.149:0/2579298110 >> 172.21.15.149:0/2579298110 conn(0x7f91280fc300 msgr2=0x7f9128105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:15.194 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.192+0000 7f91304cf700 1 -- 172.21.15.149:0/2579298110 shutdown_connections 2024-03-26T06:20:15.194 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-26T06:20:15.192+0000 7f91304cf700 1 -- 172.21.15.149:0/2579298110 wait complete. 2024-03-26T06:20:15.793 DEBUG:teuthology.orchestra.run.smithi037:mds.a> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.a.service 2024-03-26T06:20:15.796 DEBUG:teuthology.orchestra.run.smithi037:mds.d> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.d.service 2024-03-26T06:20:15.839 DEBUG:teuthology.orchestra.run.smithi037:mds.g> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.g.service 2024-03-26T06:20:15.882 DEBUG:teuthology.orchestra.run.smithi037:mds.j> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.j.service 2024-03-26T06:20:15.925 DEBUG:teuthology.orchestra.run.smithi066:mds.b> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.b.service 2024-03-26T06:20:15.928 DEBUG:teuthology.orchestra.run.smithi066:mds.e> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.e.service 2024-03-26T06:20:15.971 DEBUG:teuthology.orchestra.run.smithi066:mds.h> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.h.service 2024-03-26T06:20:16.015 DEBUG:teuthology.orchestra.run.smithi066:mds.k> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.k.service 2024-03-26T06:20:16.059 DEBUG:teuthology.orchestra.run.smithi149:mds.c> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.c.service 2024-03-26T06:20:16.061 DEBUG:teuthology.orchestra.run.smithi149:mds.f> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.f.service 2024-03-26T06:20:16.104 DEBUG:teuthology.orchestra.run.smithi149:mds.i> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.i.service 2024-03-26T06:20:16.147 DEBUG:teuthology.orchestra.run.smithi149:mds.l> sudo journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.l.service 2024-03-26T06:20:16.190 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-26T06:20:16.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:20:16.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.310+0000 7fa313945640 1 -- 172.21.15.37:0/2006570858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa30c104720 msgr2=0x7fa30c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.310+0000 7fa313945640 1 --2- 172.21.15.37:0/2006570858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa30c104720 0x7fa30c106b10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009900 tx=0x7fa2fc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.310+0000 7fa313945640 1 -- 172.21.15.37:0/2006570858 shutdown_connections 2024-03-26T06:20:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.310+0000 7fa313945640 1 --2- 172.21.15.37:0/2006570858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c107050 0x7fa30c109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.310+0000 7fa313945640 1 --2- 172.21.15.37:0/2006570858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa30c104720 0x7fa30c106b10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.310+0000 7fa313945640 1 --2- 172.21.15.37:0/2006570858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa30c101df0 0x7fa30c1041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.310+0000 7fa313945640 1 -- 172.21.15.37:0/2006570858 >> 172.21.15.37:0/2006570858 conn(0x7fa30c0fb780 msgr2=0x7fa30c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.310+0000 7fa313945640 1 -- 172.21.15.37:0/2006570858 shutdown_connections 2024-03-26T06:20:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.310+0000 7fa313945640 1 -- 172.21.15.37:0/2006570858 wait complete. 2024-03-26T06:20:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa313945640 1 Processor -- start 2024-03-26T06:20:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa313945640 1 -- start start 2024-03-26T06:20:16.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa313945640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c101df0 0x7fa30c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa313945640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa30c104720 0x7fa30c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa313945640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa30c107050 0x7fa30c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa313945640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c10cfa0 con 0x7fa30c104720 2024-03-26T06:20:16.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa313945640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c10ce20 con 0x7fa30c107050 2024-03-26T06:20:16.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa313945640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c10d120 con 0x7fa30c101df0 2024-03-26T06:20:16.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa310ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa30c104720 0x7fa30c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa310ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa30c104720 0x7fa30c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46218/0 (socket says 172.21.15.37:46218) 2024-03-26T06:20:16.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa310ee0640 1 -- 172.21.15.37:0/3644102651 learned_addr learned my addr 172.21.15.37:0/3644102651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:16.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa3116e1640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c101df0 0x7fa30c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa311ee2640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa30c107050 0x7fa30c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa310ee0640 1 -- 172.21.15.37:0/3644102651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c101df0 msgr2=0x7fa30c197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa310ee0640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c101df0 0x7fa30c197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa310ee0640 1 -- 172.21.15.37:0/3644102651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa30c107050 msgr2=0x7fa30c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa310ee0640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa30c107050 0x7fa30c19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.311+0000 7fa310ee0640 1 -- 172.21.15.37:0/3644102651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c19f240 con 0x7fa30c104720 2024-03-26T06:20:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.312+0000 7fa310ee0640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa30c104720 0x7fa30c197b60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc000c00 tx=0x7fa2fc02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.312+0000 7fa3027fc640 1 -- 172.21.15.37:0/3644102651 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc041070 con 0x7fa30c104720 2024-03-26T06:20:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.312+0000 7fa3027fc640 1 -- 172.21.15.37:0/3644102651 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2fc004450 con 0x7fa30c104720 2024-03-26T06:20:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.312+0000 7fa313945640 1 -- 172.21.15.37:0/3644102651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30c19f4d0 con 0x7fa30c104720 2024-03-26T06:20:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.312+0000 7fa3027fc640 1 -- 172.21.15.37:0/3644102651 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc034440 con 0x7fa30c104720 2024-03-26T06:20:16.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.312+0000 7fa313945640 1 -- 172.21.15.37:0/3644102651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30c19f9c0 con 0x7fa30c104720 2024-03-26T06:20:16.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.313+0000 7fa313945640 1 -- 172.21.15.37:0/3644102651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30c0fd270 con 0x7fa30c104720 2024-03-26T06:20:16.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.314+0000 7fa3027fc640 1 -- 172.21.15.37:0/3644102651 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa2fc044070 con 0x7fa30c104720 2024-03-26T06:20:16.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.314+0000 7fa3027fc640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2f80a2800 0x7fa2f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.314+0000 7fa3027fc640 1 -- 172.21.15.37:0/3644102651 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa2fc0f0290 con 0x7fa30c104720 2024-03-26T06:20:16.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.315+0000 7fa3116e1640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2f80a2800 0x7fa2f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.315+0000 7fa3116e1640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2f80a2800 0x7fa2f80a4cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa308005fd0 tx=0x7fa308009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:16.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.318+0000 7fa3027fc640 1 -- 172.21.15.37:0/3644102651 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa2fc0bd210 con 0x7fa30c104720 2024-03-26T06:20:16.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:16 smithi066 ceph-mon[36504]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:16.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:16 smithi066 ceph-mon[36504]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:20:16.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:16 smithi066 ceph-mon[36504]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-03-26T06:20:16.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:16.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:16.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:16.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:16.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:16.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:16.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:16.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:16 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:16.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:16 smithi149 ceph-mon[36681]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:16.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:16 smithi149 ceph-mon[36681]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:20:16.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:16 smithi149 ceph-mon[36681]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-03-26T06:20:16.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:16.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:16.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:16.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:16.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:16.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:16.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:16.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:16 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:16.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:16 smithi037 ceph-mon[31974]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:16.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:16 smithi037 ceph-mon[31974]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:20:16.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:16 smithi037 ceph-mon[31974]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-03-26T06:20:16.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:16.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:16.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:16.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:16.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:16.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:16.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:16.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:16 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:16.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.502+0000 7fa313945640 1 -- 172.21.15.37:0/3644102651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa30c02d040 con 0x7fa30c104720 2024-03-26T06:20:16.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.503+0000 7fa3027fc640 1 -- 172.21.15.37:0/3644102651 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18382 (secure 0 0 0) 0x7fa2fc0c12b0 con 0x7fa30c104720 2024-03-26T06:20:16.504 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:16.504 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:20:04.052278+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:20:16.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.504+0000 7fa313945640 1 -- 172.21.15.37:0/3644102651 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2f80a2800 msgr2=0x7fa2f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.504+0000 7fa313945640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2f80a2800 0x7fa2f80a4cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa308005fd0 tx=0x7fa308009500 comp rx=0 tx=0).stop 2024-03-26T06:20:16.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.504+0000 7fa313945640 1 -- 172.21.15.37:0/3644102651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa30c104720 msgr2=0x7fa30c197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.504+0000 7fa313945640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa30c104720 0x7fa30c197b60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc000c00 tx=0x7fa2fc02c640 comp rx=0 tx=0).stop 2024-03-26T06:20:16.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.505+0000 7fa313945640 1 -- 172.21.15.37:0/3644102651 shutdown_connections 2024-03-26T06:20:16.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.505+0000 7fa313945640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2f80a2800 0x7fa2f80a4cc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.505+0000 7fa313945640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa30c107050 0x7fa30c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.505+0000 7fa313945640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa30c104720 0x7fa30c197b60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.505+0000 7fa313945640 1 --2- 172.21.15.37:0/3644102651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c101df0 0x7fa30c197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.505+0000 7fa313945640 1 -- 172.21.15.37:0/3644102651 >> 172.21.15.37:0/3644102651 conn(0x7fa30c0fb780 msgr2=0x7fa30c102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:16.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.505+0000 7fa313945640 1 -- 172.21.15.37:0/3644102651 shutdown_connections 2024-03-26T06:20:16.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.505+0000 7fa313945640 1 -- 172.21.15.37:0/3644102651 wait complete. 2024-03-26T06:20:16.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T06:16:17.005303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T06:20:16.517 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T06:20:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.622+0000 7f8a256fe640 1 -- 172.21.15.37:0/1053077580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a20101d90 msgr2=0x7f8a20104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.622+0000 7f8a256fe640 1 --2- 172.21.15.37:0/1053077580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a20101d90 0x7f8a20104180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009900 tx=0x7f8a1002bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.622+0000 7f8a256fe640 1 -- 172.21.15.37:0/1053077580 shutdown_connections 2024-03-26T06:20:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.622+0000 7f8a256fe640 1 --2- 172.21.15.37:0/1053077580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a20106ff0 0x7f8a20109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.622+0000 7f8a256fe640 1 --2- 172.21.15.37:0/1053077580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a201046c0 0x7f8a20106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.622+0000 7f8a256fe640 1 --2- 172.21.15.37:0/1053077580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a20101d90 0x7f8a20104180 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.622+0000 7f8a256fe640 1 -- 172.21.15.37:0/1053077580 >> 172.21.15.37:0/1053077580 conn(0x7f8a200fb7a0 msgr2=0x7f8a200fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.622+0000 7f8a256fe640 1 -- 172.21.15.37:0/1053077580 shutdown_connections 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.622+0000 7f8a256fe640 1 -- 172.21.15.37:0/1053077580 wait complete. 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a256fe640 1 Processor -- start 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a256fe640 1 -- start start 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a256fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a20101d90 0x7f8a201976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a256fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a201046c0 0x7f8a20197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a256fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a20106ff0 0x7f8a2019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a256fe640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2010d000 con 0x7f8a201046c0 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a256fe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2010ce80 con 0x7f8a20101d90 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a256fe640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2010d180 con 0x7f8a20106ff0 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a201046c0 0x7f8a20197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a201046c0 0x7f8a20197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46230/0 (socket says 172.21.15.37:46230) 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a17fff640 1 -- 172.21.15.37:0/3384890920 learned_addr learned my addr 172.21.15.37:0/3384890920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a1effd640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a20101d90 0x7f8a201976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a1f7fe640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a20106ff0 0x7f8a2019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a17fff640 1 -- 172.21.15.37:0/3384890920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a20106ff0 msgr2=0x7f8a2019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a17fff640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a20106ff0 0x7f8a2019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a17fff640 1 -- 172.21.15.37:0/3384890920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a20101d90 msgr2=0x7f8a201976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a17fff640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a20101d90 0x7f8a201976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a17fff640 1 -- 172.21.15.37:0/3384890920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2019f2f0 con 0x7f8a201046c0 2024-03-26T06:20:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a1f7fe640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a20106ff0 0x7f8a2019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:20:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a17fff640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a201046c0 0x7f8a20197c10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800c970 tx=0x7f8a0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a1cff9640 1 -- 172.21.15.37:0/3384890920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08007bb0 con 0x7f8a201046c0 2024-03-26T06:20:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a1cff9640 1 -- 172.21.15.37:0/3384890920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a08007d50 con 0x7f8a201046c0 2024-03-26T06:20:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.623+0000 7f8a256fe640 1 -- 172.21.15.37:0/3384890920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2019f5e0 con 0x7f8a201046c0 2024-03-26T06:20:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.624+0000 7f8a1cff9640 1 -- 172.21.15.37:0/3384890920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800f450 con 0x7f8a201046c0 2024-03-26T06:20:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.624+0000 7f8a256fe640 1 -- 172.21.15.37:0/3384890920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2019fb80 con 0x7f8a201046c0 2024-03-26T06:20:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.624+0000 7f8a256fe640 1 -- 172.21.15.37:0/3384890920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a200fd290 con 0x7f8a201046c0 2024-03-26T06:20:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.625+0000 7f8a1cff9640 1 -- 172.21.15.37:0/3384890920 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8a080046c0 con 0x7f8a201046c0 2024-03-26T06:20:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.626+0000 7f8a1cff9640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8a0c0a2880 0x7f8a0c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.626+0000 7f8a1effd640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8a0c0a2880 0x7f8a0c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.626+0000 7f8a1cff9640 1 -- 172.21.15.37:0/3384890920 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8a080ce6e0 con 0x7f8a201046c0 2024-03-26T06:20:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.626+0000 7f8a1effd640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8a0c0a2880 0x7f8a0c0a4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8a10006010 tx=0x7f8a10000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.630+0000 7f8a1cff9640 1 -- 172.21.15.37:0/3384890920 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8a08014030 con 0x7f8a201046c0 2024-03-26T06:20:16.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.813+0000 7f8a256fe640 1 -- 172.21.15.37:0/3384890920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a20061f20 con 0x7f8a201046c0 2024-03-26T06:20:16.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.813+0000 7f8a1cff9640 1 -- 172.21.15.37:0/3384890920 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a0809bb80 con 0x7f8a201046c0 2024-03-26T06:20:16.815 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-26T06:20:16.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.815+0000 7f8a256fe640 1 -- 172.21.15.37:0/3384890920 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8a0c0a2880 msgr2=0x7f8a0c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.815+0000 7f8a256fe640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8a0c0a2880 0x7f8a0c0a4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8a10006010 tx=0x7f8a10000bc0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.815+0000 7f8a256fe640 1 -- 172.21.15.37:0/3384890920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a201046c0 msgr2=0x7f8a20197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.815+0000 7f8a256fe640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a201046c0 0x7f8a20197c10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800c970 tx=0x7f8a0800ce30 comp rx=0 tx=0).stop 2024-03-26T06:20:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.815+0000 7f8a256fe640 1 -- 172.21.15.37:0/3384890920 shutdown_connections 2024-03-26T06:20:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.815+0000 7f8a256fe640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8a0c0a2880 0x7f8a0c0a4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.815+0000 7f8a256fe640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a20106ff0 0x7f8a2019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.815+0000 7f8a256fe640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a201046c0 0x7f8a20197c10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.815+0000 7f8a256fe640 1 --2- 172.21.15.37:0/3384890920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a20101d90 0x7f8a201976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.815+0000 7f8a256fe640 1 -- 172.21.15.37:0/3384890920 >> 172.21.15.37:0/3384890920 conn(0x7f8a200fb7a0 msgr2=0x7f8a201026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.816+0000 7f8a256fe640 1 -- 172.21.15.37:0/3384890920 shutdown_connections 2024-03-26T06:20:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.816+0000 7f8a256fe640 1 -- 172.21.15.37:0/3384890920 wait complete. 2024-03-26T06:20:16.828 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:16.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.929+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2548236283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c14101480 msgr2=0x7f5c14101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.929+0000 7f5c1a64b640 1 --2- 172.21.15.37:0/2548236283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c14101480 0x7f5c14101900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009900 tx=0x7f5c0402bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:16.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.929+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2548236283 shutdown_connections 2024-03-26T06:20:16.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.929+0000 7f5c1a64b640 1 --2- 172.21.15.37:0/2548236283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c14101e40 0x7f5c141086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.929+0000 7f5c1a64b640 1 --2- 172.21.15.37:0/2548236283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c14101480 0x7f5c14101900 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.929+0000 7f5c1a64b640 1 --2- 172.21.15.37:0/2548236283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c14100290 0x7f5c141006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.929+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2548236283 >> 172.21.15.37:0/2548236283 conn(0x7f5c140fb820 msgr2=0x7f5c140fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:16.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.929+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2548236283 shutdown_connections 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2548236283 wait complete. 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c1a64b640 1 Processor -- start 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c1a64b640 1 -- start start 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c1a64b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c14100290 0x7f5c14197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c1a64b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c14101480 0x7f5c14197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c1a64b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c14101e40 0x7f5c1419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c1a64b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1410bf00 con 0x7f5c14100290 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c1a64b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1410bd80 con 0x7f5c14101480 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c1a64b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1410c080 con 0x7f5c14101e40 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c13fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c14100290 0x7f5c14197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c13fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c14100290 0x7f5c14197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46262/0 (socket says 172.21.15.37:46262) 2024-03-26T06:20:16.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.930+0000 7f5c13fff640 1 -- 172.21.15.37:0/2510543508 learned_addr learned my addr 172.21.15.37:0/2510543508 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:16.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c137fe640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c14101480 0x7f5c14197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c18be8640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c14101e40 0x7f5c1419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c13fff640 1 -- 172.21.15.37:0/2510543508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c14101e40 msgr2=0x7f5c1419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c13fff640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c14101e40 0x7f5c1419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c13fff640 1 -- 172.21.15.37:0/2510543508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c14101480 msgr2=0x7f5c14197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:16.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c13fff640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c14101480 0x7f5c14197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:16.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c13fff640 1 -- 172.21.15.37:0/2510543508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1419f320 con 0x7f5c14100290 2024-03-26T06:20:16.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c137fe640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c14101480 0x7f5c14197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:20:16.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c13fff640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c14100290 0x7f5c14197700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800d950 tx=0x7f5c0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:16.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c117fa640 1 -- 172.21.15.37:0/2510543508 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c08017020 con 0x7f5c14100290 2024-03-26T06:20:16.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c117fa640 1 -- 172.21.15.37:0/2510543508 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c08009c30 con 0x7f5c14100290 2024-03-26T06:20:16.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c117fa640 1 -- 172.21.15.37:0/2510543508 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800f740 con 0x7f5c14100290 2024-03-26T06:20:16.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2510543508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1419f610 con 0x7f5c14100290 2024-03-26T06:20:16.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.931+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2510543508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1419fb30 con 0x7f5c14100290 2024-03-26T06:20:16.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.932+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2510543508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1419fdf0 con 0x7f5c14100290 2024-03-26T06:20:16.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.933+0000 7f5c117fa640 1 -- 172.21.15.37:0/2510543508 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5c0801a020 con 0x7f5c14100290 2024-03-26T06:20:16.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.934+0000 7f5c117fa640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5bfc0a2800 0x7f5bfc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:16.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.934+0000 7f5c137fe640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5bfc0a2800 0x7f5bfc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:16.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.934+0000 7f5c117fa640 1 -- 172.21.15.37:0/2510543508 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5c080ce090 con 0x7f5c14100290 2024-03-26T06:20:16.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.934+0000 7f5c137fe640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5bfc0a2800 0x7f5bfc0a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006010 tx=0x7f5c0402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:16.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:16.939+0000 7f5c117fa640 1 -- 172.21.15.37:0/2510543508 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5c08021070 con 0x7f5c14100290 2024-03-26T06:20:17.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.168+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2510543508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c1419ffc0 con 0x7f5c14100290 2024-03-26T06:20:17.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.168+0000 7f5c117fa640 1 -- 172.21.15.37:0/2510543508 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f5c08009de0 con 0x7f5c14100290 2024-03-26T06:20:17.169 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:17.169 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-26T06:20:17.169 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 1 2024-03-26T06:20:17.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2510543508 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5bfc0a2800 msgr2=0x7f5bfc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:17.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5bfc0a2800 0x7f5bfc0a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006010 tx=0x7f5c0402df90 comp rx=0 tx=0).stop 2024-03-26T06:20:17.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2510543508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c14100290 msgr2=0x7f5c14197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:17.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c14100290 0x7f5c14197700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800d950 tx=0x7f5c0800de10 comp rx=0 tx=0).stop 2024-03-26T06:20:17.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2510543508 shutdown_connections 2024-03-26T06:20:17.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5bfc0a2800 0x7f5bfc0a4cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:17.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c14101e40 0x7f5c1419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:17.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c14101480 0x7f5c14197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:17.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 --2- 172.21.15.37:0/2510543508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c14100290 0x7f5c14197700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:17.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2510543508 >> 172.21.15.37:0/2510543508 conn(0x7f5c140fb820 msgr2=0x7f5c140fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:17.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2510543508 shutdown_connections 2024-03-26T06:20:17.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:17.170+0000 7f5c1a64b640 1 -- 172.21.15.37:0/2510543508 wait complete. 2024-03-26T06:20:17.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:17 smithi037 ceph-mon[31974]: Deploying daemon mds.l on smithi149 2024-03-26T06:20:17.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:17 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3644102651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:17.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:17 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3384890920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:20:17.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:17 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2510543508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:17.511 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:17 smithi149 ceph-mon[36681]: Deploying daemon mds.l on smithi149 2024-03-26T06:20:17.511 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:17 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3644102651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:17.511 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:17 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3384890920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:20:17.512 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:17 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2510543508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:17.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:17 smithi066 ceph-mon[36504]: Deploying daemon mds.l on smithi149 2024-03-26T06:20:17.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:17 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3644102651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:17.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:17 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3384890920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:20:17.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:17 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2510543508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:18.159 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:20:18 smithi149 systemd[1]: Starting Ceph mds.l for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:20:18.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:18 smithi037 ceph-mon[31974]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:18.500 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:18 smithi149 ceph-mon[36681]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:18.501 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:20:18 smithi149 podman[67120]: 2024-03-26 06:20:18.127436974 +0000 UTC m=+0.018021706 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:20:18.501 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:20:18 smithi149 podman[67120]: 2024-03-26 06:20:18.24766086 +0000 UTC m=+0.138245574 container create 2379c8bfc0c221b16b2b1ef287ba7cd929af06257051c29d05f0d28bda9d8efa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-26T06:20:18.501 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:20:18 smithi149 podman[67120]: 2024-03-26 06:20:18.500906534 +0000 UTC m=+0.391491255 container init 2379c8bfc0c221b16b2b1ef287ba7cd929af06257051c29d05f0d28bda9d8efa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-26T06:20:18.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:18 smithi066 ceph-mon[36504]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:18.910 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:20:18 smithi149 podman[67120]: 2024-03-26 06:20:18.503821864 +0000 UTC m=+0.394406579 container start 2379c8bfc0c221b16b2b1ef287ba7cd929af06257051c29d05f0d28bda9d8efa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2) 2024-03-26T06:20:18.910 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:20:18 smithi149 ceph-mds[67139]: -- 172.21.15.149:0/508363960 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ead2ce9a40 con 0x55ead2cbc800 2024-03-26T06:20:18.910 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:20:18 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l[67135]: starting mds.l at 2024-03-26T06:20:18.910 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:20:18 smithi149 bash[67120]: 2379c8bfc0c221b16b2b1ef287ba7cd929af06257051c29d05f0d28bda9d8efa 2024-03-26T06:20:18.910 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:20:18 smithi149 systemd[1]: Started Ceph mds.l for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:19.184 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:19.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.298+0000 7f3633afc640 1 -- 172.21.15.37:0/1185530775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c100290 msgr2=0x7f362c1006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:19.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.298+0000 7f3633afc640 1 --2- 172.21.15.37:0/1185530775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c100290 0x7f362c1006b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f361c009ca0 tx=0x7f361c02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:20:19.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.298+0000 7f3633afc640 1 -- 172.21.15.37:0/1185530775 shutdown_connections 2024-03-26T06:20:19.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.298+0000 7f3633afc640 1 --2- 172.21.15.37:0/1185530775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c101e40 0x7f362c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:19.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.298+0000 7f3633afc640 1 --2- 172.21.15.37:0/1185530775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c101480 0x7f362c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:19.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.298+0000 7f3633afc640 1 --2- 172.21.15.37:0/1185530775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c100290 0x7f362c1006b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:19.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.298+0000 7f3633afc640 1 -- 172.21.15.37:0/1185530775 >> 172.21.15.37:0/1185530775 conn(0x7f362c0fb820 msgr2=0x7f362c0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:19.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3633afc640 1 -- 172.21.15.37:0/1185530775 shutdown_connections 2024-03-26T06:20:19.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3633afc640 1 -- 172.21.15.37:0/1185530775 wait complete. 2024-03-26T06:20:19.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3633afc640 1 Processor -- start 2024-03-26T06:20:19.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3633afc640 1 -- start start 2024-03-26T06:20:19.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3633afc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c100290 0x7f362c196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:19.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3633afc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c101480 0x7f362c196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:19.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3633afc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c101e40 0x7f362c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:19.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3633afc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c10beb0 con 0x7f362c100290 2024-03-26T06:20:19.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3633afc640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c10bd30 con 0x7f362c101e40 2024-03-26T06:20:19.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3633afc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c10c030 con 0x7f362c101480 2024-03-26T06:20:19.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3631898640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c100290 0x7f362c196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:19.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3631898640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c100290 0x7f362c196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34038/0 (socket says 172.21.15.37:34038) 2024-03-26T06:20:19.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3631898640 1 -- 172.21.15.37:0/1417230163 learned_addr learned my addr 172.21.15.37:0/1417230163 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:19.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3631097640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c101480 0x7f362c196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:19.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.299+0000 7f3632099640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c101e40 0x7f362c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:19.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.300+0000 7f3631898640 1 -- 172.21.15.37:0/1417230163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c101480 msgr2=0x7f362c196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:19.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.300+0000 7f3631898640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c101480 0x7f362c196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:19.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.300+0000 7f3631898640 1 -- 172.21.15.37:0/1417230163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c101e40 msgr2=0x7f362c19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:19.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.300+0000 7f3631898640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c101e40 0x7f362c19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:19.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.300+0000 7f3631898640 1 -- 172.21.15.37:0/1417230163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c19e120 con 0x7f362c100290 2024-03-26T06:20:19.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.300+0000 7f3631898640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c100290 0x7f362c196500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f361c0324c0 tx=0x7f361c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:19.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.300+0000 7f362affd640 1 -- 172.21.15.37:0/1417230163 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c041020 con 0x7f362c100290 2024-03-26T06:20:19.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.300+0000 7f3633afc640 1 -- 172.21.15.37:0/1417230163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c19e3b0 con 0x7f362c100290 2024-03-26T06:20:19.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.300+0000 7f362affd640 1 -- 172.21.15.37:0/1417230163 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f361c034cd0 con 0x7f362c100290 2024-03-26T06:20:19.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.300+0000 7f362affd640 1 -- 172.21.15.37:0/1417230163 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c039040 con 0x7f362c100290 2024-03-26T06:20:19.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.300+0000 7f3633afc640 1 -- 172.21.15.37:0/1417230163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c19e8a0 con 0x7f362c100290 2024-03-26T06:20:19.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.301+0000 7f3633afc640 1 -- 172.21.15.37:0/1417230163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c1075b0 con 0x7f362c100290 2024-03-26T06:20:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.302+0000 7f362affd640 1 -- 172.21.15.37:0/1417230163 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f361c044020 con 0x7f362c100290 2024-03-26T06:20:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.303+0000 7f362affd640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36140a2800 0x7f36140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.303+0000 7f3631097640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36140a2800 0x7f36140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.303+0000 7f362affd640 1 -- 172.21.15.37:0/1417230163 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f361c0f08d0 con 0x7f362c100290 2024-03-26T06:20:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.303+0000 7f3631097640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36140a2800 0x7f36140a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f362c197a10 tx=0x7f3620005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.306+0000 7f362affd640 1 -- 172.21.15.37:0/1417230163 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f361c04b020 con 0x7f362c100290 2024-03-26T06:20:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.526+0000 7f3633afc640 1 -- 172.21.15.37:0/1417230163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f362c02ce70 con 0x7f362c100290 2024-03-26T06:20:19.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.527+0000 7f362affd640 1 -- 172.21.15.37:0/1417230163 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f361c0bde00 con 0x7f362c100290 2024-03-26T06:20:19.528 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:19.528 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-26T06:20:19.528 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-03-26T06:20:19.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 -- 172.21.15.37:0/1417230163 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36140a2800 msgr2=0x7f36140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36140a2800 0x7f36140a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f362c197a10 tx=0x7f3620005f90 comp rx=0 tx=0).stop 2024-03-26T06:20:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 -- 172.21.15.37:0/1417230163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c100290 msgr2=0x7f362c196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c100290 0x7f362c196500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f361c0324c0 tx=0x7f361c032f40 comp rx=0 tx=0).stop 2024-03-26T06:20:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 -- 172.21.15.37:0/1417230163 shutdown_connections 2024-03-26T06:20:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36140a2800 0x7f36140a4cc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c101e40 0x7f362c19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c101480 0x7f362c196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 --2- 172.21.15.37:0/1417230163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c100290 0x7f362c196500 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 -- 172.21.15.37:0/1417230163 >> 172.21.15.37:0/1417230163 conn(0x7f362c0fb820 msgr2=0x7f362c104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 -- 172.21.15.37:0/1417230163 shutdown_connections 2024-03-26T06:20:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:19.529+0000 7f3633afc640 1 -- 172.21.15.37:0/1417230163 wait complete. 2024-03-26T06:20:19.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:19 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6832/2169208399,v1:172.21.15.149:6833/2169208399] up:boot 2024-03-26T06:20:19.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:19 smithi066 ceph-mon[36504]: fsmap 1 up:standby 2024-03-26T06:20:19.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T06:20:19.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:19.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:19.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:19.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:19.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:19.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:19 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:19.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:19 smithi066 ceph-mon[36504]: Deploying daemon mds.k on smithi066 2024-03-26T06:20:19.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:19 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1417230163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:19.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:19 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6832/2169208399,v1:172.21.15.149:6833/2169208399] up:boot 2024-03-26T06:20:19.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:19 smithi149 ceph-mon[36681]: fsmap 1 up:standby 2024-03-26T06:20:19.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T06:20:19.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:19.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:19.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:19.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:19.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:19.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:19 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:19.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:19 smithi149 ceph-mon[36681]: Deploying daemon mds.k on smithi066 2024-03-26T06:20:19.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:19 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1417230163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:19.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:19 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6832/2169208399,v1:172.21.15.149:6833/2169208399] up:boot 2024-03-26T06:20:19.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:19 smithi037 ceph-mon[31974]: fsmap 1 up:standby 2024-03-26T06:20:19.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T06:20:19.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:19.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:19.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:19.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:19.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:19.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:19 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:19.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:19 smithi037 ceph-mon[31974]: Deploying daemon mds.k on smithi066 2024-03-26T06:20:19.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:19 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1417230163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:20.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:20 smithi066 ceph-mon[36504]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:20.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:20 smithi149 ceph-mon[36681]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:20.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:20 smithi037 ceph-mon[31974]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:21.323 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:20:21 smithi066 systemd[1]: Starting Ceph mds.k for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:20:21.541 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:21.597 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:20:21 smithi066 podman[66326]: 2024-03-26 06:20:21.323411971 +0000 UTC m=+0.108654235 container create 5ed6198eac4d0f093c4f2070be6b3c9facc2539e39332f43f731473b64a8123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-26T06:20:21.597 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:20:21 smithi066 podman[66326]: 2024-03-26 06:20:21.233145258 +0000 UTC m=+0.018387505 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:20:21.598 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:20:21 smithi066 podman[66326]: 2024-03-26 06:20:21.534098379 +0000 UTC m=+0.319340618 container init 5ed6198eac4d0f093c4f2070be6b3c9facc2539e39332f43f731473b64a8123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:20:21.598 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:20:21 smithi066 podman[66326]: 2024-03-26 06:20:21.536793835 +0000 UTC m=+0.322036063 container start 5ed6198eac4d0f093c4f2070be6b3c9facc2539e39332f43f731473b64a8123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-03-26T06:20:21.598 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:20:21 smithi066 ceph-mds[66345]: -- 172.21.15.66:0/695945873 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5584de4f9a40 con 0x5584de4cd000 2024-03-26T06:20:21.598 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:20:21 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k[66341]: starting mds.k at 2024-03-26T06:20:21.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.658+0000 7ff27df0a640 1 -- 172.21.15.37:0/3015512175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278101400 msgr2=0x7ff278101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:21.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.658+0000 7ff27df0a640 1 --2- 172.21.15.37:0/3015512175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278101400 0x7ff278101880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff260009900 tx=0x7ff26002bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:21.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.658+0000 7ff27df0a640 1 -- 172.21.15.37:0/3015512175 shutdown_connections 2024-03-26T06:20:21.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.658+0000 7ff27df0a640 1 --2- 172.21.15.37:0/3015512175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff278101dc0 0x7ff278108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:21.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.658+0000 7ff27df0a640 1 --2- 172.21.15.37:0/3015512175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278101400 0x7ff278101880 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:21.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.658+0000 7ff27df0a640 1 --2- 172.21.15.37:0/3015512175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff278100210 0x7ff278100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:21.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.658+0000 7ff27df0a640 1 -- 172.21.15.37:0/3015512175 >> 172.21.15.37:0/3015512175 conn(0x7ff2780fb780 msgr2=0x7ff2780fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:21.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.658+0000 7ff27df0a640 1 -- 172.21.15.37:0/3015512175 shutdown_connections 2024-03-26T06:20:21.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.658+0000 7ff27df0a640 1 -- 172.21.15.37:0/3015512175 wait complete. 2024-03-26T06:20:21.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff27df0a640 1 Processor -- start 2024-03-26T06:20:21.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff27df0a640 1 -- start start 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff27df0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff278100210 0x7ff278068fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff27df0a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff278101400 0x7ff27819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff27df0a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278101dc0 0x7ff27819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff27df0a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27810bea0 con 0x7ff278101dc0 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff27df0a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27810bd20 con 0x7ff278100210 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff27df0a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27810c020 con 0x7ff278101400 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff277fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278101dc0 0x7ff27819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff2777fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff278100210 0x7ff278068fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff277fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278101dc0 0x7ff27819e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34058/0 (socket says 172.21.15.37:34058) 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff277fff640 1 -- 172.21.15.37:0/2137703825 learned_addr learned my addr 172.21.15.37:0/2137703825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff26effd640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff278101400 0x7ff27819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:21.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff277fff640 1 -- 172.21.15.37:0/2137703825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff278101400 msgr2=0x7ff27819c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff277fff640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff278101400 0x7ff27819c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff277fff640 1 -- 172.21.15.37:0/2137703825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff278100210 msgr2=0x7ff278068fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff277fff640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff278100210 0x7ff278068fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.659+0000 7ff277fff640 1 -- 172.21.15.37:0/2137703825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27819f250 con 0x7ff278101dc0 2024-03-26T06:20:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.660+0000 7ff277fff640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278101dc0 0x7ff27819e990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff27000eb10 tx=0x7ff27000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.660+0000 7ff2757fa640 1 -- 172.21.15.37:0/2137703825 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270004b20 con 0x7ff278101dc0 2024-03-26T06:20:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.660+0000 7ff2757fa640 1 -- 172.21.15.37:0/2137703825 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff270004cc0 con 0x7ff278101dc0 2024-03-26T06:20:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.660+0000 7ff27df0a640 1 -- 172.21.15.37:0/2137703825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27819f540 con 0x7ff278101dc0 2024-03-26T06:20:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.660+0000 7ff2757fa640 1 -- 172.21.15.37:0/2137703825 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270010760 con 0x7ff278101dc0 2024-03-26T06:20:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.660+0000 7ff27df0a640 1 -- 172.21.15.37:0/2137703825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27819fa60 con 0x7ff278101dc0 2024-03-26T06:20:21.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.661+0000 7ff27df0a640 1 -- 172.21.15.37:0/2137703825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff278107530 con 0x7ff278101dc0 2024-03-26T06:20:21.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.662+0000 7ff2757fa640 1 -- 172.21.15.37:0/2137703825 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff270005020 con 0x7ff278101dc0 2024-03-26T06:20:21.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.663+0000 7ff2757fa640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2640a2880 0x7ff2640a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:21.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.663+0000 7ff2757fa640 1 -- 172.21.15.37:0/2137703825 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff2700ceaa0 con 0x7ff278101dc0 2024-03-26T06:20:21.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.663+0000 7ff2777fe640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2640a2880 0x7ff2640a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:21.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.663+0000 7ff2777fe640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2640a2880 0x7ff2640a4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff278101260 tx=0x7ff268005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:21.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.667+0000 7ff2757fa640 1 -- 172.21.15.37:0/2137703825 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff27009bfe0 con 0x7ff278101dc0 2024-03-26T06:20:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.893+0000 7ff27df0a640 1 -- 172.21.15.37:0/2137703825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff27802d080 con 0x7ff278101dc0 2024-03-26T06:20:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.893+0000 7ff2757fa640 1 -- 172.21.15.37:0/2137703825 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7ff27009b8e0 con 0x7ff278101dc0 2024-03-26T06:20:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:21.895 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T06:20:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-03-26T06:20:21.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 -- 172.21.15.37:0/2137703825 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2640a2880 msgr2=0x7ff2640a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2640a2880 0x7ff2640a4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff278101260 tx=0x7ff268005f90 comp rx=0 tx=0).stop 2024-03-26T06:20:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 -- 172.21.15.37:0/2137703825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278101dc0 msgr2=0x7ff27819e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278101dc0 0x7ff27819e990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff27000eb10 tx=0x7ff27000c2d0 comp rx=0 tx=0).stop 2024-03-26T06:20:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 -- 172.21.15.37:0/2137703825 shutdown_connections 2024-03-26T06:20:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2640a2880 0x7ff2640a4d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff278101dc0 0x7ff27819e990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff278101400 0x7ff27819c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 --2- 172.21.15.37:0/2137703825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff278100210 0x7ff278068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 -- 172.21.15.37:0/2137703825 >> 172.21.15.37:0/2137703825 conn(0x7ff2780fb780 msgr2=0x7ff278104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 -- 172.21.15.37:0/2137703825 shutdown_connections 2024-03-26T06:20:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:21.896+0000 7ff27df0a640 1 -- 172.21.15.37:0/2137703825 wait complete. 2024-03-26T06:20:22.097 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:20:21 smithi066 bash[66326]: 5ed6198eac4d0f093c4f2070be6b3c9facc2539e39332f43f731473b64a8123d 2024-03-26T06:20:22.097 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:20:21 smithi066 systemd[1]: Started Ceph mds.k for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:22.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:22 smithi149 ceph-mon[36681]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:22 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.66:6832/2811553067,v1:172.21.15.66:6833/2811553067] up:boot 2024-03-26T06:20:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:22 smithi149 ceph-mon[36681]: fsmap 2 up:standby 2024-03-26T06:20:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T06:20:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:22 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:22 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2137703825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:22.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:22 smithi037 ceph-mon[31974]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:22.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:22 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.66:6832/2811553067,v1:172.21.15.66:6833/2811553067] up:boot 2024-03-26T06:20:22.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:22 smithi037 ceph-mon[31974]: fsmap 2 up:standby 2024-03-26T06:20:22.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T06:20:22.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:22.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:22.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:22.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:22.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:22.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:22 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:22.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:22 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2137703825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:22 smithi066 ceph-mon[36504]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:22 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.66:6832/2811553067,v1:172.21.15.66:6833/2811553067] up:boot 2024-03-26T06:20:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:22 smithi066 ceph-mon[36504]: fsmap 2 up:standby 2024-03-26T06:20:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T06:20:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:22 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:22 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2137703825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:23.691 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:23 smithi037 ceph-mon[31974]: Deploying daemon mds.j on smithi037 2024-03-26T06:20:23.909 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:23.963 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:23 smithi066 ceph-mon[36504]: Deploying daemon mds.j on smithi037 2024-03-26T06:20:24.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.022+0000 7fcec8113640 1 -- 172.21.15.37:0/586237094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec0101d90 msgr2=0x7fcec0104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:24.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.022+0000 7fcec8113640 1 --2- 172.21.15.37:0/586237094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec0101d90 0x7fcec0104180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009900 tx=0x7fcebc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:24.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.022+0000 7fcec8113640 1 -- 172.21.15.37:0/586237094 shutdown_connections 2024-03-26T06:20:24.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.022+0000 7fcec8113640 1 --2- 172.21.15.37:0/586237094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec0106ff0 0x7fcec0109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:24.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.022+0000 7fcec8113640 1 --2- 172.21.15.37:0/586237094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcec01046c0 0x7fcec0106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:24.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.022+0000 7fcec8113640 1 --2- 172.21.15.37:0/586237094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec0101d90 0x7fcec0104180 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:24.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.022+0000 7fcec8113640 1 -- 172.21.15.37:0/586237094 >> 172.21.15.37:0/586237094 conn(0x7fcec00fb780 msgr2=0x7fcec00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:24.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.022+0000 7fcec8113640 1 -- 172.21.15.37:0/586237094 shutdown_connections 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.022+0000 7fcec8113640 1 -- 172.21.15.37:0/586237094 wait complete. 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec8113640 1 Processor -- start 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec8113640 1 -- start start 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec8113640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec0101d90 0x7fcec0101700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec8113640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec01046c0 0x7fcec00ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec8113640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcec0106ff0 0x7fcec0100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec8113640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec010d000 con 0x7fcec01046c0 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec8113640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec010ce80 con 0x7fcec0106ff0 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec8113640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec010d180 con 0x7fcec0101d90 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec56ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec01046c0 0x7fcec00ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec56ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec01046c0 0x7fcec00ffd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34080/0 (socket says 172.21.15.37:34080) 2024-03-26T06:20:24.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec56ae640 1 -- 172.21.15.37:0/3160410446 learned_addr learned my addr 172.21.15.37:0/3160410446 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec66b0640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcec0106ff0 0x7fcec0100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec5eaf640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec0101d90 0x7fcec0101700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec56ae640 1 -- 172.21.15.37:0/3160410446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec0101d90 msgr2=0x7fcec0101700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec56ae640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec0101d90 0x7fcec0101700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.023+0000 7fcec56ae640 1 -- 172.21.15.37:0/3160410446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcec0106ff0 msgr2=0x7fcec0100280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.024+0000 7fcec56ae640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcec0106ff0 0x7fcec0100280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.024+0000 7fcec56ae640 1 -- 172.21.15.37:0/3160410446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec0100b10 con 0x7fcec01046c0 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.024+0000 7fcec56ae640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec01046c0 0x7fcec00ffd40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fceb0005fd0 tx=0x7fceb000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.024+0000 7fceb6ffd640 1 -- 172.21.15.37:0/3160410446 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0017070 con 0x7fcec01046c0 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.024+0000 7fcec8113640 1 -- 172.21.15.37:0/3160410446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec01a17c0 con 0x7fcec01046c0 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.024+0000 7fceb6ffd640 1 -- 172.21.15.37:0/3160410446 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fceb0009c30 con 0x7fcec01046c0 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.024+0000 7fceb6ffd640 1 -- 172.21.15.37:0/3160410446 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0010460 con 0x7fcec01046c0 2024-03-26T06:20:24.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.024+0000 7fcec8113640 1 -- 172.21.15.37:0/3160410446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec01a1d10 con 0x7fcec01046c0 2024-03-26T06:20:24.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.025+0000 7fcec8113640 1 -- 172.21.15.37:0/3160410446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec00fd270 con 0x7fcec01046c0 2024-03-26T06:20:24.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.026+0000 7fceb6ffd640 1 -- 172.21.15.37:0/3160410446 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fceb0003e70 con 0x7fcec01046c0 2024-03-26T06:20:24.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.027+0000 7fceb6ffd640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fceac0a2800 0x7fceac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:24.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.027+0000 7fcec5eaf640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fceac0a2800 0x7fceac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:24.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.027+0000 7fceb6ffd640 1 -- 172.21.15.37:0/3160410446 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fceb00cea70 con 0x7fcec01046c0 2024-03-26T06:20:24.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.027+0000 7fcec5eaf640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fceac0a2800 0x7fceac0a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcebc005c20 tx=0x7fcebc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:24.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.031+0000 7fceb6ffd640 1 -- 172.21.15.37:0/3160410446 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fceb0020030 con 0x7fcec01046c0 2024-03-26T06:20:24.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:23 smithi149 ceph-mon[36681]: Deploying daemon mds.j on smithi037 2024-03-26T06:20:24.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.267+0000 7fcec8113640 1 -- 172.21.15.37:0/3160410446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcec002d0c0 con 0x7fcec01046c0 2024-03-26T06:20:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.268+0000 7fceb6ffd640 1 -- 172.21.15.37:0/3160410446 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fceb009bfb0 con 0x7fcec01046c0 2024-03-26T06:20:24.269 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:24.269 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T06:20:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-03-26T06:20:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 -- 172.21.15.37:0/3160410446 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fceac0a2800 msgr2=0x7fceac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fceac0a2800 0x7fceac0a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcebc005c20 tx=0x7fcebc005fd0 comp rx=0 tx=0).stop 2024-03-26T06:20:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 -- 172.21.15.37:0/3160410446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec01046c0 msgr2=0x7fcec00ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec01046c0 0x7fcec00ffd40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fceb0005fd0 tx=0x7fceb000dde0 comp rx=0 tx=0).stop 2024-03-26T06:20:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 -- 172.21.15.37:0/3160410446 shutdown_connections 2024-03-26T06:20:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fceac0a2800 0x7fceac0a4cc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcec0106ff0 0x7fcec0100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec01046c0 0x7fcec00ffd40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:24.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 --2- 172.21.15.37:0/3160410446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec0101d90 0x7fcec0101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:24.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 -- 172.21.15.37:0/3160410446 >> 172.21.15.37:0/3160410446 conn(0x7fcec00fb780 msgr2=0x7fcec0102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:24.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 -- 172.21.15.37:0/3160410446 shutdown_connections 2024-03-26T06:20:24.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:24.270+0000 7fcec8113640 1 -- 172.21.15.37:0/3160410446 wait complete. 2024-03-26T06:20:24.348 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:20:24 smithi037 podman[72887]: 2024-03-26 06:20:24.348379124 +0000 UTC m=+0.370219117 container init 7cd9991f93c1772a92e8d0b7a8a5abb803ef82c581e4d1e504910abcb7b2aa0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-j, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-26T06:20:24.669 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:20:24 smithi037 podman[72887]: 2024-03-26 06:20:24.351029264 +0000 UTC m=+0.372869257 container start 7cd9991f93c1772a92e8d0b7a8a5abb803ef82c581e4d1e504910abcb7b2aa0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS) 2024-03-26T06:20:24.669 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:20:24 smithi037 ceph-mds[72927]: -- 172.21.15.37:0/3431549842 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559f9e587a40 con 0x559f9e55a800 2024-03-26T06:20:24.670 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:20:24 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-j[72921]: starting mds.j at 2024-03-26T06:20:24.670 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:20:24 smithi037 bash[72887]: 7cd9991f93c1772a92e8d0b7a8a5abb803ef82c581e4d1e504910abcb7b2aa0e 2024-03-26T06:20:24.670 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:20:24 smithi037 systemd[1]: Started Ceph mds.j for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:24.933 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:24 smithi066 ceph-mon[36504]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:24.933 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:24 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3160410446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:24.934 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:24.934 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:24.934 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:24.934 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:24.934 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:24.934 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:24 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:24 smithi037 ceph-mon[31974]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:24 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3160410446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:24 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:25.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:24 smithi149 ceph-mon[36681]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:25.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:24 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3160410446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:25.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:25.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:25.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:25.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:25.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:25.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:24 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:25.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:25 smithi037 ceph-mon[31974]: Deploying daemon mds.d on smithi037 2024-03-26T06:20:25.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:25 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.37:6834/1987413446,v1:172.21.15.37:6835/1987413446] up:boot 2024-03-26T06:20:25.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:25 smithi037 ceph-mon[31974]: fsmap 3 up:standby 2024-03-26T06:20:25.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:25 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T06:20:25.946 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:25 smithi149 ceph-mon[36681]: Deploying daemon mds.d on smithi037 2024-03-26T06:20:25.946 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:25 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.37:6834/1987413446,v1:172.21.15.37:6835/1987413446] up:boot 2024-03-26T06:20:25.946 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:25 smithi149 ceph-mon[36681]: fsmap 3 up:standby 2024-03-26T06:20:25.946 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:25 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T06:20:26.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:25 smithi066 ceph-mon[36504]: Deploying daemon mds.d on smithi037 2024-03-26T06:20:26.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:25 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.37:6834/1987413446,v1:172.21.15.37:6835/1987413446] up:boot 2024-03-26T06:20:26.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:25 smithi066 ceph-mon[36504]: fsmap 3 up:standby 2024-03-26T06:20:26.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:25 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T06:20:26.285 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:26.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.404+0000 7ffab3144640 1 -- 172.21.15.37:0/956207926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101320 msgr2=0x7ffaac101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:26.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.404+0000 7ffab3144640 1 --2- 172.21.15.37:0/956207926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101320 0x7ffaac101740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009900 tx=0x7ffaa802bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:26.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.404+0000 7ffab3144640 1 -- 172.21.15.37:0/956207926 shutdown_connections 2024-03-26T06:20:26.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.404+0000 7ffab3144640 1 --2- 172.21.15.37:0/956207926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaac102ed0 0x7ffaac109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.404+0000 7ffab3144640 1 --2- 172.21.15.37:0/956207926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaac102510 0x7ffaac102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.404+0000 7ffab3144640 1 --2- 172.21.15.37:0/956207926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101320 0x7ffaac101740 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.404+0000 7ffab3144640 1 -- 172.21.15.37:0/956207926 >> 172.21.15.37:0/956207926 conn(0x7ffaac0fc8d0 msgr2=0x7ffaac0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.404+0000 7ffab3144640 1 -- 172.21.15.37:0/956207926 shutdown_connections 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.404+0000 7ffab3144640 1 -- 172.21.15.37:0/956207926 wait complete. 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab3144640 1 Processor -- start 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab3144640 1 -- start start 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab3144640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101320 0x7ffaac0750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab3144640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaac102510 0x7ffaac0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab3144640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaac102ed0 0x7ffaac073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab3144640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac10cfb0 con 0x7ffaac101320 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab3144640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac10ce30 con 0x7ffaac102ed0 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab3144640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac10d130 con 0x7ffaac102510 2024-03-26T06:20:26.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab0ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101320 0x7ffaac0750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab0ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101320 0x7ffaac0750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34120/0 (socket says 172.21.15.37:34120) 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab0ee0640 1 -- 172.21.15.37:0/1072590450 learned_addr learned my addr 172.21.15.37:0/1072590450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab16e1640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaac102ed0 0x7ffaac073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffaa3fff640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaac102510 0x7ffaac0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab0ee0640 1 -- 172.21.15.37:0/1072590450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaac102510 msgr2=0x7ffaac0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab0ee0640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaac102510 0x7ffaac0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab0ee0640 1 -- 172.21.15.37:0/1072590450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaac102ed0 msgr2=0x7ffaac073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab0ee0640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaac102ed0 0x7ffaac073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab0ee0640 1 -- 172.21.15.37:0/1072590450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0744b0 con 0x7ffaac101320 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffaa3fff640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaac102510 0x7ffaac0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.405+0000 7ffab16e1640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaac102ed0 0x7ffaac073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.406+0000 7ffab0ee0640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101320 0x7ffaac0750a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8005ab0 tx=0x7ffaa8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:26.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.406+0000 7ffaa1ffb640 1 -- 172.21.15.37:0/1072590450 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8041070 con 0x7ffaac101320 2024-03-26T06:20:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.406+0000 7ffaa1ffb640 1 -- 172.21.15.37:0/1072590450 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffaa80054e0 con 0x7ffaac101320 2024-03-26T06:20:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.406+0000 7ffab3144640 1 -- 172.21.15.37:0/1072590450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac1a16f0 con 0x7ffaac101320 2024-03-26T06:20:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.406+0000 7ffaa1ffb640 1 -- 172.21.15.37:0/1072590450 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8034440 con 0x7ffaac101320 2024-03-26T06:20:26.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.406+0000 7ffab3144640 1 -- 172.21.15.37:0/1072590450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac1a1c40 con 0x7ffaac101320 2024-03-26T06:20:26.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.407+0000 7ffab3144640 1 -- 172.21.15.37:0/1072590450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac1086c0 con 0x7ffaac101320 2024-03-26T06:20:26.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.408+0000 7ffaa1ffb640 1 -- 172.21.15.37:0/1072590450 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ffaa8044070 con 0x7ffaac101320 2024-03-26T06:20:26.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.409+0000 7ffaa1ffb640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa940a2800 0x7ffa940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:26.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.409+0000 7ffaa3fff640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa940a2800 0x7ffa940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:26.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.409+0000 7ffaa1ffb640 1 -- 172.21.15.37:0/1072590450 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffaa80f02e0 con 0x7ffaac101320 2024-03-26T06:20:26.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.409+0000 7ffaa3fff640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa940a2800 0x7ffa940a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffaac074e10 tx=0x7ffa9c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:26.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.413+0000 7ffaa1ffb640 1 -- 172.21.15.37:0/1072590450 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ffaa80bd820 con 0x7ffaac101320 2024-03-26T06:20:26.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.653+0000 7ffab3144640 1 -- 172.21.15.37:0/1072590450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaac02d0d0 con 0x7ffaac101320 2024-03-26T06:20:26.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.653+0000 7ffaa1ffb640 1 -- 172.21.15.37:0/1072590450 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7ffaa80bd120 con 0x7ffaac101320 2024-03-26T06:20:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:26.654 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T06:20:26.655 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 4 2024-03-26T06:20:26.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.655+0000 7ffab3144640 1 -- 172.21.15.37:0/1072590450 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa940a2800 msgr2=0x7ffa940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:26.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.655+0000 7ffab3144640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa940a2800 0x7ffa940a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffaac074e10 tx=0x7ffa9c005d30 comp rx=0 tx=0).stop 2024-03-26T06:20:26.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.655+0000 7ffab3144640 1 -- 172.21.15.37:0/1072590450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101320 msgr2=0x7ffaac0750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:26.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.655+0000 7ffab3144640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101320 0x7ffaac0750a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8005ab0 tx=0x7ffaa8005dc0 comp rx=0 tx=0).stop 2024-03-26T06:20:26.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.655+0000 7ffab3144640 1 -- 172.21.15.37:0/1072590450 shutdown_connections 2024-03-26T06:20:26.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.655+0000 7ffab3144640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa940a2800 0x7ffa940a4cc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:26.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.655+0000 7ffab3144640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaac102ed0 0x7ffaac073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:26.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.655+0000 7ffab3144640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaac102510 0x7ffaac0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:26.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.655+0000 7ffab3144640 1 --2- 172.21.15.37:0/1072590450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101320 0x7ffaac0750a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:26.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.655+0000 7ffab3144640 1 -- 172.21.15.37:0/1072590450 >> 172.21.15.37:0/1072590450 conn(0x7ffaac0fc8d0 msgr2=0x7ffaac105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:26.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.656+0000 7ffab3144640 1 -- 172.21.15.37:0/1072590450 shutdown_connections 2024-03-26T06:20:26.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:26.656+0000 7ffab3144640 1 -- 172.21.15.37:0/1072590450 wait complete. 2024-03-26T06:20:26.927 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:26 smithi037 ceph-mon[31974]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:26.927 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:26 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1072590450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:27.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:26 smithi066 ceph-mon[36504]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:27.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:26 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1072590450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:27.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:26 smithi149 ceph-mon[36681]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:27.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:26 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1072590450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:27.198 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:20:27 smithi037 podman[73259]: 2024-03-26 06:20:27.15148201 +0000 UTC m=+0.124421869 container create 8b6b55fa840ce4af875c991aabfff34f45c43274e3b42d800d56a3339409634a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS) 2024-03-26T06:20:27.698 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:20:27 smithi037 podman[73259]: 2024-03-26 06:20:27.378802284 +0000 UTC m=+0.351742143 container init 8b6b55fa840ce4af875c991aabfff34f45c43274e3b42d800d56a3339409634a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:20:27.698 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:20:27 smithi037 podman[73259]: 2024-03-26 06:20:27.381379281 +0000 UTC m=+0.354319141 container start 8b6b55fa840ce4af875c991aabfff34f45c43274e3b42d800d56a3339409634a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-26T06:20:27.698 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:20:27 smithi037 ceph-mds[73278]: -- 172.21.15.37:0/2257566732 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5567cfab7a40 con 0x5567cfa8ac00 2024-03-26T06:20:27.699 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:20:27 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-d[73274]: starting mds.d at 2024-03-26T06:20:27.699 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:20:27 smithi037 bash[73259]: 8b6b55fa840ce4af875c991aabfff34f45c43274e3b42d800d56a3339409634a 2024-03-26T06:20:27.699 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:20:27 smithi037 systemd[1]: Started Ceph mds.d for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:28.669 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.787+0000 7ff2b968c640 1 -- 172.21.15.37:0/3661391051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4103030 msgr2=0x7ff2b41098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.787+0000 7ff2b968c640 1 --2- 172.21.15.37:0/3661391051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4103030 0x7ff2b41098c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009cd0 tx=0x7ff2ac02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:20:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.787+0000 7ff2b968c640 1 -- 172.21.15.37:0/3661391051 shutdown_connections 2024-03-26T06:20:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.787+0000 7ff2b968c640 1 --2- 172.21.15.37:0/3661391051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4103030 0x7ff2b41098c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.787+0000 7ff2b968c640 1 --2- 172.21.15.37:0/3661391051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4102670 0x7ff2b4102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.787+0000 7ff2b968c640 1 --2- 172.21.15.37:0/3661391051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b40ff600 0x7ff2b40ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.787+0000 7ff2b968c640 1 -- 172.21.15.37:0/3661391051 >> 172.21.15.37:0/3661391051 conn(0x7ff2b40faf10 msgr2=0x7ff2b40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:28.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.787+0000 7ff2b968c640 1 -- 172.21.15.37:0/3661391051 shutdown_connections 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.787+0000 7ff2b968c640 1 -- 172.21.15.37:0/3661391051 wait complete. 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2b968c640 1 Processor -- start 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2b968c640 1 -- start start 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2b968c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b40ff600 0x7ff2b4101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2b968c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102670 0x7ff2b4100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2b968c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4103030 0x7ff2b4100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2b968c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b410cfc0 con 0x7ff2b4102670 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2b968c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b410ce40 con 0x7ff2b4103030 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2b968c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b410d140 con 0x7ff2b40ff600 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102670 0x7ff2b4100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102670 0x7ff2b4100620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51200/0 (socket says 172.21.15.37:51200) 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2abfff640 1 -- 172.21.15.37:0/1283203357 learned_addr learned my addr 172.21.15.37:0/1283203357 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:28.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4103030 0x7ff2b4100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b40ff600 0x7ff2b4101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2abfff640 1 -- 172.21.15.37:0/1283203357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b40ff600 msgr2=0x7ff2b4101fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2abfff640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b40ff600 0x7ff2b4101fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2abfff640 1 -- 172.21.15.37:0/1283203357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4103030 msgr2=0x7ff2b4100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2abfff640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4103030 0x7ff2b4100b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2abfff640 1 -- 172.21.15.37:0/1283203357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b41013f0 con 0x7ff2b4102670 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.788+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4103030 0x7ff2b4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.789+0000 7ff2abfff640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102670 0x7ff2b4100620 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00d950 tx=0x7ff29c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.789+0000 7ff2b0ff9640 1 -- 172.21.15.37:0/1283203357 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c017030 con 0x7ff2b4102670 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.789+0000 7ff2b968c640 1 -- 172.21.15.37:0/1283203357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b41a1700 con 0x7ff2b4102670 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.789+0000 7ff2b0ff9640 1 -- 172.21.15.37:0/1283203357 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff29c004020 con 0x7ff2b4102670 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.789+0000 7ff2b0ff9640 1 -- 172.21.15.37:0/1283203357 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0047a0 con 0x7ff2b4102670 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.789+0000 7ff2b968c640 1 -- 172.21.15.37:0/1283203357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b41a1ca0 con 0x7ff2b4102670 2024-03-26T06:20:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.790+0000 7ff2b968c640 1 -- 172.21.15.37:0/1283203357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b41087a0 con 0x7ff2b4102670 2024-03-26T06:20:28.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.791+0000 7ff2b0ff9640 1 -- 172.21.15.37:0/1283203357 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff29c0041d0 con 0x7ff2b4102670 2024-03-26T06:20:28.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.792+0000 7ff2b0ff9640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2a00a2800 0x7ff2a00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:28.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.792+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2a00a2800 0x7ff2a00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:28.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.792+0000 7ff2b0ff9640 1 -- 172.21.15.37:0/1283203357 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff29c0ce0c0 con 0x7ff2b4102670 2024-03-26T06:20:28.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.792+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2a00a2800 0x7ff2a00a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4005fd0 tx=0x7ff2a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:28.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:28.796+0000 7ff2b0ff9640 1 -- 172.21.15.37:0/1283203357 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff29c09b600 con 0x7ff2b4102670 2024-03-26T06:20:28.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:28 smithi066 ceph-mon[36504]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:28.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:28 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:28.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:28 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:28.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:28 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:28.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:28 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:28.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:28 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:28.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:28 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:28.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:28 smithi066 ceph-mon[36504]: Deploying daemon mds.e on smithi066 2024-03-26T06:20:28.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:28 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.37:6836/3479806072,v1:172.21.15.37:6837/3479806072] up:boot 2024-03-26T06:20:28.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:28 smithi066 ceph-mon[36504]: fsmap 4 up:standby 2024-03-26T06:20:28.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:28 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T06:20:28.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:28 smithi149 ceph-mon[36681]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:28 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:28 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:28 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:28 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:28 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:28 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:28 smithi149 ceph-mon[36681]: Deploying daemon mds.e on smithi066 2024-03-26T06:20:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:28 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.37:6836/3479806072,v1:172.21.15.37:6837/3479806072] up:boot 2024-03-26T06:20:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:28 smithi149 ceph-mon[36681]: fsmap 4 up:standby 2024-03-26T06:20:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:28 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T06:20:28.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:28 smithi037 ceph-mon[31974]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:28.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:28 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:28.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:28 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:28.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:28 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:28.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:28 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:28.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:28 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:28.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:28 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:28.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:28 smithi037 ceph-mon[31974]: Deploying daemon mds.e on smithi066 2024-03-26T06:20:28.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:28 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.37:6836/3479806072,v1:172.21.15.37:6837/3479806072] up:boot 2024-03-26T06:20:28.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:28 smithi037 ceph-mon[31974]: fsmap 4 up:standby 2024-03-26T06:20:28.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:28 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T06:20:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.016+0000 7ff2b968c640 1 -- 172.21.15.37:0/1283203357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2b402d090 con 0x7ff2b4102670 2024-03-26T06:20:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.017+0000 7ff2b0ff9640 1 -- 172.21.15.37:0/1283203357 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7ff29c09af00 con 0x7ff2b4102670 2024-03-26T06:20:29.018 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:29.018 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T06:20:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 5 2024-03-26T06:20:29.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 -- 172.21.15.37:0/1283203357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2a00a2800 msgr2=0x7ff2a00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:29.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2a00a2800 0x7ff2a00a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4005fd0 tx=0x7ff2a4005850 comp rx=0 tx=0).stop 2024-03-26T06:20:29.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 -- 172.21.15.37:0/1283203357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102670 msgr2=0x7ff2b4100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:29.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102670 0x7ff2b4100620 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00d950 tx=0x7ff29c00de10 comp rx=0 tx=0).stop 2024-03-26T06:20:29.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 -- 172.21.15.37:0/1283203357 shutdown_connections 2024-03-26T06:20:29.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2a00a2800 0x7ff2a00a4cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:29.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2b4103030 0x7ff2b4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:29.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102670 0x7ff2b4100620 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:29.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 --2- 172.21.15.37:0/1283203357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b40ff600 0x7ff2b4101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:29.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 -- 172.21.15.37:0/1283203357 >> 172.21.15.37:0/1283203357 conn(0x7ff2b40faf10 msgr2=0x7ff2b4105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:29.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 -- 172.21.15.37:0/1283203357 shutdown_connections 2024-03-26T06:20:29.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:29.019+0000 7ff2b968c640 1 -- 172.21.15.37:0/1283203357 wait complete. 2024-03-26T06:20:29.664 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:29 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1283203357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:29.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:29 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1283203357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:29.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:29 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1283203357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:30.347 INFO:journalctl@ceph.mds.e.smithi066.stdout:Mar 26 06:20:30 smithi066 systemd[1]: Started Ceph mds.e for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:30.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:30 smithi066 ceph-mon[36504]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:30.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:30.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:30.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:30.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:30.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:30.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:30 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:30.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:30 smithi149 ceph-mon[36681]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:30.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:30.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:30.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:30.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:30.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:30.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:30 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:30.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:30 smithi037 ceph-mon[31974]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:30.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:30.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:30.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:30.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:30.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:30.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:30 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:31.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1206300236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d8101e10 msgr2=0x7fe3d8104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 --2- 172.21.15.37:0/1206300236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d8101e10 0x7fe3d8104200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009900 tx=0x7fe3d002bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1206300236 shutdown_connections 2024-03-26T06:20:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 --2- 172.21.15.37:0/1206300236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3d8107070 0x7fe3d8109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 --2- 172.21.15.37:0/1206300236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3d8104740 0x7fe3d8106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 --2- 172.21.15.37:0/1206300236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d8101e10 0x7fe3d8104200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1206300236 >> 172.21.15.37:0/1206300236 conn(0x7fe3d80fb780 msgr2=0x7fe3d80fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1206300236 shutdown_connections 2024-03-26T06:20:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1206300236 wait complete. 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 Processor -- start 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 -- start start 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3d8101e10 0x7fe3d8197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d8104740 0x7fe3d8197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3d8107070 0x7fe3d819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d810cfa0 con 0x7fe3d8104740 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d810ce20 con 0x7fe3d8107070 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.148+0000 7fe3dff7e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d810d120 con 0x7fe3d8101e10 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3dd519640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d8104740 0x7fe3d8197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3dd519640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d8104740 0x7fe3d8197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51216/0 (socket says 172.21.15.37:51216) 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3dd519640 1 -- 172.21.15.37:0/1473768045 learned_addr learned my addr 172.21.15.37:0/1473768045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3ddd1a640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3d8101e10 0x7fe3d8197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3de51b640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3d8107070 0x7fe3d819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3dd519640 1 -- 172.21.15.37:0/1473768045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3d8101e10 msgr2=0x7fe3d8197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3dd519640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3d8101e10 0x7fe3d8197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3dd519640 1 -- 172.21.15.37:0/1473768045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3d8107070 msgr2=0x7fe3d819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3dd519640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3d8107070 0x7fe3d819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3dd519640 1 -- 172.21.15.37:0/1473768045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d819f270 con 0x7fe3d8104740 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3de51b640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3d8107070 0x7fe3d819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3dd519640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d8104740 0x7fe3d8197b90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00e9c0 tx=0x7fe3cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3d6ffd640 1 -- 172.21.15.37:0/1473768045 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc00cd80 con 0x7fe3d8104740 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1473768045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d819f560 con 0x7fe3d8104740 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3d6ffd640 1 -- 172.21.15.37:0/1473768045 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc005500 con 0x7fe3d8104740 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3d6ffd640 1 -- 172.21.15.37:0/1473768045 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc010690 con 0x7fe3d8104740 2024-03-26T06:20:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.149+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1473768045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d819fb00 con 0x7fe3d8104740 2024-03-26T06:20:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.150+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1473768045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d80fd270 con 0x7fe3d8104740 2024-03-26T06:20:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.151+0000 7fe3d6ffd640 1 -- 172.21.15.37:0/1473768045 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe3cc0046c0 con 0x7fe3d8104740 2024-03-26T06:20:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.152+0000 7fe3d6ffd640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe3c00a2800 0x7fe3c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.152+0000 7fe3ddd1a640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe3c00a2800 0x7fe3c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.152+0000 7fe3d6ffd640 1 -- 172.21.15.37:0/1473768045 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe3cc0cea70 con 0x7fe3d8104740 2024-03-26T06:20:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.152+0000 7fe3ddd1a640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe3c00a2800 0x7fe3c00a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe3d002c7b0 tx=0x7fe3d0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:31.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.157+0000 7fe3d6ffd640 1 -- 172.21.15.37:0/1473768045 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe3cc09bf10 con 0x7fe3d8104740 2024-03-26T06:20:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.376+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1473768045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3d8061ef0 con 0x7fe3d8104740 2024-03-26T06:20:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.377+0000 7fe3d6ffd640 1 -- 172.21.15.37:0/1473768045 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7fe3cc09b810 con 0x7fe3d8104740 2024-03-26T06:20:31.378 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:31.378 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T06:20:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 6 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.378+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1473768045 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe3c00a2800 msgr2=0x7fe3c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.378+0000 7fe3dff7e640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe3c00a2800 0x7fe3c00a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe3d002c7b0 tx=0x7fe3d0005fd0 comp rx=0 tx=0).stop 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.378+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1473768045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d8104740 msgr2=0x7fe3d8197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.378+0000 7fe3dff7e640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d8104740 0x7fe3d8197b90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00e9c0 tx=0x7fe3cc00ee80 comp rx=0 tx=0).stop 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.379+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1473768045 shutdown_connections 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.379+0000 7fe3dff7e640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe3c00a2800 0x7fe3c00a4cc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.379+0000 7fe3dff7e640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe3d8107070 0x7fe3d819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.379+0000 7fe3dff7e640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d8104740 0x7fe3d8197b90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.379+0000 7fe3dff7e640 1 --2- 172.21.15.37:0/1473768045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3d8101e10 0x7fe3d8197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.379+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1473768045 >> 172.21.15.37:0/1473768045 conn(0x7fe3d80fb780 msgr2=0x7fe3d8102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.379+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1473768045 shutdown_connections 2024-03-26T06:20:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:31.379+0000 7fe3dff7e640 1 -- 172.21.15.37:0/1473768045 wait complete. 2024-03-26T06:20:31.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:31 smithi037 ceph-mon[31974]: Deploying daemon mds.f on smithi149 2024-03-26T06:20:31.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:31 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] up:boot 2024-03-26T06:20:31.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:31 smithi037 ceph-mon[31974]: fsmap 5 up:standby 2024-03-26T06:20:31.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:31 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T06:20:31.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:31 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1473768045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:31.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:31 smithi149 ceph-mon[36681]: Deploying daemon mds.f on smithi149 2024-03-26T06:20:31.812 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:31 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] up:boot 2024-03-26T06:20:31.813 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:31 smithi149 ceph-mon[36681]: fsmap 5 up:standby 2024-03-26T06:20:31.813 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:31 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T06:20:31.813 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:31 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1473768045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:31.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:31 smithi066 ceph-mon[36504]: Deploying daemon mds.f on smithi149 2024-03-26T06:20:31.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:31 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] up:boot 2024-03-26T06:20:31.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:31 smithi066 ceph-mon[36504]: fsmap 5 up:standby 2024-03-26T06:20:31.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:31 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T06:20:31.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:31 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1473768045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:32.545 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:20:32 smithi149 podman[67434]: 2024-03-26 06:20:32.424647476 +0000 UTC m=+0.125879288 container create ad3a23bcf9d8f60343570f699f5daf2e02d3418d8b0ff34fc5cedccff135bc89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:20:32.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:32 smithi066 ceph-mon[36504]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:32.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:32 smithi037 ceph-mon[31974]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:32.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:32 smithi149 ceph-mon[36681]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:32.910 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:20:32 smithi149 podman[67434]: 2024-03-26 06:20:32.643733161 +0000 UTC m=+0.344964937 container init ad3a23bcf9d8f60343570f699f5daf2e02d3418d8b0ff34fc5cedccff135bc89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-26T06:20:32.910 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:20:32 smithi149 podman[67434]: 2024-03-26 06:20:32.646664183 +0000 UTC m=+0.347895953 container start ad3a23bcf9d8f60343570f699f5daf2e02d3418d8b0ff34fc5cedccff135bc89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:20:32.910 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:20:32 smithi149 ceph-mds[67453]: -- 172.21.15.149:0/4069863952 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562451d1ba40 con 0x562451ceec00 2024-03-26T06:20:32.910 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:20:32 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f[67449]: starting mds.f at 2024-03-26T06:20:32.910 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:20:32 smithi149 bash[67434]: ad3a23bcf9d8f60343570f699f5daf2e02d3418d8b0ff34fc5cedccff135bc89 2024-03-26T06:20:32.910 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:20:32 smithi149 systemd[1]: Started Ceph mds.f for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:33.391 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.496+0000 7f5ead57e640 1 -- 172.21.15.37:0/1286838328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea8107050 msgr2=0x7f5ea8109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.496+0000 7f5ead57e640 1 --2- 172.21.15.37:0/1286838328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea8107050 0x7f5ea8109440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009ca0 tx=0x7f5ea002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:20:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.496+0000 7f5ead57e640 1 -- 172.21.15.37:0/1286838328 shutdown_connections 2024-03-26T06:20:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.496+0000 7f5ead57e640 1 --2- 172.21.15.37:0/1286838328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea8107050 0x7f5ea8109440 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.496+0000 7f5ead57e640 1 --2- 172.21.15.37:0/1286838328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8104720 0x7f5ea8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.496+0000 7f5ead57e640 1 --2- 172.21.15.37:0/1286838328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8101df0 0x7f5ea81041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.496+0000 7f5ead57e640 1 -- 172.21.15.37:0/1286838328 >> 172.21.15.37:0/1286838328 conn(0x7f5ea80fb780 msgr2=0x7f5ea80fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.496+0000 7f5ead57e640 1 -- 172.21.15.37:0/1286838328 shutdown_connections 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.496+0000 7f5ead57e640 1 -- 172.21.15.37:0/1286838328 wait complete. 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ead57e640 1 Processor -- start 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ead57e640 1 -- start start 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ead57e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8101df0 0x7f5ea8101760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ead57e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8104720 0x7f5ea80ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ead57e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea8107050 0x7f5ea81002e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ead57e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea810cfa0 con 0x7f5ea8107050 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ead57e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea810ce20 con 0x7f5ea8101df0 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ead57e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea810d120 con 0x7f5ea8104720 2024-03-26T06:20:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ea77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea8107050 0x7f5ea81002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ea77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea8107050 0x7f5ea81002e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51234/0 (socket says 172.21.15.37:51234) 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ea77fe640 1 -- 172.21.15.37:0/1213718623 learned_addr learned my addr 172.21.15.37:0/1213718623 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ea67fc640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8104720 0x7f5ea80ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.497+0000 7f5ea6ffd640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8101df0 0x7f5ea8101760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.498+0000 7f5ea77fe640 1 -- 172.21.15.37:0/1213718623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8104720 msgr2=0x7f5ea80ffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.498+0000 7f5ea77fe640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8104720 0x7f5ea80ffda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.498+0000 7f5ea77fe640 1 -- 172.21.15.37:0/1213718623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8101df0 msgr2=0x7f5ea8101760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.498+0000 7f5ea77fe640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8101df0 0x7f5ea8101760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.498+0000 7f5ea77fe640 1 -- 172.21.15.37:0/1213718623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea8100b70 con 0x7f5ea8107050 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.498+0000 7f5ea77fe640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea8107050 0x7f5ea81002e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5ea002c700 tx=0x7f5ea0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.498+0000 7f5e9ffff640 1 -- 172.21.15.37:0/1213718623 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0039040 con 0x7f5ea8107050 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.498+0000 7f5e9ffff640 1 -- 172.21.15.37:0/1213718623 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ea0044070 con 0x7f5ea8107050 2024-03-26T06:20:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.498+0000 7f5e9ffff640 1 -- 172.21.15.37:0/1213718623 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea00042a0 con 0x7f5ea8107050 2024-03-26T06:20:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.498+0000 7f5ead57e640 1 -- 172.21.15.37:0/1213718623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea81a1620 con 0x7f5ea8107050 2024-03-26T06:20:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.498+0000 7f5ead57e640 1 -- 172.21.15.37:0/1213718623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea81a1b70 con 0x7f5ea8107050 2024-03-26T06:20:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.499+0000 7f5ead57e640 1 -- 172.21.15.37:0/1213718623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea80fd270 con 0x7f5ea8107050 2024-03-26T06:20:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.500+0000 7f5e9ffff640 1 -- 172.21.15.37:0/1213718623 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5ea0032690 con 0x7f5ea8107050 2024-03-26T06:20:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.501+0000 7f5e9ffff640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5e940a2800 0x7f5e940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.501+0000 7f5ea6ffd640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5e940a2800 0x7f5e940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.501+0000 7f5e9ffff640 1 -- 172.21.15.37:0/1213718623 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5ea00f1090 con 0x7f5ea8107050 2024-03-26T06:20:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.501+0000 7f5ea6ffd640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5e940a2800 0x7f5e940a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000bbe0 tx=0x7f5e9000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:33.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.504+0000 7f5e9ffff640 1 -- 172.21.15.37:0/1213718623 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5ea00be600 con 0x7f5ea8107050 2024-03-26T06:20:33.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.727+0000 7f5ead57e640 1 -- 172.21.15.37:0/1213718623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ea802ce70 con 0x7f5ea8107050 2024-03-26T06:20:33.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.727+0000 7f5e9ffff640 1 -- 172.21.15.37:0/1213718623 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5286 (secure 0 0 0) 0x7f5ea00bdf00 con 0x7f5ea8107050 2024-03-26T06:20:33.728 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:33.728 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T06:20:33.729 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-03-26T06:20:33.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 -- 172.21.15.37:0/1213718623 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5e940a2800 msgr2=0x7f5e940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:33.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5e940a2800 0x7f5e940a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000bbe0 tx=0x7f5e9000b390 comp rx=0 tx=0).stop 2024-03-26T06:20:33.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 -- 172.21.15.37:0/1213718623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea8107050 msgr2=0x7f5ea81002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:33.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea8107050 0x7f5ea81002e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5ea002c700 tx=0x7f5ea0032f20 comp rx=0 tx=0).stop 2024-03-26T06:20:33.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 -- 172.21.15.37:0/1213718623 shutdown_connections 2024-03-26T06:20:33.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5e940a2800 0x7f5e940a4cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:33.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea8107050 0x7f5ea81002e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:33.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8104720 0x7f5ea80ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:33.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 --2- 172.21.15.37:0/1213718623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8101df0 0x7f5ea8101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:33.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 -- 172.21.15.37:0/1213718623 >> 172.21.15.37:0/1213718623 conn(0x7f5ea80fb780 msgr2=0x7f5ea8102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:33.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 -- 172.21.15.37:0/1213718623 shutdown_connections 2024-03-26T06:20:33.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:33.729+0000 7f5ead57e640 1 -- 172.21.15.37:0/1213718623 wait complete. 2024-03-26T06:20:34.033 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:34.033 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:34.033 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:34.034 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:34.034 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:34.034 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:34.034 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:34.034 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: Deploying daemon mds.c on smithi149 2024-03-26T06:20:34.034 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6834/1494796094,v1:172.21.15.149:6835/1494796094] up:boot 2024-03-26T06:20:34.034 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: fsmap 6 up:standby 2024-03-26T06:20:34.034 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T06:20:34.034 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:33 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1213718623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:34.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:34.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:34.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:34.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:34.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:34.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:34.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:34.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: Deploying daemon mds.c on smithi149 2024-03-26T06:20:34.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6834/1494796094,v1:172.21.15.149:6835/1494796094] up:boot 2024-03-26T06:20:34.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: fsmap 6 up:standby 2024-03-26T06:20:34.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T06:20:34.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:33 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1213718623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:34.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:34.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:34.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:34.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:34.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:34.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:34.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:34.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: Deploying daemon mds.c on smithi149 2024-03-26T06:20:34.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6834/1494796094,v1:172.21.15.149:6835/1494796094] up:boot 2024-03-26T06:20:34.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: fsmap 6 up:standby 2024-03-26T06:20:34.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T06:20:34.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:33 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1213718623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:35.063 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:20:34 smithi149 systemd[1]: Starting Ceph mds.c for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:20:35.322 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:20:35 smithi149 podman[67742]: 2024-03-26 06:20:34.966089261 +0000 UTC m=+0.018027552 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:20:35.322 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:20:35 smithi149 podman[67742]: 2024-03-26 06:20:35.068980585 +0000 UTC m=+0.120918852 container create 3f6908e4e203ef9e0e9530a348fc1e72921995ba02acb6d0c221ae26ab53f572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-26T06:20:35.322 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:20:35 smithi149 podman[67742]: 2024-03-26 06:20:35.296480302 +0000 UTC m=+0.348418587 container init 3f6908e4e203ef9e0e9530a348fc1e72921995ba02acb6d0c221ae26ab53f572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:20:35.322 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:20:35 smithi149 podman[67742]: 2024-03-26 06:20:35.299118724 +0000 UTC m=+0.351057008 container start 3f6908e4e203ef9e0e9530a348fc1e72921995ba02acb6d0c221ae26ab53f572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-26T06:20:35.659 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:20:35 smithi149 ceph-mds[67761]: -- 172.21.15.149:0/3317759267 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563279ceda40 con 0x563279cc1000 2024-03-26T06:20:35.659 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:20:35 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: starting mds.c at 2024-03-26T06:20:35.659 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:20:35 smithi149 bash[67742]: 3f6908e4e203ef9e0e9530a348fc1e72921995ba02acb6d0c221ae26ab53f572 2024-03-26T06:20:35.660 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:20:35 smithi149 systemd[1]: Started Ceph mds.c for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:35.742 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:35.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.857+0000 7f18ba744640 1 -- 172.21.15.37:0/3734209874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b41082f0 msgr2=0x7f18b4108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:35.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.857+0000 7f18ba744640 1 --2- 172.21.15.37:0/3734209874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b41082f0 0x7f18b4108710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009900 tx=0x7f18a402bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:35.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.858+0000 7f18ba744640 1 -- 172.21.15.37:0/3734209874 shutdown_connections 2024-03-26T06:20:35.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.858+0000 7f18ba744640 1 --2- 172.21.15.37:0/3734209874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b410fb70 0x7f18b4111f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:35.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.858+0000 7f18ba744640 1 --2- 172.21.15.37:0/3734209874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b4108c50 0x7f18b410f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:35.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.858+0000 7f18ba744640 1 --2- 172.21.15.37:0/3734209874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b41082f0 0x7f18b4108710 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:35.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.858+0000 7f18ba744640 1 -- 172.21.15.37:0/3734209874 >> 172.21.15.37:0/3734209874 conn(0x7f18b406e500 msgr2=0x7f18b4070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:35.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.858+0000 7f18ba744640 1 -- 172.21.15.37:0/3734209874 shutdown_connections 2024-03-26T06:20:35.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.858+0000 7f18ba744640 1 -- 172.21.15.37:0/3734209874 wait complete. 2024-03-26T06:20:35.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.858+0000 7f18ba744640 1 Processor -- start 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.858+0000 7f18ba744640 1 -- start start 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18ba744640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b41082f0 0x7f18b419fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18ba744640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b4108c50 0x7f18b41a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18ba744640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b410fb70 0x7f18b41a7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18ba744640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b4115770 con 0x7f18b41082f0 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18ba744640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b41155f0 con 0x7f18b410fb70 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18ba744640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b41158f0 con 0x7f18b4108c50 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b41082f0 0x7f18b419fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b41082f0 0x7f18b419fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51252/0 (socket says 172.21.15.37:51252) 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b3fff640 1 -- 172.21.15.37:0/3647301641 learned_addr learned my addr 172.21.15.37:0/3647301641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b8ce1640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b410fb70 0x7f18b41a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:35.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b37fe640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b4108c50 0x7f18b41a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:35.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b3fff640 1 -- 172.21.15.37:0/3647301641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b4108c50 msgr2=0x7f18b41a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:35.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b3fff640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b4108c50 0x7f18b41a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:35.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b3fff640 1 -- 172.21.15.37:0/3647301641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b410fb70 msgr2=0x7f18b41a7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:35.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b3fff640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b410fb70 0x7f18b41a7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:35.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b3fff640 1 -- 172.21.15.37:0/3647301641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b41a7aa0 con 0x7f18b41082f0 2024-03-26T06:20:35.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b3fff640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b41082f0 0x7f18b419fed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009230 tx=0x7f18a4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:35.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b17fa640 1 -- 172.21.15.37:0/3647301641 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4041070 con 0x7f18b41082f0 2024-03-26T06:20:35.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b17fa640 1 -- 172.21.15.37:0/3647301641 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18a40044d0 con 0x7f18b41082f0 2024-03-26T06:20:35.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18b17fa640 1 -- 172.21.15.37:0/3647301641 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4034440 con 0x7f18b41082f0 2024-03-26T06:20:35.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18ba744640 1 -- 172.21.15.37:0/3647301641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b41a7d30 con 0x7f18b41082f0 2024-03-26T06:20:35.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.859+0000 7f18ba744640 1 -- 172.21.15.37:0/3647301641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b41a81d0 con 0x7f18b41082f0 2024-03-26T06:20:35.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.860+0000 7f18ba744640 1 -- 172.21.15.37:0/3647301641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b410e440 con 0x7f18b41082f0 2024-03-26T06:20:35.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.862+0000 7f18b17fa640 1 -- 172.21.15.37:0/3647301641 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f18a4044070 con 0x7f18b41082f0 2024-03-26T06:20:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.862+0000 7f18b17fa640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f189c0a2800 0x7f189c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.862+0000 7f18b17fa640 1 -- 172.21.15.37:0/3647301641 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f18a40f04f0 con 0x7f18b41082f0 2024-03-26T06:20:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.862+0000 7f18b37fe640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f189c0a2800 0x7f189c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.863+0000 7f18b37fe640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f189c0a2800 0x7f189c0a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18b41a1390 tx=0x7f18a800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:35.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:35.866+0000 7f18b17fa640 1 -- 172.21.15.37:0/3647301641 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f18a40bd3e0 con 0x7f18b41082f0 2024-03-26T06:20:36.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.085+0000 7f18ba744640 1 -- 172.21.15.37:0/3647301641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18b402d030 con 0x7f18b41082f0 2024-03-26T06:20:36.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.086+0000 7f18b17fa640 1 -- 172.21.15.37:0/3647301641 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7f18a40c1480 con 0x7f18b41082f0 2024-03-26T06:20:36.087 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:36.087 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T06:20:36.087 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-03-26T06:20:36.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 -- 172.21.15.37:0/3647301641 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f189c0a2800 msgr2=0x7f189c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:36.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f189c0a2800 0x7f189c0a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18b41a1390 tx=0x7f18a800b390 comp rx=0 tx=0).stop 2024-03-26T06:20:36.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 -- 172.21.15.37:0/3647301641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b41082f0 msgr2=0x7f18b419fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:36.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b41082f0 0x7f18b419fed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009230 tx=0x7f18a4005d50 comp rx=0 tx=0).stop 2024-03-26T06:20:36.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 -- 172.21.15.37:0/3647301641 shutdown_connections 2024-03-26T06:20:36.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f189c0a2800 0x7f189c0a4cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:36.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b410fb70 0x7f18b41a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:36.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b4108c50 0x7f18b41a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:36.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 --2- 172.21.15.37:0/3647301641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b41082f0 0x7f18b419fed0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:36.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 -- 172.21.15.37:0/3647301641 >> 172.21.15.37:0/3647301641 conn(0x7f18b406e500 msgr2=0x7f18b410b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:36.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 -- 172.21.15.37:0/3647301641 shutdown_connections 2024-03-26T06:20:36.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:36.088+0000 7f18ba744640 1 -- 172.21.15.37:0/3647301641 wait complete. 2024-03-26T06:20:36.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:36.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:36.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:36.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:36.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:36.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:36.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:36.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: Deploying daemon mds.h on smithi066 2024-03-26T06:20:36.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6836/801999981,v1:172.21.15.149:6837/801999981] up:boot 2024-03-26T06:20:36.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: fsmap 7 up:standby 2024-03-26T06:20:36.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:20:36.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:36 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3647301641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:36.701 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:36.701 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:36.701 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:36.701 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:36.701 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:36.701 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:36.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:36.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: Deploying daemon mds.h on smithi066 2024-03-26T06:20:36.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6836/801999981,v1:172.21.15.149:6837/801999981] up:boot 2024-03-26T06:20:36.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: fsmap 7 up:standby 2024-03-26T06:20:36.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:20:36.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:36 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3647301641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:36.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:36.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:36.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:36.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:36.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:36.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:36.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:36.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: Deploying daemon mds.h on smithi066 2024-03-26T06:20:36.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6836/801999981,v1:172.21.15.149:6837/801999981] up:boot 2024-03-26T06:20:36.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: fsmap 7 up:standby 2024-03-26T06:20:36.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:20:36.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:36 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3647301641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:37.847 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:20:37 smithi066 systemd[1]: Starting Ceph mds.h for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:20:38.101 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:38.167 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:20:37 smithi066 podman[66956]: 2024-03-26 06:20:37.896370027 +0000 UTC m=+0.114277707 container create 47ef9a70fe8e340f9fa42c8a946a67ff157188875eb53e1a96ed18ae2f850784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:20:38.167 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:20:37 smithi066 podman[66956]: 2024-03-26 06:20:37.800625381 +0000 UTC m=+0.018533074 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:20:38.167 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:20:38 smithi066 podman[66956]: 2024-03-26 06:20:38.140584993 +0000 UTC m=+0.358492677 container init 47ef9a70fe8e340f9fa42c8a946a67ff157188875eb53e1a96ed18ae2f850784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T06:20:38.167 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:20:38 smithi066 podman[66956]: 2024-03-26 06:20:38.14337255 +0000 UTC m=+0.361280238 container start 47ef9a70fe8e340f9fa42c8a946a67ff157188875eb53e1a96ed18ae2f850784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-26T06:20:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.216+0000 7ff508744640 1 -- 172.21.15.37:0/1353901202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff500100230 msgr2=0x7ff500100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.216+0000 7ff508744640 1 --2- 172.21.15.37:0/1353901202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff500100230 0x7ff500100650 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009900 tx=0x7ff4fc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.216+0000 7ff508744640 1 -- 172.21.15.37:0/1353901202 shutdown_connections 2024-03-26T06:20:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.216+0000 7ff508744640 1 --2- 172.21.15.37:0/1353901202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff500101de0 0x7ff500108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.216+0000 7ff508744640 1 --2- 172.21.15.37:0/1353901202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff500101420 0x7ff5001018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.216+0000 7ff508744640 1 --2- 172.21.15.37:0/1353901202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff500100230 0x7ff500100650 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.216+0000 7ff508744640 1 -- 172.21.15.37:0/1353901202 >> 172.21.15.37:0/1353901202 conn(0x7ff5000fb7a0 msgr2=0x7ff5000fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.216+0000 7ff508744640 1 -- 172.21.15.37:0/1353901202 shutdown_connections 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.216+0000 7ff508744640 1 -- 172.21.15.37:0/1353901202 wait complete. 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.216+0000 7ff508744640 1 Processor -- start 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.216+0000 7ff508744640 1 -- start start 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff508744640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff500100230 0x7ff500197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff508744640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff500101420 0x7ff500197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff508744640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff500101de0 0x7ff50019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff508744640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50010bec0 con 0x7ff500100230 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff508744640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50010bd40 con 0x7ff500101420 2024-03-26T06:20:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff508744640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50010c040 con 0x7ff500101de0 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff5064e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff500100230 0x7ff500197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff5064e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff500100230 0x7ff500197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48814/0 (socket says 172.21.15.37:48814) 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff5064e0640 1 -- 172.21.15.37:0/3009482603 learned_addr learned my addr 172.21.15.37:0/3009482603 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff505cdf640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff500101420 0x7ff500197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff506ce1640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff500101de0 0x7ff50019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff5064e0640 1 -- 172.21.15.37:0/3009482603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff500101de0 msgr2=0x7ff50019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff5064e0640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff500101de0 0x7ff50019ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff5064e0640 1 -- 172.21.15.37:0/3009482603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff500101420 msgr2=0x7ff500197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff5064e0640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff500101420 0x7ff500197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff5064e0640 1 -- 172.21.15.37:0/3009482603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50019f290 con 0x7ff500100230 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff505cdf640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff500101420 0x7ff500197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff5064e0640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff500100230 0x7ff500197670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc000c00 tx=0x7ff4fc02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff4f77fe640 1 -- 172.21.15.37:0/3009482603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc041070 con 0x7ff500100230 2024-03-26T06:20:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff4f77fe640 1 -- 172.21.15.37:0/3009482603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4fc004450 con 0x7ff500100230 2024-03-26T06:20:38.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff508744640 1 -- 172.21.15.37:0/3009482603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50019f520 con 0x7ff500100230 2024-03-26T06:20:38.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff4f77fe640 1 -- 172.21.15.37:0/3009482603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc034440 con 0x7ff500100230 2024-03-26T06:20:38.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.217+0000 7ff508744640 1 -- 172.21.15.37:0/3009482603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50019fa10 con 0x7ff500100230 2024-03-26T06:20:38.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.218+0000 7ff508744640 1 -- 172.21.15.37:0/3009482603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50019fc70 con 0x7ff500100230 2024-03-26T06:20:38.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.219+0000 7ff4f77fe640 1 -- 172.21.15.37:0/3009482603 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff4fc044070 con 0x7ff500100230 2024-03-26T06:20:38.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.220+0000 7ff4f77fe640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff4ec0a2800 0x7ff4ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:38.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.220+0000 7ff505cdf640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff4ec0a2800 0x7ff4ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:38.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.220+0000 7ff4f77fe640 1 -- 172.21.15.37:0/3009482603 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff4fc0f0130 con 0x7ff500100230 2024-03-26T06:20:38.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.221+0000 7ff505cdf640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff4ec0a2800 0x7ff4ec0a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff500198b80 tx=0x7ff4f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:38.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.225+0000 7ff4f77fe640 1 -- 172.21.15.37:0/3009482603 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff4fc0bd700 con 0x7ff500100230 2024-03-26T06:20:38.434 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:20:38 smithi066 ceph-mds[66975]: -- 172.21.15.66:0/317443413 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d4a08fba40 con 0x55d4a08cec00 2024-03-26T06:20:38.435 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:20:38 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h[66971]: starting mds.h at 2024-03-26T06:20:38.435 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:20:38 smithi066 bash[66956]: 47ef9a70fe8e340f9fa42c8a946a67ff157188875eb53e1a96ed18ae2f850784 2024-03-26T06:20:38.435 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:20:38 smithi066 systemd[1]: Started Ceph mds.h for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.451+0000 7ff508744640 1 -- 172.21.15.37:0/3009482603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff50019fe40 con 0x7ff500100230 2024-03-26T06:20:38.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.451+0000 7ff4f77fe640 1 -- 172.21.15.37:0/3009482603 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6868 (secure 0 0 0) 0x7ff4fc0bd000 con 0x7ff500100230 2024-03-26T06:20:38.452 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:38.452 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T06:20:38.452 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-03-26T06:20:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 -- 172.21.15.37:0/3009482603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff4ec0a2800 msgr2=0x7ff4ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff4ec0a2800 0x7ff4ec0a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff500198b80 tx=0x7ff4f0005f90 comp rx=0 tx=0).stop 2024-03-26T06:20:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 -- 172.21.15.37:0/3009482603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff500100230 msgr2=0x7ff500197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff500100230 0x7ff500197670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc000c00 tx=0x7ff4fc02c640 comp rx=0 tx=0).stop 2024-03-26T06:20:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 -- 172.21.15.37:0/3009482603 shutdown_connections 2024-03-26T06:20:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff4ec0a2800 0x7ff4ec0a4cc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:38.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff500101de0 0x7ff50019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:38.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff500101420 0x7ff500197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:38.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 --2- 172.21.15.37:0/3009482603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff500100230 0x7ff500197670 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:38.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 -- 172.21.15.37:0/3009482603 >> 172.21.15.37:0/3009482603 conn(0x7ff5000fb7a0 msgr2=0x7ff5000fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:38.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 -- 172.21.15.37:0/3009482603 shutdown_connections 2024-03-26T06:20:38.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:38.453+0000 7ff508744640 1 -- 172.21.15.37:0/3009482603 wait complete. 2024-03-26T06:20:38.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:38 smithi037 ceph-mon[31974]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:38.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:38.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:38.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:38.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:38.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:38.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:38 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:38.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:38 smithi066 ceph-mon[36504]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:38.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:38.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:38.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:38.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:38.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:38.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:38 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:38.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:38 smithi149 ceph-mon[36681]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:38 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:39.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:39 smithi149 ceph-mon[36681]: Deploying daemon mds.i on smithi149 2024-03-26T06:20:39.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:39 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] up:boot 2024-03-26T06:20:39.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:39 smithi149 ceph-mon[36681]: fsmap 8 up:standby 2024-03-26T06:20:39.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:39 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T06:20:39.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:39 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3009482603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:39.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:39 smithi037 ceph-mon[31974]: Deploying daemon mds.i on smithi149 2024-03-26T06:20:39.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:39 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] up:boot 2024-03-26T06:20:39.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:39 smithi037 ceph-mon[31974]: fsmap 8 up:standby 2024-03-26T06:20:39.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:39 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T06:20:39.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:39 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3009482603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:39.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:39 smithi066 ceph-mon[36504]: Deploying daemon mds.i on smithi149 2024-03-26T06:20:39.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:39 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] up:boot 2024-03-26T06:20:39.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:39 smithi066 ceph-mon[36504]: fsmap 8 up:standby 2024-03-26T06:20:39.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:39 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T06:20:39.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:39 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3009482603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:40.410 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:20:40 smithi149 systemd[1]: Starting Ceph mds.i for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:20:40.466 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.580+0000 7f87c6b44640 1 -- 172.21.15.37:0/3121201541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c00ff9a0 msgr2=0x7f87c00ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.580+0000 7f87c6b44640 1 --2- 172.21.15.37:0/3121201541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c00ff9a0 0x7f87c00ffdc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009900 tx=0x7f87b402bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.580+0000 7f87c6b44640 1 -- 172.21.15.37:0/3121201541 shutdown_connections 2024-03-26T06:20:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.580+0000 7f87c6b44640 1 --2- 172.21.15.37:0/3121201541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87c0101550 0x7f87c0107de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.580+0000 7f87c6b44640 1 --2- 172.21.15.37:0/3121201541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87c0100b90 0x7f87c0101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.580+0000 7f87c6b44640 1 --2- 172.21.15.37:0/3121201541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c00ff9a0 0x7f87c00ffdc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.580+0000 7f87c6b44640 1 -- 172.21.15.37:0/3121201541 >> 172.21.15.37:0/3121201541 conn(0x7f87c00faf10 msgr2=0x7f87c00fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.580+0000 7f87c6b44640 1 -- 172.21.15.37:0/3121201541 shutdown_connections 2024-03-26T06:20:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.580+0000 7f87c6b44640 1 -- 172.21.15.37:0/3121201541 wait complete. 2024-03-26T06:20:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.580+0000 7f87c6b44640 1 Processor -- start 2024-03-26T06:20:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.580+0000 7f87c6b44640 1 -- start start 2024-03-26T06:20:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c6b44640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c00ff9a0 0x7f87c0109310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c6b44640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87c0100b90 0x7f87c019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c6b44640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87c0101550 0x7f87c019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c6b44640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c010beb0 con 0x7f87c00ff9a0 2024-03-26T06:20:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c6b44640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c010bd30 con 0x7f87c0100b90 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c6b44640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c010c030 con 0x7f87c0101550 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c48e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c00ff9a0 0x7f87c0109310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c48e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c00ff9a0 0x7f87c0109310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48846/0 (socket says 172.21.15.37:48846) 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c48e0640 1 -- 172.21.15.37:0/2052297189 learned_addr learned my addr 172.21.15.37:0/2052297189 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c50e1640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87c0101550 0x7f87c019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c48e0640 1 -- 172.21.15.37:0/2052297189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87c0101550 msgr2=0x7f87c019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c48e0640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87c0101550 0x7f87c019e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c48e0640 1 -- 172.21.15.37:0/2052297189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87c0100b90 msgr2=0x7f87c019c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c48e0640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87c0100b90 0x7f87c019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c48e0640 1 -- 172.21.15.37:0/2052297189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c019f280 con 0x7f87c00ff9a0 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c48e0640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c00ff9a0 0x7f87c0109310 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000c00 tx=0x7f87b4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87bdffb640 1 -- 172.21.15.37:0/2052297189 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4041070 con 0x7f87c00ff9a0 2024-03-26T06:20:40.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87bdffb640 1 -- 172.21.15.37:0/2052297189 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87b40044d0 con 0x7f87c00ff9a0 2024-03-26T06:20:40.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87c6b44640 1 -- 172.21.15.37:0/2052297189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c019f4b0 con 0x7f87c00ff9a0 2024-03-26T06:20:40.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.581+0000 7f87bdffb640 1 -- 172.21.15.37:0/2052297189 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4034440 con 0x7f87c00ff9a0 2024-03-26T06:20:40.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.582+0000 7f87c6b44640 1 -- 172.21.15.37:0/2052297189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c019f9f0 con 0x7f87c00ff9a0 2024-03-26T06:20:40.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.583+0000 7f87c6b44640 1 -- 172.21.15.37:0/2052297189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c0106cc0 con 0x7f87c00ff9a0 2024-03-26T06:20:40.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.584+0000 7f87bdffb640 1 -- 172.21.15.37:0/2052297189 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f87b4044070 con 0x7f87c00ff9a0 2024-03-26T06:20:40.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.584+0000 7f87bdffb640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f87b00a2880 0x7f87b00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:40.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.584+0000 7f87bffff640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f87b00a2880 0x7f87b00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:40.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.585+0000 7f87bdffb640 1 -- 172.21.15.37:0/2052297189 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f87b40f0190 con 0x7f87c00ff9a0 2024-03-26T06:20:40.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.585+0000 7f87bffff640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f87b00a2880 0x7f87b00a4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87c0109080 tx=0x7f87ac005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.589+0000 7f87bdffb640 1 -- 172.21.15.37:0/2052297189 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f87b40bd6d0 con 0x7f87c00ff9a0 2024-03-26T06:20:40.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:40 smithi037 ceph-mon[31974]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:40.774 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:40 smithi149 ceph-mon[36681]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:40.774 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:20:40 smithi149 podman[68051]: 2024-03-26 06:20:40.420728316 +0000 UTC m=+0.017472034 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:20:40.774 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:20:40 smithi149 podman[68051]: 2024-03-26 06:20:40.547045479 +0000 UTC m=+0.143789179 container create 4f27664c4382a5d65423952cd0aabb92e36154c790e77ac83b105e74e6d045f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:20:40.775 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:20:40 smithi149 podman[68051]: 2024-03-26 06:20:40.774389567 +0000 UTC m=+0.371133279 container init 4f27664c4382a5d65423952cd0aabb92e36154c790e77ac83b105e74e6d045f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:20:40.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.813+0000 7f87c6b44640 1 -- 172.21.15.37:0/2052297189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87c002ce70 con 0x7f87c00ff9a0 2024-03-26T06:20:40.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.814+0000 7f87bdffb640 1 -- 172.21.15.37:0/2052297189 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6868 (secure 0 0 0) 0x7f87b40bcfd0 con 0x7f87c00ff9a0 2024-03-26T06:20:40.815 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:40.815 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T06:20:40.815 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-03-26T06:20:40.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 -- 172.21.15.37:0/2052297189 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f87b00a2880 msgr2=0x7f87b00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:40.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f87b00a2880 0x7f87b00a4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87c0109080 tx=0x7f87ac005ed0 comp rx=0 tx=0).stop 2024-03-26T06:20:40.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 -- 172.21.15.37:0/2052297189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c00ff9a0 msgr2=0x7f87c0109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:40.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c00ff9a0 0x7f87c0109310 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000c00 tx=0x7f87b4005d50 comp rx=0 tx=0).stop 2024-03-26T06:20:40.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 -- 172.21.15.37:0/2052297189 shutdown_connections 2024-03-26T06:20:40.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f87b00a2880 0x7f87b00a4d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:40.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87c0101550 0x7f87c019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:40.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87c0100b90 0x7f87c019c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:40.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 --2- 172.21.15.37:0/2052297189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c00ff9a0 0x7f87c0109310 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:40.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 -- 172.21.15.37:0/2052297189 >> 172.21.15.37:0/2052297189 conn(0x7f87c00faf10 msgr2=0x7f87c0103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:40.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 -- 172.21.15.37:0/2052297189 shutdown_connections 2024-03-26T06:20:40.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:40.816+0000 7f87c6b44640 1 -- 172.21.15.37:0/2052297189 wait complete. 2024-03-26T06:20:40.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:40 smithi066 ceph-mon[36504]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:41.159 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:20:40 smithi149 podman[68051]: 2024-03-26 06:20:40.7771288 +0000 UTC m=+0.373872504 container start 4f27664c4382a5d65423952cd0aabb92e36154c790e77ac83b105e74e6d045f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-26T06:20:41.160 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:20:40 smithi149 ceph-mds[68070]: -- 172.21.15.149:0/2884639197 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560862fc5a40 con 0x560862f98c00 2024-03-26T06:20:41.160 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:20:40 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i[68066]: starting mds.i at 2024-03-26T06:20:41.160 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:20:40 smithi149 bash[68051]: 4f27664c4382a5d65423952cd0aabb92e36154c790e77ac83b105e74e6d045f2 2024-03-26T06:20:41.160 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:20:40 smithi149 systemd[1]: Started Ceph mds.i for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:41 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2052297189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:41.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:41.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:41 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:41.739 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:41 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2052297189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:41.739 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:41.739 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:41.739 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:41.740 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:41.740 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:41.740 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:41 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:41.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:41 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2052297189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:41.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:41.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:41.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:41.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:41.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:41.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:41 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:42.828 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:42.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:42 smithi066 ceph-mon[36504]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:42.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:42 smithi066 ceph-mon[36504]: Deploying daemon mds.g on smithi037 2024-03-26T06:20:42.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:42 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] up:boot 2024-03-26T06:20:42.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:42 smithi066 ceph-mon[36504]: fsmap 9 up:standby 2024-03-26T06:20:42.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:42 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T06:20:42.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:42 smithi037 ceph-mon[31974]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:42.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:42 smithi037 ceph-mon[31974]: Deploying daemon mds.g on smithi037 2024-03-26T06:20:42.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:42 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] up:boot 2024-03-26T06:20:42.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:42 smithi037 ceph-mon[31974]: fsmap 9 up:standby 2024-03-26T06:20:42.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:42 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T06:20:42.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:42 smithi149 ceph-mon[36681]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:42.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:42 smithi149 ceph-mon[36681]: Deploying daemon mds.g on smithi037 2024-03-26T06:20:42.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:42 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] up:boot 2024-03-26T06:20:42.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:42 smithi149 ceph-mon[36681]: fsmap 9 up:standby 2024-03-26T06:20:42.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:42 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T06:20:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.945+0000 7fde91b5b640 1 -- 172.21.15.37:0/371940353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c102530 msgr2=0x7fde8c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.945+0000 7fde91b5b640 1 --2- 172.21.15.37:0/371940353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c102530 0x7fde8c1029b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fde80009900 tx=0x7fde8002bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.945+0000 7fde91b5b640 1 -- 172.21.15.37:0/371940353 shutdown_connections 2024-03-26T06:20:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.945+0000 7fde91b5b640 1 --2- 172.21.15.37:0/371940353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde8c102ef0 0x7fde8c109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.945+0000 7fde91b5b640 1 --2- 172.21.15.37:0/371940353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c102530 0x7fde8c1029b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.945+0000 7fde91b5b640 1 --2- 172.21.15.37:0/371940353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde8c101340 0x7fde8c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.945+0000 7fde91b5b640 1 -- 172.21.15.37:0/371940353 >> 172.21.15.37:0/371940353 conn(0x7fde8c0fc890 msgr2=0x7fde8c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.945+0000 7fde91b5b640 1 -- 172.21.15.37:0/371940353 shutdown_connections 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.945+0000 7fde91b5b640 1 -- 172.21.15.37:0/371940353 wait complete. 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.946+0000 7fde91b5b640 1 Processor -- start 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.946+0000 7fde91b5b640 1 -- start start 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.946+0000 7fde91b5b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde8c101340 0x7fde8c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.946+0000 7fde91b5b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde8c102530 0x7fde8c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.946+0000 7fde91b5b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c102ef0 0x7fde8c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.946+0000 7fde91b5b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c10cfd0 con 0x7fde8c102ef0 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.946+0000 7fde91b5b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c10ce50 con 0x7fde8c101340 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.946+0000 7fde91b5b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c10d150 con 0x7fde8c102530 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.946+0000 7fde8bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c102ef0 0x7fde8c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.946+0000 7fde8bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c102ef0 0x7fde8c19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48856/0 (socket says 172.21.15.37:48856) 2024-03-26T06:20:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.946+0000 7fde8bfff640 1 -- 172.21.15.37:0/3768165409 learned_addr learned my addr 172.21.15.37:0/3768165409 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde8b7fe640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde8c101340 0x7fde8c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde8affd640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde8c102530 0x7fde8c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde8bfff640 1 -- 172.21.15.37:0/3768165409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde8c102530 msgr2=0x7fde8c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde8bfff640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde8c102530 0x7fde8c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde8bfff640 1 -- 172.21.15.37:0/3768165409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde8c101340 msgr2=0x7fde8c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde8bfff640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde8c101340 0x7fde8c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde8bfff640 1 -- 172.21.15.37:0/3768165409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c19f200 con 0x7fde8c102ef0 2024-03-26T06:20:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde8b7fe640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde8c101340 0x7fde8c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:20:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde8bfff640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c102ef0 0x7fde8c19eb90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fde8400eb10 tx=0x7fde8400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:42.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde88ff9640 1 -- 172.21.15.37:0/3768165409 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84004b20 con 0x7fde8c102ef0 2024-03-26T06:20:42.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde91b5b640 1 -- 172.21.15.37:0/3768165409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8c19f4f0 con 0x7fde8c102ef0 2024-03-26T06:20:42.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde88ff9640 1 -- 172.21.15.37:0/3768165409 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde84004cc0 con 0x7fde8c102ef0 2024-03-26T06:20:42.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde88ff9640 1 -- 172.21.15.37:0/3768165409 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84010760 con 0x7fde8c102ef0 2024-03-26T06:20:42.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.947+0000 7fde91b5b640 1 -- 172.21.15.37:0/3768165409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8c19f9c0 con 0x7fde8c102ef0 2024-03-26T06:20:42.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.948+0000 7fde91b5b640 1 -- 172.21.15.37:0/3768165409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8c108660 con 0x7fde8c102ef0 2024-03-26T06:20:42.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.949+0000 7fde88ff9640 1 -- 172.21.15.37:0/3768165409 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fde84005020 con 0x7fde8c102ef0 2024-03-26T06:20:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.950+0000 7fde88ff9640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fde740a2800 0x7fde740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.950+0000 7fde8b7fe640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fde740a2800 0x7fde740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.950+0000 7fde88ff9640 1 -- 172.21.15.37:0/3768165409 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fde840cec70 con 0x7fde8c102ef0 2024-03-26T06:20:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.950+0000 7fde8b7fe640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fde740a2800 0x7fde740a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fde8c102390 tx=0x7fde7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:42.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:42.955+0000 7fde88ff9640 1 -- 172.21.15.37:0/3768165409 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fde8409c1b0 con 0x7fde8c102ef0 2024-03-26T06:20:43.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.193+0000 7fde91b5b640 1 -- 172.21.15.37:0/3768165409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde8c061ef0 con 0x7fde8c102ef0 2024-03-26T06:20:43.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.194+0000 7fde88ff9640 1 -- 172.21.15.37:0/3768165409 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7664 (secure 0 0 0) 0x7fde8409bab0 con 0x7fde8c102ef0 2024-03-26T06:20:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:43.195 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T06:20:43.195 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 10 2024-03-26T06:20:43.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 -- 172.21.15.37:0/3768165409 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fde740a2800 msgr2=0x7fde740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:43.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fde740a2800 0x7fde740a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fde8c102390 tx=0x7fde7c005f90 comp rx=0 tx=0).stop 2024-03-26T06:20:43.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 -- 172.21.15.37:0/3768165409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c102ef0 msgr2=0x7fde8c19eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:43.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c102ef0 0x7fde8c19eb90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fde8400eb10 tx=0x7fde8400c2d0 comp rx=0 tx=0).stop 2024-03-26T06:20:43.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 -- 172.21.15.37:0/3768165409 shutdown_connections 2024-03-26T06:20:43.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fde740a2800 0x7fde740a4cc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:43.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c102ef0 0x7fde8c19eb90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde8c102530 0x7fde8c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 --2- 172.21.15.37:0/3768165409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde8c101340 0x7fde8c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 -- 172.21.15.37:0/3768165409 >> 172.21.15.37:0/3768165409 conn(0x7fde8c0fc890 msgr2=0x7fde8c105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 -- 172.21.15.37:0/3768165409 shutdown_connections 2024-03-26T06:20:43.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:43.196+0000 7fde91b5b640 1 -- 172.21.15.37:0/3768165409 wait complete. 2024-03-26T06:20:43.198 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:20:43 smithi037 systemd[1]: Starting Ceph mds.g for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:20:43.463 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:43 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3768165409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:43.463 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:20:43 smithi037 podman[73860]: 2024-03-26 06:20:43.195560342 +0000 UTC m=+0.018272315 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:20:43.463 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:20:43 smithi037 podman[73860]: 2024-03-26 06:20:43.313471427 +0000 UTC m=+0.136183402 container create 3eccb2b6cbd6d3765d67f62b867f5c6c4ec30e2263dfca49a2efe842971d916c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:20:43.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:43 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3768165409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:43.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:43 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3768165409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:43.927 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:20:43 smithi037 podman[73860]: 2024-03-26 06:20:43.583566054 +0000 UTC m=+0.406278034 container init 3eccb2b6cbd6d3765d67f62b867f5c6c4ec30e2263dfca49a2efe842971d916c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-26T06:20:43.927 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:20:43 smithi037 podman[73860]: 2024-03-26 06:20:43.586369575 +0000 UTC m=+0.409081551 container start 3eccb2b6cbd6d3765d67f62b867f5c6c4ec30e2263dfca49a2efe842971d916c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5) 2024-03-26T06:20:43.927 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:20:43 smithi037 ceph-mds[73881]: -- 172.21.15.37:0/1704963096 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5642cca15a40 con 0x5642cc9e8800 2024-03-26T06:20:43.928 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:20:43 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g[73877]: starting mds.g at 2024-03-26T06:20:43.928 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:20:43 smithi037 bash[73860]: 3eccb2b6cbd6d3765d67f62b867f5c6c4ec30e2263dfca49a2efe842971d916c 2024-03-26T06:20:43.928 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:20:43 smithi037 systemd[1]: Started Ceph mds.g for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:45.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:44 smithi037 ceph-mon[31974]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:45.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:45.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:45.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:45.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:45.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:45.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:45.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:44 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.37:6838/1944002831,v1:172.21.15.37:6839/1944002831] up:boot 2024-03-26T06:20:45.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:44 smithi037 ceph-mon[31974]: fsmap 10 up:standby 2024-03-26T06:20:45.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:44 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T06:20:45.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:44 smithi066 ceph-mon[36504]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:45.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:45.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:45.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:45.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:45.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:45.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:45.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:44 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.37:6838/1944002831,v1:172.21.15.37:6839/1944002831] up:boot 2024-03-26T06:20:45.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:44 smithi066 ceph-mon[36504]: fsmap 10 up:standby 2024-03-26T06:20:45.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:44 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T06:20:45.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:44 smithi149 ceph-mon[36681]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:45.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:45.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:45.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:45.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:45.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:45.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:45.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:44 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.37:6838/1944002831,v1:172.21.15.37:6839/1944002831] up:boot 2024-03-26T06:20:45.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:44 smithi149 ceph-mon[36681]: fsmap 10 up:standby 2024-03-26T06:20:45.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:44 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T06:20:45.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:45.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.322+0000 7fb1702d7640 1 -- 172.21.15.37:0/820988168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb168101e10 msgr2=0x7fb168104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:45.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.322+0000 7fb1702d7640 1 --2- 172.21.15.37:0/820988168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb168101e10 0x7fb168104200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb158009900 tx=0x7fb15802bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:45.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.322+0000 7fb1702d7640 1 -- 172.21.15.37:0/820988168 shutdown_connections 2024-03-26T06:20:45.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.322+0000 7fb1702d7640 1 --2- 172.21.15.37:0/820988168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb168107070 0x7fb168109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:45.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.322+0000 7fb1702d7640 1 --2- 172.21.15.37:0/820988168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb168104740 0x7fb168106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:45.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.322+0000 7fb1702d7640 1 --2- 172.21.15.37:0/820988168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb168101e10 0x7fb168104200 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:45.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.322+0000 7fb1702d7640 1 -- 172.21.15.37:0/820988168 >> 172.21.15.37:0/820988168 conn(0x7fb1680fb780 msgr2=0x7fb1680fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:45.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.322+0000 7fb1702d7640 1 -- 172.21.15.37:0/820988168 shutdown_connections 2024-03-26T06:20:45.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.322+0000 7fb1702d7640 1 -- 172.21.15.37:0/820988168 wait complete. 2024-03-26T06:20:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb1702d7640 1 Processor -- start 2024-03-26T06:20:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb1702d7640 1 -- start start 2024-03-26T06:20:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb1702d7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb168101e10 0x7fb168101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb1702d7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb168104740 0x7fb1680ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb1702d7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb168107070 0x7fb168100300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb1702d7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16810cfa0 con 0x7fb168101e10 2024-03-26T06:20:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb1702d7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16810ce20 con 0x7fb168104740 2024-03-26T06:20:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb1702d7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16810d120 con 0x7fb168107070 2024-03-26T06:20:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb16e073640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb168101e10 0x7fb168101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb16e073640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb168101e10 0x7fb168101780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48882/0 (socket says 172.21.15.37:48882) 2024-03-26T06:20:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb16e073640 1 -- 172.21.15.37:0/1479056659 learned_addr learned my addr 172.21.15.37:0/1479056659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb16e874640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb168107070 0x7fb168100300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.323+0000 7fb16d872640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb168104740 0x7fb1680ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.324+0000 7fb16e073640 1 -- 172.21.15.37:0/1479056659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb168107070 msgr2=0x7fb168100300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.324+0000 7fb16e073640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb168107070 0x7fb168100300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.324+0000 7fb16e073640 1 -- 172.21.15.37:0/1479056659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb168104740 msgr2=0x7fb1680ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.324+0000 7fb16e073640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb168104740 0x7fb1680ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.324+0000 7fb16e073640 1 -- 172.21.15.37:0/1479056659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb168100b90 con 0x7fb168101e10 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.324+0000 7fb16e073640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb168101e10 0x7fb168101780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb158009230 tx=0x7fb158005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.324+0000 7fb15f7fe640 1 -- 172.21.15.37:0/1479056659 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158041070 con 0x7fb168101e10 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.324+0000 7fb15f7fe640 1 -- 172.21.15.37:0/1479056659 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1580054e0 con 0x7fb168101e10 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.324+0000 7fb1702d7640 1 -- 172.21.15.37:0/1479056659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1681a1700 con 0x7fb168101e10 2024-03-26T06:20:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.324+0000 7fb15f7fe640 1 -- 172.21.15.37:0/1479056659 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158034440 con 0x7fb168101e10 2024-03-26T06:20:45.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.324+0000 7fb1702d7640 1 -- 172.21.15.37:0/1479056659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1681a1c50 con 0x7fb168101e10 2024-03-26T06:20:45.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.325+0000 7fb1702d7640 1 -- 172.21.15.37:0/1479056659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1680fd270 con 0x7fb168101e10 2024-03-26T06:20:45.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.327+0000 7fb15f7fe640 1 -- 172.21.15.37:0/1479056659 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb158044070 con 0x7fb168101e10 2024-03-26T06:20:45.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.328+0000 7fb15f7fe640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1540a2800 0x7fb1540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:45.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.328+0000 7fb16d872640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1540a2800 0x7fb1540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:45.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.328+0000 7fb15f7fe640 1 -- 172.21.15.37:0/1479056659 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb1580f0690 con 0x7fb168101e10 2024-03-26T06:20:45.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.329+0000 7fb16d872640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1540a2800 0x7fb1540a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1681014f0 tx=0x7fb16400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.331+0000 7fb15f7fe640 1 -- 172.21.15.37:0/1479056659 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb1580bd4e0 con 0x7fb168101e10 2024-03-26T06:20:45.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.576+0000 7fb1702d7640 1 -- 172.21.15.37:0/1479056659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb16802ce70 con 0x7fb168101e10 2024-03-26T06:20:45.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.577+0000 7fb15f7fe640 1 -- 172.21.15.37:0/1479056659 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7fb1580c1580 con 0x7fb168101e10 2024-03-26T06:20:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:45.578 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14670,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1944002831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1944002831},{"type":"v1","addr":"172.21.15.37:6839","nonce":1944002831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T06:20:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 11 2024-03-26T06:20:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.579+0000 7fb1702d7640 1 -- 172.21.15.37:0/1479056659 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1540a2800 msgr2=0x7fb1540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.579+0000 7fb1702d7640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1540a2800 0x7fb1540a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1681014f0 tx=0x7fb16400b390 comp rx=0 tx=0).stop 2024-03-26T06:20:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.579+0000 7fb1702d7640 1 -- 172.21.15.37:0/1479056659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb168101e10 msgr2=0x7fb168101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.579+0000 7fb1702d7640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb168101e10 0x7fb168101780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb158009230 tx=0x7fb158005dd0 comp rx=0 tx=0).stop 2024-03-26T06:20:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.579+0000 7fb1702d7640 1 -- 172.21.15.37:0/1479056659 shutdown_connections 2024-03-26T06:20:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.580+0000 7fb1702d7640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb1540a2800 0x7fb1540a4cc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.580+0000 7fb1702d7640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb168107070 0x7fb168100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.580+0000 7fb1702d7640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb168104740 0x7fb1680ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.580+0000 7fb1702d7640 1 --2- 172.21.15.37:0/1479056659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb168101e10 0x7fb168101780 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.580+0000 7fb1702d7640 1 -- 172.21.15.37:0/1479056659 >> 172.21.15.37:0/1479056659 conn(0x7fb1680fb780 msgr2=0x7fb168102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.580+0000 7fb1702d7640 1 -- 172.21.15.37:0/1479056659 shutdown_connections 2024-03-26T06:20:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:45.580+0000 7fb1702d7640 1 -- 172.21.15.37:0/1479056659 wait complete. 2024-03-26T06:20:45.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:45 smithi037 ceph-mon[31974]: Deploying daemon mds.a on smithi037 2024-03-26T06:20:45.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:45 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1479056659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:45.949 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:20:45 smithi037 systemd[1]: Starting Ceph mds.a for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:20:46.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:45 smithi066 ceph-mon[36504]: Deploying daemon mds.a on smithi037 2024-03-26T06:20:46.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:45 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1479056659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:46.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:45 smithi149 ceph-mon[36681]: Deploying daemon mds.a on smithi037 2024-03-26T06:20:46.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:45 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1479056659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:46.281 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:20:46 smithi037 podman[74223]: 2024-03-26 06:20:45.932192149 +0000 UTC m=+0.017951327 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:20:46.282 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:20:46 smithi037 podman[74223]: 2024-03-26 06:20:46.042886903 +0000 UTC m=+0.128646068 container create 9e8f8e1ed77c37325bfca120a9dd4cbf6508b2c2699a8fa63fe47f8e3b0ec68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-26T06:20:46.282 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:20:46 smithi037 podman[74223]: 2024-03-26 06:20:46.253502151 +0000 UTC m=+0.339261316 container init 9e8f8e1ed77c37325bfca120a9dd4cbf6508b2c2699a8fa63fe47f8e3b0ec68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-26T06:20:46.282 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:20:46 smithi037 podman[74223]: 2024-03-26 06:20:46.256635216 +0000 UTC m=+0.342394382 container start 9e8f8e1ed77c37325bfca120a9dd4cbf6508b2c2699a8fa63fe47f8e3b0ec68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-26T06:20:46.282 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:20:46 smithi037 ceph-mds[74246]: -- 172.21.15.37:0/4293945002 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ef63391a40 con 0x55ef63364c00 2024-03-26T06:20:46.618 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:20:46 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a[74242]: starting mds.a at 2024-03-26T06:20:46.618 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:20:46 smithi037 bash[74223]: 9e8f8e1ed77c37325bfca120a9dd4cbf6508b2c2699a8fa63fe47f8e3b0ec68b 2024-03-26T06:20:46.618 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:20:46 smithi037 systemd[1]: Started Ceph mds.a for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:46 smithi037 ceph-mon[31974]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:46.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:46.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:46.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:46.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:46.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:46 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:47.083 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:46 smithi066 ceph-mon[36504]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:47.083 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:47.083 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:47.083 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:47.083 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:47.083 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:47.083 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:46 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:47.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:46 smithi149 ceph-mon[36681]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:47.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:47.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:47.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:47.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T06:20:47.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T06:20:47.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:46 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:47.595 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.715+0000 7fccca39d640 1 -- 172.21.15.37:0/688385441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccc4100b30 msgr2=0x7fccc41013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.715+0000 7fccca39d640 1 --2- 172.21.15.37:0/688385441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccc4100b30 0x7fccc41013b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009900 tx=0x7fccb402bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.715+0000 7fccca39d640 1 -- 172.21.15.37:0/688385441 shutdown_connections 2024-03-26T06:20:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.715+0000 7fccca39d640 1 --2- 172.21.15.37:0/688385441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccc4100b30 0x7fccc41013b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.715+0000 7fccca39d640 1 --2- 172.21.15.37:0/688385441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc4100170 0x7fccc41005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.715+0000 7fccca39d640 1 --2- 172.21.15.37:0/688385441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccc40ff410 0x7fccc40ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.715+0000 7fccca39d640 1 -- 172.21.15.37:0/688385441 >> 172.21.15.37:0/688385441 conn(0x7fccc40fad40 msgr2=0x7fccc40fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.715+0000 7fccca39d640 1 -- 172.21.15.37:0/688385441 shutdown_connections 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.715+0000 7fccca39d640 1 -- 172.21.15.37:0/688385441 wait complete. 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccca39d640 1 Processor -- start 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccca39d640 1 -- start start 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccca39d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccc40ff410 0x7fccc4199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccca39d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccc4100170 0x7fccc4199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccca39d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc4100b30 0x7fccc41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccca39d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc410f040 con 0x7fccc40ff410 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccca39d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc410eec0 con 0x7fccc4100b30 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccca39d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc410f1c0 con 0x7fccc4100170 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc939b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccc40ff410 0x7fccc4199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc939b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccc40ff410 0x7fccc4199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50258/0 (socket says 172.21.15.37:50258) 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc939b640 1 -- 172.21.15.37:0/3022675807 learned_addr learned my addr 172.21.15.37:0/3022675807 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc9b9c640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc4100b30 0x7fccc41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc8b9a640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccc4100170 0x7fccc4199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc939b640 1 -- 172.21.15.37:0/3022675807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccc4100170 msgr2=0x7fccc4199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc939b640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccc4100170 0x7fccc4199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc939b640 1 -- 172.21.15.37:0/3022675807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc4100b30 msgr2=0x7fccc41a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc939b640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc4100b30 0x7fccc41a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc939b640 1 -- 172.21.15.37:0/3022675807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc41a1240 con 0x7fccc40ff410 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc939b640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccc40ff410 0x7fccc4199620 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00b4a0 tx=0x7fccbc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.716+0000 7fccc27fc640 1 -- 172.21.15.37:0/3022675807 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc014070 con 0x7fccc40ff410 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.717+0000 7fccc27fc640 1 -- 172.21.15.37:0/3022675807 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccbc010af0 con 0x7fccc40ff410 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.717+0000 7fccca39d640 1 -- 172.21.15.37:0/3022675807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc41a1530 con 0x7fccc40ff410 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.717+0000 7fccc27fc640 1 -- 172.21.15.37:0/3022675807 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc017d10 con 0x7fccc40ff410 2024-03-26T06:20:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.717+0000 7fccca39d640 1 -- 172.21.15.37:0/3022675807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc41a1ad0 con 0x7fccc40ff410 2024-03-26T06:20:47.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.718+0000 7fccca39d640 1 -- 172.21.15.37:0/3022675807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc410b030 con 0x7fccc40ff410 2024-03-26T06:20:47.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.719+0000 7fccc27fc640 1 -- 172.21.15.37:0/3022675807 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fccbc017070 con 0x7fccc40ff410 2024-03-26T06:20:47.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.720+0000 7fccc27fc640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fccac0a2880 0x7fccac0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:47.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.720+0000 7fccc8b9a640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fccac0a2880 0x7fccac0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:47.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.720+0000 7fccc27fc640 1 -- 172.21.15.37:0/3022675807 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fccbc0cd890 con 0x7fccc40ff410 2024-03-26T06:20:47.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.720+0000 7fccc8b9a640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fccac0a2880 0x7fccac0a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fccc419ab30 tx=0x7fccb8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:47.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.724+0000 7fccc27fc640 1 -- 172.21.15.37:0/3022675807 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fccbc09add0 con 0x7fccc40ff410 2024-03-26T06:20:47.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.946+0000 7fccca39d640 1 -- 172.21.15.37:0/3022675807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccc402ce70 con 0x7fccc40ff410 2024-03-26T06:20:47.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.947+0000 7fccc27fc640 1 -- 172.21.15.37:0/3022675807 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9245 (secure 0 0 0) 0x7fccbc09a6d0 con 0x7fccc40ff410 2024-03-26T06:20:47.948 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:47.948 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14670,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1944002831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1944002831},{"type":"v1","addr":"172.21.15.37:6839","nonce":1944002831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/933565564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":933565564},{"type":"v1","addr":"172.21.15.37:6841","nonce":933565564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T06:20:47.948 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 12 2024-03-26T06:20:47.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:47 smithi037 ceph-mon[31974]: Deploying daemon mds.b on smithi066 2024-03-26T06:20:47.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:47 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] up:boot 2024-03-26T06:20:47.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:47 smithi037 ceph-mon[31974]: fsmap 11 up:standby 2024-03-26T06:20:47.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T06:20:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 -- 172.21.15.37:0/3022675807 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fccac0a2880 msgr2=0x7fccac0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fccac0a2880 0x7fccac0a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fccc419ab30 tx=0x7fccb8009500 comp rx=0 tx=0).stop 2024-03-26T06:20:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 -- 172.21.15.37:0/3022675807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccc40ff410 msgr2=0x7fccc4199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccc40ff410 0x7fccc4199620 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00b4a0 tx=0x7fccbc00b960 comp rx=0 tx=0).stop 2024-03-26T06:20:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 -- 172.21.15.37:0/3022675807 shutdown_connections 2024-03-26T06:20:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fccac0a2880 0x7fccac0a4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccc4100b30 0x7fccc41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccc4100170 0x7fccc4199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 --2- 172.21.15.37:0/3022675807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccc40ff410 0x7fccc4199620 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 -- 172.21.15.37:0/3022675807 >> 172.21.15.37:0/3022675807 conn(0x7fccc40fad40 msgr2=0x7fccc40fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:47.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 -- 172.21.15.37:0/3022675807 shutdown_connections 2024-03-26T06:20:47.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:47.949+0000 7fccca39d640 1 -- 172.21.15.37:0/3022675807 wait complete. 2024-03-26T06:20:48.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:47 smithi066 ceph-mon[36504]: Deploying daemon mds.b on smithi066 2024-03-26T06:20:48.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:47 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] up:boot 2024-03-26T06:20:48.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:47 smithi066 ceph-mon[36504]: fsmap 11 up:standby 2024-03-26T06:20:48.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T06:20:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:47 smithi149 ceph-mon[36681]: Deploying daemon mds.b on smithi066 2024-03-26T06:20:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:47 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] up:boot 2024-03-26T06:20:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:47 smithi149 ceph-mon[36681]: fsmap 11 up:standby 2024-03-26T06:20:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T06:20:48.731 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:20:48 smithi066 systemd[1]: Starting Ceph mds.b for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:20:49.002 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:48 smithi066 ceph-mon[36504]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:49.002 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:48 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3022675807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:49.003 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:20:48 smithi066 podman[67260]: 2024-03-26 06:20:48.759385668 +0000 UTC m=+0.018148491 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:20:49.003 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:20:48 smithi066 podman[67260]: 2024-03-26 06:20:48.874855379 +0000 UTC m=+0.133618229 container create 442f7c8a9335ae8c712d1665c787d179c7d85d15abb7d917e122673501a49588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-26T06:20:49.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:48 smithi149 ceph-mon[36681]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:48 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3022675807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:49.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:48 smithi037 ceph-mon[31974]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:49.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:48 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3022675807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:49.347 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:20:49 smithi066 podman[67260]: 2024-03-26 06:20:49.102244908 +0000 UTC m=+0.361007717 container init 442f7c8a9335ae8c712d1665c787d179c7d85d15abb7d917e122673501a49588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-26T06:20:49.347 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:20:49 smithi066 podman[67260]: 2024-03-26 06:20:49.104965795 +0000 UTC m=+0.363728601 container start 442f7c8a9335ae8c712d1665c787d179c7d85d15abb7d917e122673501a49588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-26T06:20:49.348 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:20:49 smithi066 ceph-mds[67279]: -- 172.21.15.66:0/811589212 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55729f86fa40 con 0x55729f842800 2024-03-26T06:20:49.348 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:20:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b[67275]: starting mds.b at 2024-03-26T06:20:49.348 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:20:49 smithi066 bash[67260]: 442f7c8a9335ae8c712d1665c787d179c7d85d15abb7d917e122673501a49588 2024-03-26T06:20:49.348 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:20:49 smithi066 systemd[1]: Started Ceph mds.b for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:20:49.962 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:20:50.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.078+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/1558621241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd64101db0 msgr2=0x7fdd641041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.078+0000 7fdd6bf7e640 1 --2- 172.21.15.37:0/1558621241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd64101db0 0x7fdd641041a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009900 tx=0x7fdd6002bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:50.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.078+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/1558621241 shutdown_connections 2024-03-26T06:20:50.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.078+0000 7fdd6bf7e640 1 --2- 172.21.15.37:0/1558621241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd64107010 0x7fdd64109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.078+0000 7fdd6bf7e640 1 --2- 172.21.15.37:0/1558621241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd641046e0 0x7fdd64106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.078+0000 7fdd6bf7e640 1 --2- 172.21.15.37:0/1558621241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd64101db0 0x7fdd641041a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.078+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/1558621241 >> 172.21.15.37:0/1558621241 conn(0x7fdd640fb7a0 msgr2=0x7fdd640fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:50.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.078+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/1558621241 shutdown_connections 2024-03-26T06:20:50.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.078+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/1558621241 wait complete. 2024-03-26T06:20:50.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.079+0000 7fdd6bf7e640 1 Processor -- start 2024-03-26T06:20:50.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.079+0000 7fdd6bf7e640 1 -- start start 2024-03-26T06:20:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.079+0000 7fdd6bf7e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd64101db0 0x7fdd64197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.079+0000 7fdd6bf7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd641046e0 0x7fdd64197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.079+0000 7fdd6bf7e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd64107010 0x7fdd6419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.079+0000 7fdd6bf7e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6410cfc0 con 0x7fdd641046e0 2024-03-26T06:20:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.079+0000 7fdd6bf7e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6410ce40 con 0x7fdd64107010 2024-03-26T06:20:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.079+0000 7fdd6bf7e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6410d140 con 0x7fdd64101db0 2024-03-26T06:20:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd69519640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd641046e0 0x7fdd64197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd69519640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd641046e0 0x7fdd64197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50284/0 (socket says 172.21.15.37:50284) 2024-03-26T06:20:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd69519640 1 -- 172.21.15.37:0/3070416959 learned_addr learned my addr 172.21.15.37:0/3070416959 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd69d1a640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd64101db0 0x7fdd64197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd6a51b640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd64107010 0x7fdd6419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd69519640 1 -- 172.21.15.37:0/3070416959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd64101db0 msgr2=0x7fdd64197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd69519640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd64101db0 0x7fdd64197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd69519640 1 -- 172.21.15.37:0/3070416959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd64107010 msgr2=0x7fdd6419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd69519640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd64107010 0x7fdd6419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd69519640 1 -- 172.21.15.37:0/3070416959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6419f290 con 0x7fdd641046e0 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd6a51b640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd64107010 0x7fdd6419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd69d1a640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd64101db0 0x7fdd64197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd69519640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd641046e0 0x7fdd64197bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdd54005fd0 tx=0x7fdd5400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd5affd640 1 -- 172.21.15.37:0/3070416959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54017070 con 0x7fdd641046e0 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/3070416959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6419f580 con 0x7fdd641046e0 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd5affd640 1 -- 172.21.15.37:0/3070416959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd54009c30 con 0x7fdd641046e0 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd5affd640 1 -- 172.21.15.37:0/3070416959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54010460 con 0x7fdd641046e0 2024-03-26T06:20:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.080+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/3070416959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6419fad0 con 0x7fdd641046e0 2024-03-26T06:20:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.081+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/3070416959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd640fd290 con 0x7fdd641046e0 2024-03-26T06:20:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.082+0000 7fdd5affd640 1 -- 172.21.15.37:0/3070416959 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdd54003e70 con 0x7fdd641046e0 2024-03-26T06:20:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.083+0000 7fdd5affd640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdd500a2800 0x7fdd500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.083+0000 7fdd69d1a640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdd500a2800 0x7fdd500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.083+0000 7fdd5affd640 1 -- 172.21.15.37:0/3070416959 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdd540ce270 con 0x7fdd641046e0 2024-03-26T06:20:50.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.084+0000 7fdd69d1a640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdd500a2800 0x7fdd500a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009230 tx=0x7fdd60005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:50.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.088+0000 7fdd5affd640 1 -- 172.21.15.37:0/3070416959 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fdd54020030 con 0x7fdd641046e0 2024-03-26T06:20:50.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.313+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/3070416959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd6402d040 con 0x7fdd641046e0 2024-03-26T06:20:50.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.314+0000 7fdd5affd640 1 -- 172.21.15.37:0/3070416959 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10037 (secure 0 0 0) 0x7fdd5409b7b0 con 0x7fdd641046e0 2024-03-26T06:20:50.315 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:50.315 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14670,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1944002831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1944002831},{"type":"v1","addr":"172.21.15.37:6839","nonce":1944002831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/933565564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":933565564},{"type":"v1","addr":"172.21.15.37:6841","nonce":933565564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1420321367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1420321367},{"type":"v1","addr":"172.21.15.66:6839","nonce":1420321367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-26T06:20:50.315 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 13 2024-03-26T06:20:50.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/3070416959 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdd500a2800 msgr2=0x7fdd500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdd500a2800 0x7fdd500a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009230 tx=0x7fdd60005fd0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/3070416959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd641046e0 msgr2=0x7fdd64197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd641046e0 0x7fdd64197bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdd54005fd0 tx=0x7fdd5400dde0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/3070416959 shutdown_connections 2024-03-26T06:20:50.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdd500a2800 0x7fdd500a4cc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd64107010 0x7fdd6419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd641046e0 0x7fdd64197bb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 --2- 172.21.15.37:0/3070416959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd64101db0 0x7fdd64197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/3070416959 >> 172.21.15.37:0/3070416959 conn(0x7fdd640fb7a0 msgr2=0x7fdd641027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:50.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/3070416959 shutdown_connections 2024-03-26T06:20:50.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.316+0000 7fdd6bf7e640 1 -- 172.21.15.37:0/3070416959 wait complete. 2024-03-26T06:20:50.329 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.434+0000 7fafb9c9a640 1 -- 172.21.15.37:0/4002494930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb40ff5f0 msgr2=0x7fafb40ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.434+0000 7fafb9c9a640 1 --2- 172.21.15.37:0/4002494930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb40ff5f0 0x7fafb40ffa10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009930 tx=0x7fafa402bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.434+0000 7fafb9c9a640 1 -- 172.21.15.37:0/4002494930 shutdown_connections 2024-03-26T06:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.434+0000 7fafb9c9a640 1 --2- 172.21.15.37:0/4002494930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb4102f80 0x7fafb4109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.434+0000 7fafb9c9a640 1 --2- 172.21.15.37:0/4002494930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafb41025c0 0x7fafb4102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.434+0000 7fafb9c9a640 1 --2- 172.21.15.37:0/4002494930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb40ff5f0 0x7fafb40ffa10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.434+0000 7fafb9c9a640 1 -- 172.21.15.37:0/4002494930 >> 172.21.15.37:0/4002494930 conn(0x7fafb40faf20 msgr2=0x7fafb40fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.434+0000 7fafb9c9a640 1 -- 172.21.15.37:0/4002494930 shutdown_connections 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.434+0000 7fafb9c9a640 1 -- 172.21.15.37:0/4002494930 wait complete. 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.434+0000 7fafb9c9a640 1 Processor -- start 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb9c9a640 1 -- start start 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb9c9a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb40ff5f0 0x7fafb41976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb9c9a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafb41025c0 0x7fafb4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb9c9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4102f80 0x7fafb419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb9c9a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb410d020 con 0x7fafb4102f80 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb9c9a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb410cea0 con 0x7fafb41025c0 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb9c9a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb410d1a0 con 0x7fafb40ff5f0 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4102f80 0x7fafb419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4102f80 0x7fafb419ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50312/0 (socket says 172.21.15.37:50312) 2024-03-26T06:20:50.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb3fff640 1 -- 172.21.15.37:0/2457184455 learned_addr learned my addr 172.21.15.37:0/2457184455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb2ffd640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafb41025c0 0x7fafb4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb37fe640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb40ff5f0 0x7fafb41976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb3fff640 1 -- 172.21.15.37:0/2457184455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb40ff5f0 msgr2=0x7fafb41976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb3fff640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb40ff5f0 0x7fafb41976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb3fff640 1 -- 172.21.15.37:0/2457184455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafb41025c0 msgr2=0x7fafb4197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb3fff640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafb41025c0 0x7fafb4197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb3fff640 1 -- 172.21.15.37:0/2457184455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb419f300 con 0x7fafb4102f80 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb2ffd640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafb41025c0 0x7fafb4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.435+0000 7fafb3fff640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4102f80 0x7fafb419ec90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fafac00b970 tx=0x7fafac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.436+0000 7fafb0ff9640 1 -- 172.21.15.37:0/2457184455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac019070 con 0x7fafb4102f80 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.436+0000 7fafb0ff9640 1 -- 172.21.15.37:0/2457184455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafac00cb40 con 0x7fafb4102f80 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.436+0000 7fafb0ff9640 1 -- 172.21.15.37:0/2457184455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac007650 con 0x7fafb4102f80 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.436+0000 7fafb9c9a640 1 -- 172.21.15.37:0/2457184455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb419f5f0 con 0x7fafb4102f80 2024-03-26T06:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.436+0000 7fafb9c9a640 1 -- 172.21.15.37:0/2457184455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb419faf0 con 0x7fafb4102f80 2024-03-26T06:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.437+0000 7fafb9c9a640 1 -- 172.21.15.37:0/2457184455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb41086f0 con 0x7fafb4102f80 2024-03-26T06:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.438+0000 7fafb0ff9640 1 -- 172.21.15.37:0/2457184455 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fafac01c070 con 0x7fafb4102f80 2024-03-26T06:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.438+0000 7fafb0ff9640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf9c0a2800 0x7faf9c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.438+0000 7fafb37fe640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf9c0a2800 0x7faf9c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.438+0000 7fafb0ff9640 1 -- 172.21.15.37:0/2457184455 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fafac0cdb90 con 0x7fafb4102f80 2024-03-26T06:20:50.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.439+0000 7fafb37fe640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf9c0a2800 0x7faf9c0a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fafa402c7b0 tx=0x7fafa4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:50.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.443+0000 7fafb0ff9640 1 -- 172.21.15.37:0/2457184455 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fafac09aa80 con 0x7fafb4102f80 2024-03-26T06:20:50.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:50 smithi037 ceph-mon[31974]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:50.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:50.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:50 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] up:boot 2024-03-26T06:20:50.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:50 smithi037 ceph-mon[31974]: fsmap 12 up:standby 2024-03-26T06:20:50.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:50 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T06:20:50.563 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:50 smithi066 ceph-mon[36504]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:50.564 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.564 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.564 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.564 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.564 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.564 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:50.564 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:50 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] up:boot 2024-03-26T06:20:50.564 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:50 smithi066 ceph-mon[36504]: fsmap 12 up:standby 2024-03-26T06:20:50.564 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:50 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T06:20:50.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.628+0000 7fafb9c9a640 1 -- 172.21.15.37:0/2457184455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafb4061ef0 con 0x7fafb4102f80 2024-03-26T06:20:50.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.629+0000 7fafb0ff9640 1 -- 172.21.15.37:0/2457184455 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18382 (secure 0 0 0) 0x7fafac09eb20 con 0x7fafb4102f80 2024-03-26T06:20:50.630 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:50.630 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:20:04.052278+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:20:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 -- 172.21.15.37:0/2457184455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf9c0a2800 msgr2=0x7faf9c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf9c0a2800 0x7faf9c0a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fafa402c7b0 tx=0x7fafa4000bc0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 -- 172.21.15.37:0/2457184455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4102f80 msgr2=0x7fafb419ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4102f80 0x7fafb419ec90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fafac00b970 tx=0x7fafac00be30 comp rx=0 tx=0).stop 2024-03-26T06:20:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 -- 172.21.15.37:0/2457184455 shutdown_connections 2024-03-26T06:20:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf9c0a2800 0x7faf9c0a4cc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb4102f80 0x7fafb419ec90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fafb41025c0 0x7fafb4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 --2- 172.21.15.37:0/2457184455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb40ff5f0 0x7fafb41976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 -- 172.21.15.37:0/2457184455 >> 172.21.15.37:0/2457184455 conn(0x7fafb40faf20 msgr2=0x7fafb41058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:50.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 -- 172.21.15.37:0/2457184455 shutdown_connections 2024-03-26T06:20:50.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.631+0000 7fafb9c9a640 1 -- 172.21.15.37:0/2457184455 wait complete. 2024-03-26T06:20:50.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T06:16:17.005303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T06:20:50.643 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T06:20:50.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:50 smithi149 ceph-mon[36681]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:50.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:50.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:50.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:50 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] up:boot 2024-03-26T06:20:50.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:50 smithi149 ceph-mon[36681]: fsmap 12 up:standby 2024-03-26T06:20:50.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:50 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T06:20:50.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.749+0000 7f364f777640 1 -- 172.21.15.37:0/2204993195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36481046a0 msgr2=0x7f3648106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.749+0000 7f364f777640 1 --2- 172.21.15.37:0/2204993195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36481046a0 0x7f3648106a90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3644009900 tx=0x7f364402bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:50.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.749+0000 7f364f777640 1 -- 172.21.15.37:0/2204993195 shutdown_connections 2024-03-26T06:20:50.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.749+0000 7f364f777640 1 --2- 172.21.15.37:0/2204993195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3648106fd0 0x7f3648109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.749+0000 7f364f777640 1 --2- 172.21.15.37:0/2204993195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36481046a0 0x7f3648106a90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.749+0000 7f364f777640 1 --2- 172.21.15.37:0/2204993195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3648101d70 0x7f3648104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.749+0000 7f364f777640 1 -- 172.21.15.37:0/2204993195 >> 172.21.15.37:0/2204993195 conn(0x7f36480fb780 msgr2=0x7f36480fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.749+0000 7f364f777640 1 -- 172.21.15.37:0/2204993195 shutdown_connections 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.749+0000 7f364f777640 1 -- 172.21.15.37:0/2204993195 wait complete. 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.749+0000 7f364f777640 1 Processor -- start 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.749+0000 7f364f777640 1 -- start start 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364f777640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3648101d70 0x7f3648197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364f777640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36481046a0 0x7f3648197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364f777640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3648106fd0 0x7f364819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364f777640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364810cf80 con 0x7f3648101d70 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364f777640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364810ce00 con 0x7f36481046a0 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364f777640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364810d100 con 0x7f3648106fd0 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364d513640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3648101d70 0x7f3648197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364d513640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3648101d70 0x7f3648197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50320/0 (socket says 172.21.15.37:50320) 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364d513640 1 -- 172.21.15.37:0/2077381673 learned_addr learned my addr 172.21.15.37:0/2077381673 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364cd12640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36481046a0 0x7f3648197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364dd14640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3648106fd0 0x7f364819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364cd12640 1 -- 172.21.15.37:0/2077381673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3648106fd0 msgr2=0x7f364819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364cd12640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3648106fd0 0x7f364819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364cd12640 1 -- 172.21.15.37:0/2077381673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3648101d70 msgr2=0x7f3648197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364cd12640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3648101d70 0x7f3648197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364cd12640 1 -- 172.21.15.37:0/2077381673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364819f270 con 0x7f36481046a0 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364d513640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3648101d70 0x7f3648197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f364cd12640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36481046a0 0x7f3648197b90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f364402c170 tx=0x7f364402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.750+0000 7f363e7fc640 1 -- 172.21.15.37:0/2077381673 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644041070 con 0x7f36481046a0 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.751+0000 7f364f777640 1 -- 172.21.15.37:0/2077381673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364819f500 con 0x7f36481046a0 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.751+0000 7f363e7fc640 1 -- 172.21.15.37:0/2077381673 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3644004450 con 0x7f36481046a0 2024-03-26T06:20:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.751+0000 7f363e7fc640 1 -- 172.21.15.37:0/2077381673 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644003e50 con 0x7f36481046a0 2024-03-26T06:20:50.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.751+0000 7f364f777640 1 -- 172.21.15.37:0/2077381673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364819fa10 con 0x7f36481046a0 2024-03-26T06:20:50.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.751+0000 7f364f777640 1 -- 172.21.15.37:0/2077381673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36480fd270 con 0x7f36481046a0 2024-03-26T06:20:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.753+0000 7f363e7fc640 1 -- 172.21.15.37:0/2077381673 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3644044070 con 0x7f36481046a0 2024-03-26T06:20:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.753+0000 7f363e7fc640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a29a0 0x7f36340a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.753+0000 7f363e7fc640 1 -- 172.21.15.37:0/2077381673 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f36440f02e0 con 0x7f36481046a0 2024-03-26T06:20:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.753+0000 7f364d513640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a29a0 0x7f36340a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.754+0000 7f364d513640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a29a0 0x7f36340a4e60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3638011550 tx=0x7f3638005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.757+0000 7f363e7fc640 1 -- 172.21.15.37:0/2077381673 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f36440bd130 con 0x7f36481046a0 2024-03-26T06:20:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.954+0000 7f364f777640 1 -- 172.21.15.37:0/2077381673 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f364802d070 con 0x7f36481046a0 2024-03-26T06:20:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.954+0000 7f363e7fc640 1 -- 172.21.15.37:0/2077381673 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36440c11d0 con 0x7f36481046a0 2024-03-26T06:20:50.955 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-26T06:20:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 -- 172.21.15.37:0/2077381673 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a29a0 msgr2=0x7f36340a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a29a0 0x7f36340a4e60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3638011550 tx=0x7f3638005f90 comp rx=0 tx=0).stop 2024-03-26T06:20:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 -- 172.21.15.37:0/2077381673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36481046a0 msgr2=0x7f3648197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36481046a0 0x7f3648197b90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f364402c170 tx=0x7f364402c650 comp rx=0 tx=0).stop 2024-03-26T06:20:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 -- 172.21.15.37:0/2077381673 shutdown_connections 2024-03-26T06:20:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a29a0 0x7f36340a4e60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3648106fd0 0x7f364819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36481046a0 0x7f3648197b90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 --2- 172.21.15.37:0/2077381673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3648101d70 0x7f3648197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:50.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 -- 172.21.15.37:0/2077381673 >> 172.21.15.37:0/2077381673 conn(0x7f36480fb780 msgr2=0x7f3648102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:50.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 -- 172.21.15.37:0/2077381673 shutdown_connections 2024-03-26T06:20:50.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:50.957+0000 7f364f777640 1 -- 172.21.15.37:0/2077381673 wait complete. 2024-03-26T06:20:50.970 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-26T06:20:51.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.083+0000 7fc27ca27640 1 -- 172.21.15.37:0/3953840650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc278101420 msgr2=0x7fc2781018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:51.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.083+0000 7fc27ca27640 1 --2- 172.21.15.37:0/3953840650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc278101420 0x7fc2781018a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc260009900 tx=0x7fc26002bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:51.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.083+0000 7fc27ca27640 1 -- 172.21.15.37:0/3953840650 shutdown_connections 2024-03-26T06:20:51.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.083+0000 7fc27ca27640 1 --2- 172.21.15.37:0/3953840650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc278101de0 0x7fc278108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.083+0000 7fc27ca27640 1 --2- 172.21.15.37:0/3953840650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc278101420 0x7fc2781018a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.083+0000 7fc27ca27640 1 --2- 172.21.15.37:0/3953840650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc278100230 0x7fc278100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.083+0000 7fc27ca27640 1 -- 172.21.15.37:0/3953840650 >> 172.21.15.37:0/3953840650 conn(0x7fc2780fb7a0 msgr2=0x7fc2780fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.083+0000 7fc27ca27640 1 -- 172.21.15.37:0/3953840650 shutdown_connections 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.083+0000 7fc27ca27640 1 -- 172.21.15.37:0/3953840650 wait complete. 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.084+0000 7fc27ca27640 1 Processor -- start 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.084+0000 7fc27ca27640 1 -- start start 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.084+0000 7fc27ca27640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc278100230 0x7fc278197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.084+0000 7fc27ca27640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc278101420 0x7fc278197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.084+0000 7fc27ca27640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc278101de0 0x7fc27819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.084+0000 7fc27ca27640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27810bec0 con 0x7fc278101420 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.084+0000 7fc27ca27640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27810bd40 con 0x7fc278101de0 2024-03-26T06:20:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.084+0000 7fc27ca27640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27810c040 con 0x7fc278100230 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.084+0000 7fc26dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc278101420 0x7fc278197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc27659c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc278100230 0x7fc278197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc276d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc278101de0 0x7fc27819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc26dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc278101420 0x7fc278197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50332/0 (socket says 172.21.15.37:50332) 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc27659c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc278100230 0x7fc278197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:57110/0 (socket says 172.21.15.37:57110) 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc26dd9b640 1 -- 172.21.15.37:0/3640019961 learned_addr learned my addr 172.21.15.37:0/3640019961 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc276d9d640 1 -- 172.21.15.37:0/3640019961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc278100230 msgr2=0x7fc278197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc276d9d640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc278100230 0x7fc278197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc276d9d640 1 -- 172.21.15.37:0/3640019961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc278101420 msgr2=0x7fc278197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc276d9d640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc278101420 0x7fc278197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc276d9d640 1 -- 172.21.15.37:0/3640019961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27819f2e0 con 0x7fc278101de0 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc26dd9b640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc278101420 0x7fc278197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc27659c640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc278100230 0x7fc278197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:20:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc276d9d640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc278101de0 0x7fc27819ec70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc27000da10 tx=0x7fc27000ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc26ffff640 1 -- 172.21.15.37:0/3640019961 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27000bd70 con 0x7fc278101de0 2024-03-26T06:20:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc27ca27640 1 -- 172.21.15.37:0/3640019961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27819f5d0 con 0x7fc278101de0 2024-03-26T06:20:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc26ffff640 1 -- 172.21.15.37:0/3640019961 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc270003e50 con 0x7fc278101de0 2024-03-26T06:20:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc26ffff640 1 -- 172.21.15.37:0/3640019961 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2700106a0 con 0x7fc278101de0 2024-03-26T06:20:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.085+0000 7fc27ca27640 1 -- 172.21.15.37:0/3640019961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27819fb20 con 0x7fc278101de0 2024-03-26T06:20:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.087+0000 7fc27ca27640 1 -- 172.21.15.37:0/3640019961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27819fde0 con 0x7fc278101de0 2024-03-26T06:20:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.087+0000 7fc26ffff640 1 -- 172.21.15.37:0/3640019961 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc270003970 con 0x7fc278101de0 2024-03-26T06:20:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.088+0000 7fc26ffff640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc2640a2800 0x7fc2640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:51.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.088+0000 7fc27659c640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc2640a2800 0x7fc2640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:51.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.088+0000 7fc26ffff640 1 -- 172.21.15.37:0/3640019961 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc2700ce990 con 0x7fc278101de0 2024-03-26T06:20:51.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.088+0000 7fc27659c640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc2640a2800 0x7fc2640a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc278101280 tx=0x7fc26800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:51.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.093+0000 7fc26ffff640 1 -- 172.21.15.37:0/3640019961 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc27009bed0 con 0x7fc278101de0 2024-03-26T06:20:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.290+0000 7fc27ca27640 1 -- 172.21.15.37:0/3640019961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc27819ffb0 con 0x7fc278101de0 2024-03-26T06:20:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.290+0000 7fc26ffff640 1 -- 172.21.15.37:0/3640019961 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc27009b7d0 con 0x7fc278101de0 2024-03-26T06:20:51.291 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:51.291 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-03-26T06:20:51.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:20:51.291 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-03-26T06:20:51.291 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-03-26T06:20:51.292 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:20:51.292 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-03-26T06:20:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.292+0000 7fc27ca27640 1 -- 172.21.15.37:0/3640019961 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc2640a2800 msgr2=0x7fc2640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.292+0000 7fc27ca27640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc2640a2800 0x7fc2640a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc278101280 tx=0x7fc26800b390 comp rx=0 tx=0).stop 2024-03-26T06:20:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.292+0000 7fc27ca27640 1 -- 172.21.15.37:0/3640019961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc278101de0 msgr2=0x7fc27819ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.292+0000 7fc27ca27640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc278101de0 0x7fc27819ec70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc27000da10 tx=0x7fc27000ded0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.292+0000 7fc27ca27640 1 -- 172.21.15.37:0/3640019961 shutdown_connections 2024-03-26T06:20:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.292+0000 7fc27ca27640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc2640a2800 0x7fc2640a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.292+0000 7fc27ca27640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc278101de0 0x7fc27819ec70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.292+0000 7fc27ca27640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc278101420 0x7fc278197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.292+0000 7fc27ca27640 1 --2- 172.21.15.37:0/3640019961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc278100230 0x7fc278197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.292+0000 7fc27ca27640 1 -- 172.21.15.37:0/3640019961 >> 172.21.15.37:0/3640019961 conn(0x7fc2780fb7a0 msgr2=0x7fc2780fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.293+0000 7fc27ca27640 1 -- 172.21.15.37:0/3640019961 shutdown_connections 2024-03-26T06:20:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.293+0000 7fc27ca27640 1 -- 172.21.15.37:0/3640019961 wait complete. 2024-03-26T06:20:51.304 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-26T06:20:51.305 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-26T06:20:51.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 -- 172.21.15.37:0/1287254080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e6c101db0 msgr2=0x7f9e6c108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 --2- 172.21.15.37:0/1287254080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e6c101db0 0x7f9e6c108640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009ca0 tx=0x7f9e6402c0f0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 -- 172.21.15.37:0/1287254080 shutdown_connections 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 --2- 172.21.15.37:0/1287254080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e6c101db0 0x7f9e6c108640 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 --2- 172.21.15.37:0/1287254080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6c1013f0 0x7f9e6c101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 --2- 172.21.15.37:0/1287254080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e6c100200 0x7f9e6c100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 -- 172.21.15.37:0/1287254080 >> 172.21.15.37:0/1287254080 conn(0x7f9e6c0fb790 msgr2=0x7f9e6c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 -- 172.21.15.37:0/1287254080 shutdown_connections 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 -- 172.21.15.37:0/1287254080 wait complete. 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 Processor -- start 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 -- start start 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e6c100200 0x7f9e6c197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:51.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.411+0000 7f9e73481640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e6c1013f0 0x7f9e6c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e73481640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6c101db0 0x7f9e6c19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e73481640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c10be90 con 0x7f9e6c100200 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e73481640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c10bd10 con 0x7f9e6c101db0 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e73481640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c10c010 con 0x7f9e6c1013f0 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e7121d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e6c100200 0x7f9e6c197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e7121d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e6c100200 0x7f9e6c197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50338/0 (socket says 172.21.15.37:50338) 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e7121d640 1 -- 172.21.15.37:0/664040289 learned_addr learned my addr 172.21.15.37:0/664040289 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e71a1e640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6c101db0 0x7f9e6c19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e70a1c640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e6c1013f0 0x7f9e6c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e7121d640 1 -- 172.21.15.37:0/664040289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e6c1013f0 msgr2=0x7f9e6c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e7121d640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e6c1013f0 0x7f9e6c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e7121d640 1 -- 172.21.15.37:0/664040289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6c101db0 msgr2=0x7f9e6c19eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e7121d640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6c101db0 0x7f9e6c19eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e7121d640 1 -- 172.21.15.37:0/664040289 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c19f1a0 con 0x7f9e6c100200 2024-03-26T06:20:51.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e71a1e640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6c101db0 0x7f9e6c19eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:20:51.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e70a1c640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e6c1013f0 0x7f9e6c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:20:51.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e7121d640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e6c100200 0x7f9e6c197580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800d950 tx=0x7f9e6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:51.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e627fc640 1 -- 172.21.15.37:0/664040289 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68017070 con 0x7f9e6c100200 2024-03-26T06:20:51.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e627fc640 1 -- 172.21.15.37:0/664040289 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e68009c30 con 0x7f9e6c100200 2024-03-26T06:20:51.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e73481640 1 -- 172.21.15.37:0/664040289 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6c19f490 con 0x7f9e6c100200 2024-03-26T06:20:51.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e627fc640 1 -- 172.21.15.37:0/664040289 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6800f600 con 0x7f9e6c100200 2024-03-26T06:20:51.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.412+0000 7f9e73481640 1 -- 172.21.15.37:0/664040289 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c19f990 con 0x7f9e6c100200 2024-03-26T06:20:51.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.413+0000 7f9e73481640 1 -- 172.21.15.37:0/664040289 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6c19fcb0 con 0x7f9e6c100200 2024-03-26T06:20:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.414+0000 7f9e627fc640 1 -- 172.21.15.37:0/664040289 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9e6801a070 con 0x7f9e6c100200 2024-03-26T06:20:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.415+0000 7f9e627fc640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f9e580a2800 0x7f9e580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.415+0000 7f9e627fc640 1 -- 172.21.15.37:0/664040289 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9e680ce1b0 con 0x7f9e6c100200 2024-03-26T06:20:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.415+0000 7f9e70a1c640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f9e580a2800 0x7f9e580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.415+0000 7f9e70a1c640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f9e580a2800 0x7f9e580a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c198a90 tx=0x7f9e5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:51.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.418+0000 7f9e627fc640 1 -- 172.21.15.37:0/664040289 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9e68009de0 con 0x7f9e6c100200 2024-03-26T06:20:51.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:51 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3070416959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:51.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:51 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2457184455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:51.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:51 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2077381673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:20:51.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:51 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:51.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:51 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:51.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:51 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:51.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:51 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:51.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:51.630+0000 7f9e73481640 1 -- 172.21.15.37:0/664040289 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9e6c19fe80 con 0x7f9e6c100200 2024-03-26T06:20:51.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:51 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3070416959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:51.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:51 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2457184455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:51.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:51 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2077381673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:20:51.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:51 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:51.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:51 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:51.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:51 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:51.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:51 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:51.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:51 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3070416959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:20:51.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:51 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2457184455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:51.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:51 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2077381673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:20:51.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:51 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:51.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:51 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:51.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:51 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:51.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:51 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.260+0000 7f9e627fc640 1 -- 172.21.15.37:0/664040289 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9e6809b650 con 0x7f9e6c100200 2024-03-26T06:20:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-03-26T06:20:52.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 -- 172.21.15.37:0/664040289 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f9e580a2800 msgr2=0x7f9e580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:52.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f9e580a2800 0x7f9e580a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c198a90 tx=0x7f9e5c005f90 comp rx=0 tx=0).stop 2024-03-26T06:20:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 -- 172.21.15.37:0/664040289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e6c100200 msgr2=0x7f9e6c197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e6c100200 0x7f9e6c197580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800d950 tx=0x7f9e6800de10 comp rx=0 tx=0).stop 2024-03-26T06:20:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 -- 172.21.15.37:0/664040289 shutdown_connections 2024-03-26T06:20:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f9e580a2800 0x7f9e580a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e6c101db0 0x7f9e6c19eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e6c1013f0 0x7f9e6c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 --2- 172.21.15.37:0/664040289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e6c100200 0x7f9e6c197580 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 -- 172.21.15.37:0/664040289 >> 172.21.15.37:0/664040289 conn(0x7f9e6c0fb790 msgr2=0x7f9e6c0698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 -- 172.21.15.37:0/664040289 shutdown_connections 2024-03-26T06:20:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.262+0000 7f9e73481640 1 -- 172.21.15.37:0/664040289 wait complete. 2024-03-26T06:20:52.278 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-26T06:20:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.389+0000 7f18b6f69640 1 -- 172.21.15.37:0/1255645271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b01001f0 msgr2=0x7f18b0100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.389+0000 7f18b6f69640 1 --2- 172.21.15.37:0/1255645271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b01001f0 0x7f18b0100610 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009ca0 tx=0x7f18a802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:20:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.389+0000 7f18b6f69640 1 -- 172.21.15.37:0/1255645271 shutdown_connections 2024-03-26T06:20:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.389+0000 7f18b6f69640 1 --2- 172.21.15.37:0/1255645271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b0101da0 0x7f18b0108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.389+0000 7f18b6f69640 1 --2- 172.21.15.37:0/1255645271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b01013e0 0x7f18b0101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.389+0000 7f18b6f69640 1 --2- 172.21.15.37:0/1255645271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b01001f0 0x7f18b0100610 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.389+0000 7f18b6f69640 1 -- 172.21.15.37:0/1255645271 >> 172.21.15.37:0/1255645271 conn(0x7f18b00fb760 msgr2=0x7f18b00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.389+0000 7f18b6f69640 1 -- 172.21.15.37:0/1255645271 shutdown_connections 2024-03-26T06:20:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.389+0000 7f18b6f69640 1 -- 172.21.15.37:0/1255645271 wait complete. 2024-03-26T06:20:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18b6f69640 1 Processor -- start 2024-03-26T06:20:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18b6f69640 1 -- start start 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18b6f69640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b01001f0 0x7f18b0196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18b6f69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b01013e0 0x7f18b0196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18b6f69640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b0101da0 0x7f18b019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18b6f69640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b010be80 con 0x7f18b01013e0 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18b6f69640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b010bd00 con 0x7f18b01001f0 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18b6f69640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b010c000 con 0x7f18b0101da0 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b01013e0 0x7f18b0196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b01013e0 0x7f18b0196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50346/0 (socket says 172.21.15.37:50346) 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18affff640 1 -- 172.21.15.37:0/448428944 learned_addr learned my addr 172.21.15.37:0/448428944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18b5506640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b0101da0 0x7f18b019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18b4d05640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b01001f0 0x7f18b0196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18affff640 1 -- 172.21.15.37:0/448428944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b0101da0 msgr2=0x7f18b019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18affff640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b0101da0 0x7f18b019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18affff640 1 -- 172.21.15.37:0/448428944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b01001f0 msgr2=0x7f18b0196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18affff640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b01001f0 0x7f18b0196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18affff640 1 -- 172.21.15.37:0/448428944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b019e120 con 0x7f18b01013e0 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.390+0000 7f18b5506640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b0101da0 0x7f18b019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.391+0000 7f18affff640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b01013e0 0x7f18b0196a40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f18a400d950 tx=0x7f18a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.391+0000 7f18adffb640 1 -- 172.21.15.37:0/448428944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4017030 con 0x7f18b01013e0 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.391+0000 7f18adffb640 1 -- 172.21.15.37:0/448428944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18a4004020 con 0x7f18b01013e0 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.391+0000 7f18b6f69640 1 -- 172.21.15.37:0/448428944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b019e410 con 0x7f18b01013e0 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.391+0000 7f18adffb640 1 -- 172.21.15.37:0/448428944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40047a0 con 0x7f18b01013e0 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.391+0000 7f18b6f69640 1 -- 172.21.15.37:0/448428944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b019e9b0 con 0x7f18b01013e0 2024-03-26T06:20:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.392+0000 7f18b6f69640 1 -- 172.21.15.37:0/448428944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b01075e0 con 0x7f18b01013e0 2024-03-26T06:20:52.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.393+0000 7f18adffb640 1 -- 172.21.15.37:0/448428944 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f18a40041d0 con 0x7f18b01013e0 2024-03-26T06:20:52.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.393+0000 7f18adffb640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18980a2880 0x7f18980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:52.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.393+0000 7f18b4d05640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18980a2880 0x7f18980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:52.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.394+0000 7f18adffb640 1 -- 172.21.15.37:0/448428944 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f18a40ce270 con 0x7f18b01013e0 2024-03-26T06:20:52.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.394+0000 7f18b4d05640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18980a2880 0x7f18980a4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f18a802c700 tx=0x7f18a8009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.398+0000 7f18adffb640 1 -- 172.21.15.37:0/448428944 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f18a409b5a0 con 0x7f18b01013e0 2024-03-26T06:20:52.483 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:52 smithi066 ceph-mon[36504]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:52.483 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.483 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.483 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:52.483 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:52.483 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.483 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:52 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3640019961' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T06:20:52.483 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:52.483 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:52 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/664040289' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T06:20:52.484 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:52 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.531 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:52 smithi149 ceph-mon[36681]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:52.531 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.531 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.531 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:52.531 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:52.531 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.531 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:52 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3640019961' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T06:20:52.531 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:52.531 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:52 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/664040289' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T06:20:52.531 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:52 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:52.583+0000 7f18b6f69640 1 -- 172.21.15.37:0/448428944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f18b0061ef0 con 0x7f18b01013e0 2024-03-26T06:20:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:52 smithi037 ceph-mon[31974]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:52.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:52.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:52 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3640019961' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T06:20:52.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:20:52.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:52 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/664040289' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T06:20:52.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:52 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.266+0000 7f18adffb640 1 -- 172.21.15.37:0/448428944 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7f18a409aea0 con 0x7f18b01013e0 2024-03-26T06:20:53.267 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 -- 172.21.15.37:0/448428944 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18980a2880 msgr2=0x7f18980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18980a2880 0x7f18980a4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f18a802c700 tx=0x7f18a8009360 comp rx=0 tx=0).stop 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 -- 172.21.15.37:0/448428944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b01013e0 msgr2=0x7f18b0196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b01013e0 0x7f18b0196a40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f18a400d950 tx=0x7f18a400de10 comp rx=0 tx=0).stop 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 -- 172.21.15.37:0/448428944 shutdown_connections 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18980a2880 0x7f18980a4d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b0101da0 0x7f18b019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18b01013e0 0x7f18b0196a40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 --2- 172.21.15.37:0/448428944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18b01001f0 0x7f18b0196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 -- 172.21.15.37:0/448428944 >> 172.21.15.37:0/448428944 conn(0x7f18b00fb760 msgr2=0x7f18b01047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 -- 172.21.15.37:0/448428944 shutdown_connections 2024-03-26T06:20:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.270+0000 7f18b6f69640 1 -- 172.21.15.37:0/448428944 wait complete. 2024-03-26T06:20:53.292 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-26T06:20:53.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 -- 172.21.15.37:0/1340318643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82700ffd20 msgr2=0x7f82701098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:53.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 --2- 172.21.15.37:0/1340318643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82700ffd20 0x7f82701098d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8268009ca0 tx=0x7f826802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:20:53.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 -- 172.21.15.37:0/1340318643 shutdown_connections 2024-03-26T06:20:53.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 --2- 172.21.15.37:0/1340318643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82700ffd20 0x7f82701098d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:53.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 --2- 172.21.15.37:0/1340318643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82700ff360 0x7f82700ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:53.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 --2- 172.21.15.37:0/1340318643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82701028f0 0x7f8270102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:53.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 -- 172.21.15.37:0/1340318643 >> 172.21.15.37:0/1340318643 conn(0x7f82700faee0 msgr2=0x7f82700fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 -- 172.21.15.37:0/1340318643 shutdown_connections 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 -- 172.21.15.37:0/1340318643 wait complete. 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 Processor -- start 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 -- start start 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82700ff360 0x7f8270197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82700ffd20 0x7f8270197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82701028f0 0x7f827019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827010cf90 con 0x7f82701028f0 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827010ce10 con 0x7f82700ff360 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.397+0000 7f8278140640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827010d110 con 0x7f82700ffd20 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f82766dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82701028f0 0x7f827019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f82766dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82701028f0 0x7f827019eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50364/0 (socket says 172.21.15.37:50364) 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f8275edc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82700ff360 0x7f8270197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f82766dd640 1 -- 172.21.15.37:0/2048198655 learned_addr learned my addr 172.21.15.37:0/2048198655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f82756db640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82700ffd20 0x7f8270197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f82766dd640 1 -- 172.21.15.37:0/2048198655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82700ffd20 msgr2=0x7f8270197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f82766dd640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82700ffd20 0x7f8270197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f82766dd640 1 -- 172.21.15.37:0/2048198655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82700ff360 msgr2=0x7f8270197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f82766dd640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82700ff360 0x7f8270197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f82766dd640 1 -- 172.21.15.37:0/2048198655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827019f1b0 con 0x7f82701028f0 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f82756db640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82700ffd20 0x7f8270197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f82766dd640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82701028f0 0x7f827019eb40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f826802c700 tx=0x7f82680346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f8266ffd640 1 -- 172.21.15.37:0/2048198655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268041020 con 0x7f82701028f0 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f8266ffd640 1 -- 172.21.15.37:0/2048198655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8268032890 con 0x7f82701028f0 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f8278140640 1 -- 172.21.15.37:0/2048198655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827019f440 con 0x7f82701028f0 2024-03-26T06:20:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f8266ffd640 1 -- 172.21.15.37:0/2048198655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268039040 con 0x7f82701028f0 2024-03-26T06:20:53.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.398+0000 7f8278140640 1 -- 172.21.15.37:0/2048198655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827019f980 con 0x7f82701028f0 2024-03-26T06:20:53.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.399+0000 7f8278140640 1 -- 172.21.15.37:0/2048198655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82701087b0 con 0x7f82701028f0 2024-03-26T06:20:53.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.400+0000 7f8266ffd640 1 -- 172.21.15.37:0/2048198655 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8268044020 con 0x7f82701028f0 2024-03-26T06:20:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.401+0000 7f8266ffd640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f825c0a2880 0x7f825c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.401+0000 7f8275edc640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f825c0a2880 0x7f825c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.401+0000 7f8266ffd640 1 -- 172.21.15.37:0/2048198655 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f82680f0d40 con 0x7f82701028f0 2024-03-26T06:20:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.401+0000 7f8275edc640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f825c0a2880 0x7f825c0a4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f82700695e0 tx=0x7f826c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:53.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.405+0000 7f8266ffd640 1 -- 172.21.15.37:0/2048198655 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f826804b020 con 0x7f82701028f0 2024-03-26T06:20:53.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/664040289' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T06:20:53.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T06:20:53.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/448428944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T06:20:53.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:53.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:53.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:53.630+0000 7f8278140640 1 -- 172.21.15.37:0/2048198655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f827002ce70 con 0x7f82701028f0 2024-03-26T06:20:53.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/664040289' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T06:20:53.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T06:20:53.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/448428944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T06:20:53.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:53.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:53.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/664040289' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T06:20:53.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T06:20:53.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/448428944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T06:20:53.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:53.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:20:53.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:20:53.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:20:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.268+0000 7f8266ffd640 1 -- 172.21.15.37:0/2048198655 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f82680bdef0 con 0x7f82701028f0 2024-03-26T06:20:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-03-26T06:20:54.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.271+0000 7f8278140640 1 -- 172.21.15.37:0/2048198655 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f825c0a2880 msgr2=0x7f825c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.271+0000 7f8278140640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f825c0a2880 0x7f825c0a4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f82700695e0 tx=0x7f826c00b390 comp rx=0 tx=0).stop 2024-03-26T06:20:54.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.271+0000 7f8278140640 1 -- 172.21.15.37:0/2048198655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82701028f0 msgr2=0x7f827019eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.271+0000 7f8278140640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82701028f0 0x7f827019eb40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f826802c700 tx=0x7f82680346f0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.272+0000 7f8278140640 1 -- 172.21.15.37:0/2048198655 shutdown_connections 2024-03-26T06:20:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.272+0000 7f8278140640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f825c0a2880 0x7f825c0a4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.272+0000 7f8278140640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82701028f0 0x7f827019eb40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.272+0000 7f8278140640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82700ffd20 0x7f8270197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.272+0000 7f8278140640 1 --2- 172.21.15.37:0/2048198655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82700ff360 0x7f8270197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.272+0000 7f8278140640 1 -- 172.21.15.37:0/2048198655 >> 172.21.15.37:0/2048198655 conn(0x7f82700faee0 msgr2=0x7f8270105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.272+0000 7f8278140640 1 -- 172.21.15.37:0/2048198655 shutdown_connections 2024-03-26T06:20:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.272+0000 7f8278140640 1 -- 172.21.15.37:0/2048198655 wait complete. 2024-03-26T06:20:54.296 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:20:54.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.405+0000 7f0ab6659640 1 -- 172.21.15.37:0/2627959029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0102ec0 msgr2=0x7f0ab0109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.405+0000 7f0ab6659640 1 --2- 172.21.15.37:0/2627959029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0102ec0 0x7f0ab0109750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009ca0 tx=0x7f0aa802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.405+0000 7f0ab6659640 1 -- 172.21.15.37:0/2627959029 shutdown_connections 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.405+0000 7f0ab6659640 1 --2- 172.21.15.37:0/2627959029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0102ec0 0x7f0ab0109750 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.405+0000 7f0ab6659640 1 --2- 172.21.15.37:0/2627959029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ab0102500 0x7f0ab0102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.405+0000 7f0ab6659640 1 --2- 172.21.15.37:0/2627959029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ab0101310 0x7f0ab0101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.405+0000 7f0ab6659640 1 -- 172.21.15.37:0/2627959029 >> 172.21.15.37:0/2627959029 conn(0x7f0ab00fc8a0 msgr2=0x7f0ab00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.405+0000 7f0ab6659640 1 -- 172.21.15.37:0/2627959029 shutdown_connections 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.405+0000 7f0ab6659640 1 -- 172.21.15.37:0/2627959029 wait complete. 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0ab6659640 1 Processor -- start 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0ab6659640 1 -- start start 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0ab6659640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ab0101310 0x7f0ab00750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0ab6659640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0102500 0x7f0ab00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0ab6659640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ab0102ec0 0x7f0ab0073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0ab6659640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab010cfe0 con 0x7f0ab0102500 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0ab6659640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab010ce60 con 0x7f0ab0101310 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0ab6659640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab010d160 con 0x7f0ab0102ec0 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0aaf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0102500 0x7f0ab00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0aaf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0102500 0x7f0ab00736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50386/0 (socket says 172.21.15.37:50386) 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0aaf7fe640 1 -- 172.21.15.37:0/1097864789 learned_addr learned my addr 172.21.15.37:0/1097864789 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.406+0000 7f0ab4bf6640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ab0102ec0 0x7f0ab0073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0aaffff640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ab0101310 0x7f0ab00750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0aaf7fe640 1 -- 172.21.15.37:0/1097864789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ab0102ec0 msgr2=0x7f0ab0073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0aaf7fe640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ab0102ec0 0x7f0ab0073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0aaf7fe640 1 -- 172.21.15.37:0/1097864789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ab0101310 msgr2=0x7f0ab00750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0aaf7fe640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ab0101310 0x7f0ab00750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0aaf7fe640 1 -- 172.21.15.37:0/1097864789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab00744b0 con 0x7f0ab0102500 2024-03-26T06:20:54.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0aaf7fe640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0102500 0x7f0ab00736e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400b4a0 tx=0x7f0aa400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:54.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0aad7fa640 1 -- 172.21.15.37:0/1097864789 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4014070 con 0x7f0ab0102500 2024-03-26T06:20:54.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0aad7fa640 1 -- 172.21.15.37:0/1097864789 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0aa4010af0 con 0x7f0ab0102500 2024-03-26T06:20:54.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0ab6659640 1 -- 172.21.15.37:0/1097864789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab01a1710 con 0x7f0ab0102500 2024-03-26T06:20:54.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0aad7fa640 1 -- 172.21.15.37:0/1097864789 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4017d10 con 0x7f0ab0102500 2024-03-26T06:20:54.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.407+0000 7f0ab6659640 1 -- 172.21.15.37:0/1097864789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab01a1c60 con 0x7f0ab0102500 2024-03-26T06:20:54.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.408+0000 7f0ab6659640 1 -- 172.21.15.37:0/1097864789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab0108630 con 0x7f0ab0102500 2024-03-26T06:20:54.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.409+0000 7f0aad7fa640 1 -- 172.21.15.37:0/1097864789 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0aa4017070 con 0x7f0ab0102500 2024-03-26T06:20:54.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.409+0000 7f0aad7fa640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a980a2880 0x7f0a980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:54.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.410+0000 7f0aaffff640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a980a2880 0x7f0a980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:54.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.410+0000 7f0aad7fa640 1 -- 172.21.15.37:0/1097864789 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0aa40cdd40 con 0x7f0ab0102500 2024-03-26T06:20:54.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.410+0000 7f0aaffff640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a980a2880 0x7f0a980a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0005fd0 tx=0x7f0aa0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:54.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.414+0000 7f0aad7fa640 1 -- 172.21.15.37:0/1097864789 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0aa409af30 con 0x7f0ab0102500 2024-03-26T06:20:54.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:54 smithi066 ceph-mon[36504]: pgmap v285: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:54.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:54 smithi066 ceph-mon[36504]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T06:20:54.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:54 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/448428944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T06:20:54.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:54 smithi066 ceph-mon[36504]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T06:20:54.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:54 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2048198655' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T06:20:54.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:54 smithi066 ceph-mon[36504]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T06:20:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.599+0000 7f0ab6659640 1 -- 172.21.15.37:0/1097864789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ab002d100 con 0x7f0ab0102500 2024-03-26T06:20:54.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.601+0000 7f0aad7fa640 1 -- 172.21.15.37:0/1097864789 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22158 (secure 0 0 0) 0x7f0aa409a830 con 0x7f0ab0102500 2024-03-26T06:20:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:54.602 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":84,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:20:53.631992+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:20:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 -- 172.21.15.37:0/1097864789 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a980a2880 msgr2=0x7f0a980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a980a2880 0x7f0a980a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0005fd0 tx=0x7f0aa0005850 comp rx=0 tx=0).stop 2024-03-26T06:20:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 -- 172.21.15.37:0/1097864789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0102500 msgr2=0x7f0ab00736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0102500 0x7f0ab00736e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400b4a0 tx=0x7f0aa400b960 comp rx=0 tx=0).stop 2024-03-26T06:20:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 -- 172.21.15.37:0/1097864789 shutdown_connections 2024-03-26T06:20:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f0a980a2880 0x7f0a980a4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ab0102ec0 0x7f0ab0073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab0102500 0x7f0ab00736e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 --2- 172.21.15.37:0/1097864789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ab0101310 0x7f0ab00750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 -- 172.21.15.37:0/1097864789 >> 172.21.15.37:0/1097864789 conn(0x7f0ab00fc8a0 msgr2=0x7f0ab0105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 -- 172.21.15.37:0/1097864789 shutdown_connections 2024-03-26T06:20:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.603+0000 7f0ab6659640 1 -- 172.21.15.37:0/1097864789 wait complete. 2024-03-26T06:20:54.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:54 smithi037 ceph-mon[31974]: pgmap v285: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:54.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:54 smithi037 ceph-mon[31974]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T06:20:54.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:54 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/448428944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T06:20:54.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:54 smithi037 ceph-mon[31974]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T06:20:54.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:54 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2048198655' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T06:20:54.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:54 smithi037 ceph-mon[31974]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T06:20:54.615 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:20:54.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:54 smithi149 ceph-mon[36681]: pgmap v285: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:54.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:54 smithi149 ceph-mon[36681]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T06:20:54.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:54 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/448428944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T06:20:54.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:54 smithi149 ceph-mon[36681]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T06:20:54.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:54 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2048198655' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T06:20:54.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:54 smithi149 ceph-mon[36681]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T06:20:54.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.717+0000 7ff2f14b2640 1 -- 172.21.15.37:0/1955062706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2ec102920 msgr2=0x7ff2ec102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.717+0000 7ff2f14b2640 1 --2- 172.21.15.37:0/1955062706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2ec102920 0x7ff2ec102d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009900 tx=0x7ff2e402bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:54.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.717+0000 7ff2f14b2640 1 -- 172.21.15.37:0/1955062706 shutdown_connections 2024-03-26T06:20:54.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.717+0000 7ff2f14b2640 1 --2- 172.21.15.37:0/1955062706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec0ffd50 0x7ff2ec109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.717+0000 7ff2f14b2640 1 --2- 172.21.15.37:0/1955062706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2ec0ff390 0x7ff2ec0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.717+0000 7ff2f14b2640 1 --2- 172.21.15.37:0/1955062706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2ec102920 0x7ff2ec102d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.717+0000 7ff2f14b2640 1 -- 172.21.15.37:0/1955062706 >> 172.21.15.37:0/1955062706 conn(0x7ff2ec0faf10 msgr2=0x7ff2ec0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.717+0000 7ff2f14b2640 1 -- 172.21.15.37:0/1955062706 shutdown_connections 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.717+0000 7ff2f14b2640 1 -- 172.21.15.37:0/1955062706 wait complete. 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2f14b2640 1 Processor -- start 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2f14b2640 1 -- start start 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2f14b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2ec0ff390 0x7ff2ec102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2f14b2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2ec0ffd50 0x7ff2ec100660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2f14b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec102920 0x7ff2ec100ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2f14b2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec10d000 con 0x7ff2ec0ff390 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2f14b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec10ce80 con 0x7ff2ec102920 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2f14b2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec10d180 con 0x7ff2ec0ffd50 2024-03-26T06:20:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2eaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2ec0ff390 0x7ff2ec102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2eaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2ec0ff390 0x7ff2ec102020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50400/0 (socket says 172.21.15.37:50400) 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2eaffd640 1 -- 172.21.15.37:0/740863800 learned_addr learned my addr 172.21.15.37:0/740863800 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2eb7fe640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec102920 0x7ff2ec100ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2ea7fc640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2ec0ffd50 0x7ff2ec100660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2eaffd640 1 -- 172.21.15.37:0/740863800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2ec0ffd50 msgr2=0x7ff2ec100660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2eaffd640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2ec0ffd50 0x7ff2ec100660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2eaffd640 1 -- 172.21.15.37:0/740863800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec102920 msgr2=0x7ff2ec100ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2eaffd640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec102920 0x7ff2ec100ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2eaffd640 1 -- 172.21.15.37:0/740863800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec101430 con 0x7ff2ec0ff390 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.718+0000 7ff2ea7fc640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2ec0ffd50 0x7ff2ec100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.719+0000 7ff2eaffd640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2ec0ff390 0x7ff2ec102020 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005ab0 tx=0x7ff2e4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.719+0000 7ff2d3fff640 1 -- 172.21.15.37:0/740863800 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4041070 con 0x7ff2ec0ff390 2024-03-26T06:20:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.719+0000 7ff2d3fff640 1 -- 172.21.15.37:0/740863800 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2e40054e0 con 0x7ff2ec0ff390 2024-03-26T06:20:54.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.719+0000 7ff2f14b2640 1 -- 172.21.15.37:0/740863800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ec1a16e0 con 0x7ff2ec0ff390 2024-03-26T06:20:54.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.719+0000 7ff2d3fff640 1 -- 172.21.15.37:0/740863800 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4034440 con 0x7ff2ec0ff390 2024-03-26T06:20:54.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.719+0000 7ff2f14b2640 1 -- 172.21.15.37:0/740863800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ec1a1c80 con 0x7ff2ec0ff390 2024-03-26T06:20:54.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.720+0000 7ff2f14b2640 1 -- 172.21.15.37:0/740863800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ec108860 con 0x7ff2ec0ff390 2024-03-26T06:20:54.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.721+0000 7ff2d3fff640 1 -- 172.21.15.37:0/740863800 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff2e4044070 con 0x7ff2ec0ff390 2024-03-26T06:20:54.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.722+0000 7ff2d3fff640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2d40a2880 0x7ff2d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:54.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.722+0000 7ff2ea7fc640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2d40a2880 0x7ff2d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:54.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.722+0000 7ff2d3fff640 1 -- 172.21.15.37:0/740863800 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff2e40f0570 con 0x7ff2ec0ff390 2024-03-26T06:20:54.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.722+0000 7ff2ea7fc640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2d40a2880 0x7ff2d40a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec101d90 tx=0x7ff2e0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:54.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.726+0000 7ff2d3fff640 1 -- 172.21.15.37:0/740863800 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff2e40bd6c0 con 0x7ff2ec0ff390 2024-03-26T06:20:54.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.912+0000 7ff2f14b2640 1 -- 172.21.15.37:0/740863800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2ec02d090 con 0x7ff2ec0ff390 2024-03-26T06:20:54.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.913+0000 7ff2d3fff640 1 -- 172.21.15.37:0/740863800 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22158 (secure 0 0 0) 0x7ff2e40bcfc0 con 0x7ff2ec0ff390 2024-03-26T06:20:54.914 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:20:54.915 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":84,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:20:53.631992+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:20:54.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.915+0000 7ff2f14b2640 1 -- 172.21.15.37:0/740863800 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2d40a2880 msgr2=0x7ff2d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.915+0000 7ff2f14b2640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2d40a2880 0x7ff2d40a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec101d90 tx=0x7ff2e0005d30 comp rx=0 tx=0).stop 2024-03-26T06:20:54.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.915+0000 7ff2f14b2640 1 -- 172.21.15.37:0/740863800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2ec0ff390 msgr2=0x7ff2ec102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:54.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.915+0000 7ff2f14b2640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2ec0ff390 0x7ff2ec102020 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005ab0 tx=0x7ff2e4005dc0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.916+0000 7ff2f14b2640 1 -- 172.21.15.37:0/740863800 shutdown_connections 2024-03-26T06:20:54.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.916+0000 7ff2f14b2640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2d40a2880 0x7ff2d40a4d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.916+0000 7ff2f14b2640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec102920 0x7ff2ec100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.916+0000 7ff2f14b2640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2ec0ffd50 0x7ff2ec100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.916+0000 7ff2f14b2640 1 --2- 172.21.15.37:0/740863800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2ec0ff390 0x7ff2ec102020 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:54.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.916+0000 7ff2f14b2640 1 -- 172.21.15.37:0/740863800 >> 172.21.15.37:0/740863800 conn(0x7ff2ec0faf10 msgr2=0x7ff2ec105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:54.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.916+0000 7ff2f14b2640 1 -- 172.21.15.37:0/740863800 shutdown_connections 2024-03-26T06:20:54.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:54.916+0000 7ff2f14b2640 1 -- 172.21.15.37:0/740863800 wait complete. 2024-03-26T06:20:54.928 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-26T06:20:55.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.032+0000 7f740ee68640 1 -- 172.21.15.37:0/738989859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74081001f0 msgr2=0x7f7408100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:55.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.032+0000 7f740ee68640 1 --2- 172.21.15.37:0/738989859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74081001f0 0x7f7408100610 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009900 tx=0x7f73fc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:55.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.032+0000 7f740ee68640 1 -- 172.21.15.37:0/738989859 shutdown_connections 2024-03-26T06:20:55.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.032+0000 7f740ee68640 1 --2- 172.21.15.37:0/738989859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7408101da0 0x7f7408108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.032+0000 7f740ee68640 1 --2- 172.21.15.37:0/738989859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74081013e0 0x7f7408101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.032+0000 7f740ee68640 1 --2- 172.21.15.37:0/738989859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74081001f0 0x7f7408100610 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.032+0000 7f740ee68640 1 -- 172.21.15.37:0/738989859 >> 172.21.15.37:0/738989859 conn(0x7f74080fb780 msgr2=0x7f74080fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:55.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.032+0000 7f740ee68640 1 -- 172.21.15.37:0/738989859 shutdown_connections 2024-03-26T06:20:55.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.032+0000 7f740ee68640 1 -- 172.21.15.37:0/738989859 wait complete. 2024-03-26T06:20:55.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.032+0000 7f740ee68640 1 Processor -- start 2024-03-26T06:20:55.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f740ee68640 1 -- start start 2024-03-26T06:20:55.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f740ee68640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74081001f0 0x7f7408068fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:55.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f740ee68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74081013e0 0x7f740819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:55.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f740ee68640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7408101da0 0x7f740819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:55.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f740ee68640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740810be50 con 0x7f74081013e0 2024-03-26T06:20:55.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f740ee68640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740810bcd0 con 0x7f74081001f0 2024-03-26T06:20:55.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f740ee68640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740810bfd0 con 0x7f7408101da0 2024-03-26T06:20:55.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7407fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74081013e0 0x7f740819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:55.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7407fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74081013e0 0x7f740819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50418/0 (socket says 172.21.15.37:50418) 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7407fff640 1 -- 172.21.15.37:0/2522170889 learned_addr learned my addr 172.21.15.37:0/2522170889 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7407fff640 1 -- 172.21.15.37:0/2522170889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7408101da0 msgr2=0x7f740819ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7407fff640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7408101da0 0x7f740819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7407fff640 1 -- 172.21.15.37:0/2522170889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74081001f0 msgr2=0x7f7408068fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7407fff640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74081001f0 0x7f7408068fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7407fff640 1 -- 172.21.15.37:0/2522170889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740819f310 con 0x7f74081013e0 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7407fff640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74081013e0 0x7f740819c3e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f73f400e9f0 tx=0x7f73f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7405ffb640 1 -- 172.21.15.37:0/2522170889 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f400cd80 con 0x7f74081013e0 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f740ee68640 1 -- 172.21.15.37:0/2522170889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740819f5a0 con 0x7f74081013e0 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7405ffb640 1 -- 172.21.15.37:0/2522170889 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73f4010910 con 0x7f74081013e0 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.033+0000 7f7405ffb640 1 -- 172.21.15.37:0/2522170889 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f40188f0 con 0x7f74081013e0 2024-03-26T06:20:55.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.034+0000 7f740ee68640 1 -- 172.21.15.37:0/2522170889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740819faf0 con 0x7f74081013e0 2024-03-26T06:20:55.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.035+0000 7f740ee68640 1 -- 172.21.15.37:0/2522170889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7408107630 con 0x7f74081013e0 2024-03-26T06:20:55.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.036+0000 7f7405ffb640 1 -- 172.21.15.37:0/2522170889 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f73f4010ac0 con 0x7f74081013e0 2024-03-26T06:20:55.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.036+0000 7f7405ffb640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73f80a2800 0x7f73f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:55.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.036+0000 7f740cc04640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73f80a2800 0x7f73f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:55.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.036+0000 7f7405ffb640 1 -- 172.21.15.37:0/2522170889 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f73f40cee20 con 0x7f74081013e0 2024-03-26T06:20:55.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.037+0000 7f740cc04640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73f80a2800 0x7f73f80a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f73fc036040 tx=0x7f73fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:55.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.041+0000 7f7405ffb640 1 -- 172.21.15.37:0/2522170889 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f73f409c010 con 0x7f74081013e0 2024-03-26T06:20:55.097 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:20:54 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b[67275]: 2024-03-26T06:20:54.744+0000 7f1a96d60700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-26T06:20:55.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.263+0000 7f740ee68640 1 -- 172.21.15.37:0/2522170889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f740802ce70 con 0x7f74081013e0 2024-03-26T06:20:55.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.584+0000 7f7405ffb640 1 -- 172.21.15.37:0/2522170889 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f73f409b910 con 0x7f74081013e0 2024-03-26T06:20:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 -- 172.21.15.37:0/2522170889 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73f80a2800 msgr2=0x7f73f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73f80a2800 0x7f73f80a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f73fc036040 tx=0x7f73fc000bc0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 -- 172.21.15.37:0/2522170889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74081013e0 msgr2=0x7f740819c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74081013e0 0x7f740819c3e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f73f400e9f0 tx=0x7f73f400eeb0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 -- 172.21.15.37:0/2522170889 shutdown_connections 2024-03-26T06:20:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73f80a2800 0x7f73f80a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7408101da0 0x7f740819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74081013e0 0x7f740819c3e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 --2- 172.21.15.37:0/2522170889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74081001f0 0x7f7408068fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 -- 172.21.15.37:0/2522170889 >> 172.21.15.37:0/2522170889 conn(0x7f74080fb780 msgr2=0x7f7408104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 -- 172.21.15.37:0/2522170889 shutdown_connections 2024-03-26T06:20:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.586+0000 7f740ee68640 1 -- 172.21.15.37:0/2522170889 wait complete. 2024-03-26T06:20:55.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:55 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2048198655' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T06:20:55.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:55 smithi066 ceph-mon[36504]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T06:20:55.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:55 smithi066 ceph-mon[36504]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T06:20:55.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:55 smithi066 ceph-mon[36504]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T06:20:55.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:55 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] up:active 2024-03-26T06:20:55.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:55 smithi066 ceph-mon[36504]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T06:20:55.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:55 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1097864789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:55.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:55 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/740863800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:55.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:55 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2522170889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T06:20:55.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:55 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2048198655' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T06:20:55.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:55 smithi037 ceph-mon[31974]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T06:20:55.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:55 smithi037 ceph-mon[31974]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T06:20:55.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:55 smithi037 ceph-mon[31974]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T06:20:55.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:55 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] up:active 2024-03-26T06:20:55.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:55 smithi037 ceph-mon[31974]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T06:20:55.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:55 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1097864789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:55.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:55 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/740863800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:55.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:55 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2522170889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T06:20:55.600 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-26T06:20:55.600 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-26T06:20:55.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:55 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2048198655' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T06:20:55.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:55 smithi149 ceph-mon[36681]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T06:20:55.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:55 smithi149 ceph-mon[36681]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T06:20:55.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:55 smithi149 ceph-mon[36681]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T06:20:55.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:55 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] up:active 2024-03-26T06:20:55.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:55 smithi149 ceph-mon[36681]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T06:20:55.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:55 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1097864789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:55.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:55 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/740863800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:20:55.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:55 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2522170889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T06:20:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.704+0000 7f3d14116640 1 -- 172.21.15.37:0/2063178068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d0c101da0 msgr2=0x7f3d0c108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.704+0000 7f3d14116640 1 --2- 172.21.15.37:0/2063178068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d0c101da0 0x7f3d0c108630 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009900 tx=0x7f3cfc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.704+0000 7f3d14116640 1 -- 172.21.15.37:0/2063178068 shutdown_connections 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.704+0000 7f3d14116640 1 --2- 172.21.15.37:0/2063178068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d0c101da0 0x7f3d0c108630 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.704+0000 7f3d14116640 1 --2- 172.21.15.37:0/2063178068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c1013e0 0x7f3d0c101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.704+0000 7f3d14116640 1 --2- 172.21.15.37:0/2063178068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d0c1001f0 0x7f3d0c100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.704+0000 7f3d14116640 1 -- 172.21.15.37:0/2063178068 >> 172.21.15.37:0/2063178068 conn(0x7f3d0c0fb780 msgr2=0x7f3d0c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.704+0000 7f3d14116640 1 -- 172.21.15.37:0/2063178068 shutdown_connections 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.704+0000 7f3d14116640 1 -- 172.21.15.37:0/2063178068 wait complete. 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d14116640 1 Processor -- start 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d14116640 1 -- start start 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d14116640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d0c1001f0 0x7f3d0c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d14116640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c1013e0 0x7f3d0c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d14116640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d0c101da0 0x7f3d0c19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d14116640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c10be80 con 0x7f3d0c101da0 2024-03-26T06:20:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d14116640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c10bd00 con 0x7f3d0c1001f0 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d14116640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c10c000 con 0x7f3d0c1013e0 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d126b3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d0c101da0 0x7f3d0c19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d126b3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d0c101da0 0x7f3d0c19da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50436/0 (socket says 172.21.15.37:50436) 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d126b3640 1 -- 172.21.15.37:0/3328707481 learned_addr learned my addr 172.21.15.37:0/3328707481 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d11eb2640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d0c1001f0 0x7f3d0c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d116b1640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c1013e0 0x7f3d0c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d126b3640 1 -- 172.21.15.37:0/3328707481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c1013e0 msgr2=0x7f3d0c196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d126b3640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c1013e0 0x7f3d0c196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d126b3640 1 -- 172.21.15.37:0/3328707481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d0c1001f0 msgr2=0x7f3d0c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d126b3640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d0c1001f0 0x7f3d0c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.705+0000 7f3d126b3640 1 -- 172.21.15.37:0/3328707481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c19e0f0 con 0x7f3d0c101da0 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.706+0000 7f3d126b3640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d0c101da0 0x7f3d0c19da80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc02ce70 tx=0x7f3cfc005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.706+0000 7f3d0affd640 1 -- 172.21.15.37:0/3328707481 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc041070 con 0x7f3d0c101da0 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.706+0000 7f3d14116640 1 -- 172.21.15.37:0/3328707481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0c19e380 con 0x7f3d0c101da0 2024-03-26T06:20:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.706+0000 7f3d0affd640 1 -- 172.21.15.37:0/3328707481 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cfc0054e0 con 0x7f3d0c101da0 2024-03-26T06:20:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.706+0000 7f3d0affd640 1 -- 172.21.15.37:0/3328707481 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0344a0 con 0x7f3d0c101da0 2024-03-26T06:20:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.706+0000 7f3d14116640 1 -- 172.21.15.37:0/3328707481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0c19e870 con 0x7f3d0c101da0 2024-03-26T06:20:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.707+0000 7f3d14116640 1 -- 172.21.15.37:0/3328707481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0c107510 con 0x7f3d0c101da0 2024-03-26T06:20:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.708+0000 7f3d0affd640 1 -- 172.21.15.37:0/3328707481 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3cfc044070 con 0x7f3d0c101da0 2024-03-26T06:20:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.708+0000 7f3d0affd640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f3cf40a2800 0x7f3cf40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.708+0000 7f3d11eb2640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f3cf40a2800 0x7f3cf40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.709+0000 7f3d0affd640 1 -- 172.21.15.37:0/3328707481 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3cfc0f0760 con 0x7f3d0c101da0 2024-03-26T06:20:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.709+0000 7f3d11eb2640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f3cf40a2800 0x7f3cf40a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c101240 tx=0x7f3d04009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.713+0000 7f3d0affd640 1 -- 172.21.15.37:0/3328707481 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3cfc0bd8b0 con 0x7f3d0c101da0 2024-03-26T06:20:55.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:55.938+0000 7f3d14116640 1 -- 172.21.15.37:0/3328707481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f3d0c061ef0 con 0x7f3d0c101da0 2024-03-26T06:20:56.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:56 smithi037 ceph-mon[31974]: pgmap v288: 97 pgs: 25 creating+peering, 67 unknown, 5 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:56.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:56 smithi037 ceph-mon[31974]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T06:20:56.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:56 smithi037 ceph-mon[31974]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T06:20:56.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:56 smithi037 ceph-mon[31974]: Cluster is now healthy 2024-03-26T06:20:56.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:56 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2522170889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T06:20:56.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:56 smithi037 ceph-mon[31974]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T06:20:56.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:56 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3328707481' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T06:20:56.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:56 smithi149 ceph-mon[36681]: pgmap v288: 97 pgs: 25 creating+peering, 67 unknown, 5 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:56.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:56 smithi149 ceph-mon[36681]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T06:20:56.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:56 smithi149 ceph-mon[36681]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T06:20:56.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:56 smithi149 ceph-mon[36681]: Cluster is now healthy 2024-03-26T06:20:56.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:56 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2522170889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T06:20:56.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:56 smithi149 ceph-mon[36681]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T06:20:56.561 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:56 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3328707481' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T06:20:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.585+0000 7f3d0affd640 1 -- 172.21.15.37:0/3328707481 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3cfc0bd1b0 con 0x7f3d0c101da0 2024-03-26T06:20:56.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 -- 172.21.15.37:0/3328707481 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f3cf40a2800 msgr2=0x7f3cf40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:56.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f3cf40a2800 0x7f3cf40a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c101240 tx=0x7f3d04009670 comp rx=0 tx=0).stop 2024-03-26T06:20:56.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 -- 172.21.15.37:0/3328707481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d0c101da0 msgr2=0x7f3d0c19da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:56.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d0c101da0 0x7f3d0c19da80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc02ce70 tx=0x7f3cfc005e40 comp rx=0 tx=0).stop 2024-03-26T06:20:56.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 -- 172.21.15.37:0/3328707481 shutdown_connections 2024-03-26T06:20:56.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f3cf40a2800 0x7f3cf40a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:56.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d0c101da0 0x7f3d0c19da80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:56.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c1013e0 0x7f3d0c196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:56.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 --2- 172.21.15.37:0/3328707481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d0c1001f0 0x7f3d0c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:56.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 -- 172.21.15.37:0/3328707481 >> 172.21.15.37:0/3328707481 conn(0x7f3d0c0fb780 msgr2=0x7f3d0c1046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:56.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 -- 172.21.15.37:0/3328707481 shutdown_connections 2024-03-26T06:20:56.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.588+0000 7f3d14116640 1 -- 172.21.15.37:0/3328707481 wait complete. 2024-03-26T06:20:56.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:56 smithi066 ceph-mon[36504]: pgmap v288: 97 pgs: 25 creating+peering, 67 unknown, 5 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:20:56.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:56 smithi066 ceph-mon[36504]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T06:20:56.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:56 smithi066 ceph-mon[36504]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T06:20:56.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:56 smithi066 ceph-mon[36504]: Cluster is now healthy 2024-03-26T06:20:56.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:56 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2522170889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T06:20:56.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:56 smithi066 ceph-mon[36504]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T06:20:56.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:56 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3328707481' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T06:20:56.603 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-26T06:20:56.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.712+0000 7f437c82b640 1 -- 172.21.15.37:0/1342378202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4378102ec0 msgr2=0x7f4378109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:56.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.712+0000 7f437c82b640 1 --2- 172.21.15.37:0/1342378202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4378102ec0 0x7f4378109750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4370009ca0 tx=0x7f437002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:20:56.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.712+0000 7f437c82b640 1 -- 172.21.15.37:0/1342378202 shutdown_connections 2024-03-26T06:20:56.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.712+0000 7f437c82b640 1 --2- 172.21.15.37:0/1342378202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4378102ec0 0x7f4378109750 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:56.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.712+0000 7f437c82b640 1 --2- 172.21.15.37:0/1342378202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4378102500 0x7f4378102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:56.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.712+0000 7f437c82b640 1 --2- 172.21.15.37:0/1342378202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4378101310 0x7f4378101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:56.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.712+0000 7f437c82b640 1 -- 172.21.15.37:0/1342378202 >> 172.21.15.37:0/1342378202 conn(0x7f43780fc880 msgr2=0x7f43780fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:56.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.712+0000 7f437c82b640 1 -- 172.21.15.37:0/1342378202 shutdown_connections 2024-03-26T06:20:56.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.712+0000 7f437c82b640 1 -- 172.21.15.37:0/1342378202 wait complete. 2024-03-26T06:20:56.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437c82b640 1 Processor -- start 2024-03-26T06:20:56.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437c82b640 1 -- start start 2024-03-26T06:20:56.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437c82b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4378101310 0x7f43780750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437c82b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4378102500 0x7f43780736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437c82b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4378102ec0 0x7f4378073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437c82b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437810d080 con 0x7f4378101310 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437c82b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437810cf00 con 0x7f4378102500 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437c82b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437810d200 con 0x7f4378102ec0 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4378101310 0x7f43780750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4378101310 0x7f43780750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50446/0 (socket says 172.21.15.37:50446) 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437659c640 1 -- 172.21.15.37:0/2687199736 learned_addr learned my addr 172.21.15.37:0/2687199736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f436ffff640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4378102500 0x7f43780736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437659c640 1 -- 172.21.15.37:0/2687199736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4378102ec0 msgr2=0x7f4378073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437659c640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4378102ec0 0x7f4378073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437659c640 1 -- 172.21.15.37:0/2687199736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4378102500 msgr2=0x7f43780736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437659c640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4378102500 0x7f43780736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.713+0000 7f437659c640 1 -- 172.21.15.37:0/2687199736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43780744b0 con 0x7f4378101310 2024-03-26T06:20:56.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.714+0000 7f437659c640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4378101310 0x7f43780750a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f436800b450 tx=0x7f436800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:56.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.714+0000 7f436f7fe640 1 -- 172.21.15.37:0/2687199736 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4368015020 con 0x7f4378101310 2024-03-26T06:20:56.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.714+0000 7f437c82b640 1 -- 172.21.15.37:0/2687199736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43781a17b0 con 0x7f4378101310 2024-03-26T06:20:56.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.714+0000 7f436f7fe640 1 -- 172.21.15.37:0/2687199736 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4368011d70 con 0x7f4378101310 2024-03-26T06:20:56.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.714+0000 7f436f7fe640 1 -- 172.21.15.37:0/2687199736 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436800f340 con 0x7f4378101310 2024-03-26T06:20:56.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.714+0000 7f437c82b640 1 -- 172.21.15.37:0/2687199736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43781a1d80 con 0x7f4378101310 2024-03-26T06:20:56.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.715+0000 7f437c82b640 1 -- 172.21.15.37:0/2687199736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4378108630 con 0x7f4378101310 2024-03-26T06:20:56.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.716+0000 7f436f7fe640 1 -- 172.21.15.37:0/2687199736 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4368018020 con 0x7f4378101310 2024-03-26T06:20:56.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.716+0000 7f436f7fe640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f43640a2800 0x7f43640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:56.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.717+0000 7f436ffff640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f43640a2800 0x7f43640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:56.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.717+0000 7f436f7fe640 1 -- 172.21.15.37:0/2687199736 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f43680ce560 con 0x7f4378101310 2024-03-26T06:20:56.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.717+0000 7f436ffff640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f43640a2800 0x7f43640a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4378074e10 tx=0x7f4360008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:56.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.721+0000 7f436f7fe640 1 -- 172.21.15.37:0/2687199736 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f436801f020 con 0x7f4378101310 2024-03-26T06:20:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:56.944+0000 7f437c82b640 1 -- 172.21.15.37:0/2687199736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f437802ce70 con 0x7f4378101310 2024-03-26T06:20:57.574 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:57 smithi149 ceph-mon[36681]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T06:20:57.575 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:57 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3328707481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T06:20:57.575 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:57 smithi149 ceph-mon[36681]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T06:20:57.575 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:57 smithi149 ceph-mon[36681]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T06:20:57.575 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:57 smithi149 ceph-mon[36681]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-26T06:20:57.575 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:57 smithi149 ceph-mon[36681]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-26T06:20:57.575 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:57 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2687199736' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T06:20:57.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:57 smithi066 ceph-mon[36504]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T06:20:57.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:57 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3328707481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T06:20:57.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:57 smithi066 ceph-mon[36504]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T06:20:57.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:57 smithi066 ceph-mon[36504]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T06:20:57.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:57 smithi066 ceph-mon[36504]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-26T06:20:57.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:57 smithi066 ceph-mon[36504]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-26T06:20:57.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:57 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2687199736' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T06:20:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.600+0000 7f436f7fe640 1 -- 172.21.15.37:0/2687199736 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f436800d040 con 0x7f4378101310 2024-03-26T06:20:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.602+0000 7f437c82b640 1 -- 172.21.15.37:0/2687199736 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f43640a2800 msgr2=0x7f43640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.602+0000 7f437c82b640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f43640a2800 0x7f43640a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4378074e10 tx=0x7f4360008040 comp rx=0 tx=0).stop 2024-03-26T06:20:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.602+0000 7f437c82b640 1 -- 172.21.15.37:0/2687199736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4378101310 msgr2=0x7f43780750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.602+0000 7f437c82b640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4378101310 0x7f43780750a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f436800b450 tx=0x7f436800b910 comp rx=0 tx=0).stop 2024-03-26T06:20:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.602+0000 7f437c82b640 1 -- 172.21.15.37:0/2687199736 shutdown_connections 2024-03-26T06:20:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.603+0000 7f437c82b640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f43640a2800 0x7f43640a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.603+0000 7f437c82b640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4378102ec0 0x7f4378073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.603+0000 7f437c82b640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4378102500 0x7f43780736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.603+0000 7f437c82b640 1 --2- 172.21.15.37:0/2687199736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4378101310 0x7f43780750a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.603+0000 7f437c82b640 1 -- 172.21.15.37:0/2687199736 >> 172.21.15.37:0/2687199736 conn(0x7f43780fc880 msgr2=0x7f4378105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.603+0000 7f437c82b640 1 -- 172.21.15.37:0/2687199736 shutdown_connections 2024-03-26T06:20:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.603+0000 7f437c82b640 1 -- 172.21.15.37:0/2687199736 wait complete. 2024-03-26T06:20:57.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:57 smithi037 ceph-mon[31974]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T06:20:57.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:57 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3328707481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T06:20:57.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:57 smithi037 ceph-mon[31974]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T06:20:57.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:57 smithi037 ceph-mon[31974]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T06:20:57.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:57 smithi037 ceph-mon[31974]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-26T06:20:57.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:57 smithi037 ceph-mon[31974]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-26T06:20:57.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:57 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2687199736' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T06:20:57.620 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-26T06:20:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.730+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/4158898318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c68100200 msgr2=0x7f2c68100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.730+0000 7f2c6ca2b640 1 --2- 172.21.15.37:0/4158898318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c68100200 0x7f2c68100620 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009900 tx=0x7f2c5802bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.730+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/4158898318 shutdown_connections 2024-03-26T06:20:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.730+0000 7f2c6ca2b640 1 --2- 172.21.15.37:0/4158898318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c68101db0 0x7f2c68108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.730+0000 7f2c6ca2b640 1 --2- 172.21.15.37:0/4158898318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c681013f0 0x7f2c68101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.730+0000 7f2c6ca2b640 1 --2- 172.21.15.37:0/4158898318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c68100200 0x7f2c68100620 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.730+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/4158898318 >> 172.21.15.37:0/4158898318 conn(0x7f2c680fb790 msgr2=0x7f2c680fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.730+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/4158898318 shutdown_connections 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.730+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/4158898318 wait complete. 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c6ca2b640 1 Processor -- start 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c6ca2b640 1 -- start start 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c6ca2b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c68100200 0x7f2c68104190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c6ca2b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c681013f0 0x7f2c681027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c6ca2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c68101db0 0x7f2c68102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c6ca2b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6810be90 con 0x7f2c68101db0 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c6ca2b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6810bd10 con 0x7f2c681013f0 2024-03-26T06:20:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c6ca2b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6810c010 con 0x7f2c68100200 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c66d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c68101db0 0x7f2c68102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c66d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c68101db0 0x7f2c68102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46586/0 (socket says 172.21.15.37:46586) 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c66d9d640 1 -- 172.21.15.37:0/2752979110 learned_addr learned my addr 172.21.15.37:0/2752979110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c6659c640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c68100200 0x7f2c68104190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.731+0000 7f2c5ffff640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c681013f0 0x7f2c681027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c66d9d640 1 -- 172.21.15.37:0/2752979110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c68100200 msgr2=0x7f2c68104190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c66d9d640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c68100200 0x7f2c68104190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c66d9d640 1 -- 172.21.15.37:0/2752979110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c681013f0 msgr2=0x7f2c681027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c66d9d640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c681013f0 0x7f2c681027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c66d9d640 1 -- 172.21.15.37:0/2752979110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c681035a0 con 0x7f2c68101db0 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c5ffff640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c681013f0 0x7f2c681027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:20:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c66d9d640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c68101db0 0x7f2c68102d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000eaa0 tx=0x7f2c6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c5f7fe640 1 -- 172.21.15.37:0/2752979110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000cdb0 con 0x7f2c68101db0 2024-03-26T06:20:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c5f7fe640 1 -- 172.21.15.37:0/2752979110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c60005020 con 0x7f2c68101db0 2024-03-26T06:20:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/2752979110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c681a17b0 con 0x7f2c68101db0 2024-03-26T06:20:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c5f7fe640 1 -- 172.21.15.37:0/2752979110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c600116b0 con 0x7f2c68101db0 2024-03-26T06:20:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.732+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/2752979110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c681a1cb0 con 0x7f2c68101db0 2024-03-26T06:20:57.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.733+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/2752979110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c681a1fd0 con 0x7f2c68101db0 2024-03-26T06:20:57.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.734+0000 7f2c5f7fe640 1 -- 172.21.15.37:0/2752979110 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2c60005560 con 0x7f2c68101db0 2024-03-26T06:20:57.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.735+0000 7f2c5f7fe640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2c540a2800 0x7f2c540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:57.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.735+0000 7f2c6659c640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2c540a2800 0x7f2c540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:57.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.735+0000 7f2c5f7fe640 1 -- 172.21.15.37:0/2752979110 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2c600cf070 con 0x7f2c68101db0 2024-03-26T06:20:57.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.735+0000 7f2c6659c640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2c540a2800 0x7f2c540a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2c58000c00 tx=0x7f2c58005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:57.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.738+0000 7f2c5f7fe640 1 -- 172.21.15.37:0/2752979110 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2c60016030 con 0x7f2c68101db0 2024-03-26T06:20:57.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:57.959+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/2752979110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f2c681a2320 con 0x7f2c68101db0 2024-03-26T06:20:58.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.606+0000 7f2c5f7fe640 1 -- 172.21.15.37:0/2752979110 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2c6009c1c0 con 0x7f2c68101db0 2024-03-26T06:20:58.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/2752979110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2c540a2800 msgr2=0x7f2c540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:58.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2c540a2800 0x7f2c540a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2c58000c00 tx=0x7f2c58005fd0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/2752979110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c68101db0 msgr2=0x7f2c68102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:58.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c68101db0 0x7f2c68102d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000eaa0 tx=0x7f2c6000ef60 comp rx=0 tx=0).stop 2024-03-26T06:20:58.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/2752979110 shutdown_connections 2024-03-26T06:20:58.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2c540a2800 0x7f2c540a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c68101db0 0x7f2c68102d10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c681013f0 0x7f2c681027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 --2- 172.21.15.37:0/2752979110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c68100200 0x7f2c68104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/2752979110 >> 172.21.15.37:0/2752979110 conn(0x7f2c680fb790 msgr2=0x7f2c680fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:58.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/2752979110 shutdown_connections 2024-03-26T06:20:58.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.609+0000 7f2c6ca2b640 1 -- 172.21.15.37:0/2752979110 wait complete. 2024-03-26T06:20:58.627 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-26T06:20:58.628 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-26T06:20:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.738+0000 7fd6e70a4640 1 -- 172.21.15.37:0/626382083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6e0100210 msgr2=0x7fd6e0100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:58.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.738+0000 7fd6e70a4640 1 --2- 172.21.15.37:0/626382083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6e0100210 0x7fd6e0100630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009900 tx=0x7fd6d402bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:58.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.738+0000 7fd6e70a4640 1 -- 172.21.15.37:0/626382083 shutdown_connections 2024-03-26T06:20:58.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.738+0000 7fd6e70a4640 1 --2- 172.21.15.37:0/626382083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6e0101dc0 0x7fd6e0108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.738+0000 7fd6e70a4640 1 --2- 172.21.15.37:0/626382083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e0101400 0x7fd6e0101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.738+0000 7fd6e70a4640 1 --2- 172.21.15.37:0/626382083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6e0100210 0x7fd6e0100630 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.738+0000 7fd6e70a4640 1 -- 172.21.15.37:0/626382083 >> 172.21.15.37:0/626382083 conn(0x7fd6e00fb7a0 msgr2=0x7fd6e00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:58.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.738+0000 7fd6e70a4640 1 -- 172.21.15.37:0/626382083 shutdown_connections 2024-03-26T06:20:58.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.738+0000 7fd6e70a4640 1 -- 172.21.15.37:0/626382083 wait complete. 2024-03-26T06:20:58.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e70a4640 1 Processor -- start 2024-03-26T06:20:58.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e70a4640 1 -- start start 2024-03-26T06:20:58.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e70a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6e0100210 0x7fd6e01964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e70a4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6e0101400 0x7fd6e0196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e70a4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e0101dc0 0x7fd6e019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e70a4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e010bea0 con 0x7fd6e0100210 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e70a4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e010bd20 con 0x7fd6e0101400 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e70a4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e010c020 con 0x7fd6e0101dc0 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e4e40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6e0100210 0x7fd6e01964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e4e40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6e0100210 0x7fd6e01964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46602/0 (socket says 172.21.15.37:46602) 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e4e40640 1 -- 172.21.15.37:0/185452027 learned_addr learned my addr 172.21.15.37:0/185452027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e5641640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e0101dc0 0x7fd6e019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6dffff640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6e0101400 0x7fd6e0196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e4e40640 1 -- 172.21.15.37:0/185452027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e0101dc0 msgr2=0x7fd6e019da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e4e40640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e0101dc0 0x7fd6e019da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e4e40640 1 -- 172.21.15.37:0/185452027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6e0101400 msgr2=0x7fd6e0196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:58.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e4e40640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6e0101400 0x7fd6e0196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e4e40640 1 -- 172.21.15.37:0/185452027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e019e0f0 con 0x7fd6e0100210 2024-03-26T06:20:58.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6dffff640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6e0101400 0x7fd6e0196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:20:58.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.739+0000 7fd6e4e40640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6e0100210 0x7fd6e01964d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4000c00 tx=0x7fd6d4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:58.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.740+0000 7fd6ddffb640 1 -- 172.21.15.37:0/185452027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4041070 con 0x7fd6e0100210 2024-03-26T06:20:58.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.740+0000 7fd6ddffb640 1 -- 172.21.15.37:0/185452027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6d40044d0 con 0x7fd6e0100210 2024-03-26T06:20:58.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.740+0000 7fd6e70a4640 1 -- 172.21.15.37:0/185452027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e019e380 con 0x7fd6e0100210 2024-03-26T06:20:58.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.740+0000 7fd6ddffb640 1 -- 172.21.15.37:0/185452027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4034440 con 0x7fd6e0100210 2024-03-26T06:20:58.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.740+0000 7fd6e70a4640 1 -- 172.21.15.37:0/185452027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e019e870 con 0x7fd6e0100210 2024-03-26T06:20:58.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.741+0000 7fd6e70a4640 1 -- 172.21.15.37:0/185452027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e0107530 con 0x7fd6e0100210 2024-03-26T06:20:58.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.742+0000 7fd6ddffb640 1 -- 172.21.15.37:0/185452027 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd6d4044070 con 0x7fd6e0100210 2024-03-26T06:20:58.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.742+0000 7fd6ddffb640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd6d00a2800 0x7fd6d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:58.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.743+0000 7fd6dffff640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd6d00a2800 0x7fd6d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:58.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.743+0000 7fd6ddffb640 1 -- 172.21.15.37:0/185452027 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd6d40f0770 con 0x7fd6e0100210 2024-03-26T06:20:58.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.743+0000 7fd6dffff640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd6d00a2800 0x7fd6d00a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd6e01979e0 tx=0x7fd6cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:58.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.746+0000 7fd6ddffb640 1 -- 172.21.15.37:0/185452027 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd6d40bd8c0 con 0x7fd6e0100210 2024-03-26T06:20:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:58 smithi149 ceph-mon[36681]: pgmap v290: 97 pgs: 25 creating+peering, 35 unknown, 37 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-03-26T06:20:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:58 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2687199736' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T06:20:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:58 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] up:active 2024-03-26T06:20:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:58 smithi149 ceph-mon[36681]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T06:20:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:58 smithi149 ceph-mon[36681]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-26T06:20:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:58 smithi149 ceph-mon[36681]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-26T06:20:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:58 smithi149 ceph-mon[36681]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-26T06:20:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:58 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2752979110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T06:20:58.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-mon[31974]: pgmap v290: 97 pgs: 25 creating+peering, 35 unknown, 37 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-03-26T06:20:58.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2687199736' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T06:20:58.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] up:active 2024-03-26T06:20:58.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-mon[31974]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T06:20:58.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-mon[31974]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-26T06:20:58.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-mon[31974]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-26T06:20:58.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-mon[31974]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-26T06:20:58.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2752979110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T06:20:58.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.956+0000 7fd6e70a4640 1 -- 172.21.15.37:0/185452027 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6e0061730 con 0x7fd6d00a2800 2024-03-26T06:20:58.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.979+0000 7fd6ddffb640 1 -- 172.21.15.37:0/185452027 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd6e0061730 con 0x7fd6d00a2800 2024-03-26T06:20:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 -- 172.21.15.37:0/185452027 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd6d00a2800 msgr2=0x7fd6d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd6d00a2800 0x7fd6d00a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd6e01979e0 tx=0x7fd6cc005f90 comp rx=0 tx=0).stop 2024-03-26T06:20:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 -- 172.21.15.37:0/185452027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6e0100210 msgr2=0x7fd6e01964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6e0100210 0x7fd6e01964d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4000c00 tx=0x7fd6d4005d50 comp rx=0 tx=0).stop 2024-03-26T06:20:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 -- 172.21.15.37:0/185452027 shutdown_connections 2024-03-26T06:20:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd6d00a2800 0x7fd6d00a4cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e0101dc0 0x7fd6e019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6e0101400 0x7fd6e0196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 --2- 172.21.15.37:0/185452027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6e0100210 0x7fd6e01964d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 -- 172.21.15.37:0/185452027 >> 172.21.15.37:0/185452027 conn(0x7fd6e00fb7a0 msgr2=0x7fd6e0104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 -- 172.21.15.37:0/185452027 shutdown_connections 2024-03-26T06:20:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:58.981+0000 7fd6e70a4640 1 -- 172.21.15.37:0/185452027 wait complete. 2024-03-26T06:20:58.993 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-26T06:20:59.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:58 smithi066 ceph-mon[36504]: pgmap v290: 97 pgs: 25 creating+peering, 35 unknown, 37 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-03-26T06:20:59.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:58 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2687199736' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T06:20:59.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:58 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] up:active 2024-03-26T06:20:59.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:58 smithi066 ceph-mon[36504]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T06:20:59.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:58 smithi066 ceph-mon[36504]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-26T06:20:59.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:58 smithi066 ceph-mon[36504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-26T06:20:59.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:58 smithi066 ceph-mon[36504]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-26T06:20:59.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:58 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2752979110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T06:20:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.100+0000 7f5272a62640 1 -- 172.21.15.37:0/2824823983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c10cef0 msgr2=0x7f526c10d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.100+0000 7f5272a62640 1 --2- 172.21.15.37:0/2824823983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c10cef0 0x7f526c10d370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5254009900 tx=0x7f525402bc30 comp rx=0 tx=0).stop 2024-03-26T06:20:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.100+0000 7f5272a62640 1 -- 172.21.15.37:0/2824823983 shutdown_connections 2024-03-26T06:20:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.100+0000 7f5272a62640 1 --2- 172.21.15.37:0/2824823983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f526c10d8b0 0x7f526c114140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.100+0000 7f5272a62640 1 --2- 172.21.15.37:0/2824823983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c10cef0 0x7f526c10d370 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.100+0000 7f5272a62640 1 --2- 172.21.15.37:0/2824823983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526c10bd00 0x7f526c10c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.100+0000 7f5272a62640 1 -- 172.21.15.37:0/2824823983 >> 172.21.15.37:0/2824823983 conn(0x7f526c06bcb0 msgr2=0x7f526c06c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.100+0000 7f5272a62640 1 -- 172.21.15.37:0/2824823983 shutdown_connections 2024-03-26T06:20:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.100+0000 7f5272a62640 1 -- 172.21.15.37:0/2824823983 wait complete. 2024-03-26T06:20:59.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.100+0000 7f5272a62640 1 Processor -- start 2024-03-26T06:20:59.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.100+0000 7f5272a62640 1 -- start start 2024-03-26T06:20:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f5272a62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c10bd00 0x7f526c1a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f5272a62640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526c10cef0 0x7f526c1a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f5272a62640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f526c10d8b0 0x7f526c1a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f5272a62640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c117a30 con 0x7f526c10bd00 2024-03-26T06:20:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f5272a62640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c1178b0 con 0x7f526c10cef0 2024-03-26T06:20:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f5272a62640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c117bb0 con 0x7f526c10d8b0 2024-03-26T06:20:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f526bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c10bd00 0x7f526c1a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f526bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c10bd00 0x7f526c1a20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46630/0 (socket says 172.21.15.37:46630) 2024-03-26T06:20:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f526bfff640 1 -- 172.21.15.37:0/1185746204 learned_addr learned my addr 172.21.15.37:0/1185746204 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f526b7fe640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526c10cef0 0x7f526c1a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f5270fff640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f526c10d8b0 0x7f526c1a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f526bfff640 1 -- 172.21.15.37:0/1185746204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f526c10d8b0 msgr2=0x7f526c1a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f526bfff640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f526c10d8b0 0x7f526c1a9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f526bfff640 1 -- 172.21.15.37:0/1185746204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526c10cef0 msgr2=0x7f526c1a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f526bfff640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526c10cef0 0x7f526c1a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f526bfff640 1 -- 172.21.15.37:0/1185746204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c1a9cc0 con 0x7f526c10bd00 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f5270fff640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f526c10d8b0 0x7f526c1a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f526bfff640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c10bd00 0x7f526c1a20a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f525c00d770 tx=0x7f525c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f52697fa640 1 -- 172.21.15.37:0/1185746204 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c017070 con 0x7f526c10bd00 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f52697fa640 1 -- 172.21.15.37:0/1185746204 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f525c009c30 con 0x7f526c10bd00 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f5272a62640 1 -- 172.21.15.37:0/1185746204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526c1a9fb0 con 0x7f526c10bd00 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f52697fa640 1 -- 172.21.15.37:0/1185746204 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c003dd0 con 0x7f526c10bd00 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.101+0000 7f5272a62640 1 -- 172.21.15.37:0/1185746204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526c1aa410 con 0x7f526c10bd00 2024-03-26T06:20:59.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.102+0000 7f5272a62640 1 -- 172.21.15.37:0/1185746204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526c113020 con 0x7f526c10bd00 2024-03-26T06:20:59.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.104+0000 7f52697fa640 1 -- 172.21.15.37:0/1185746204 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f525c003f90 con 0x7f526c10bd00 2024-03-26T06:20:59.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.104+0000 7f52697fa640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52580a2800 0x7f52580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.104+0000 7f526b7fe640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52580a2800 0x7f52580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.104+0000 7f52697fa640 1 -- 172.21.15.37:0/1185746204 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f525c0ce6d0 con 0x7f526c10bd00 2024-03-26T06:20:59.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.105+0000 7f526b7fe640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52580a2800 0x7f52580a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f525402c240 tx=0x7f525402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:59.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.108+0000 7f52697fa640 1 -- 172.21.15.37:0/1185746204 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f525c009de0 con 0x7f526c10bd00 2024-03-26T06:20:59.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.320+0000 7f5272a62640 1 -- 172.21.15.37:0/1185746204 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f526c0617c0 con 0x7f52580a2800 2024-03-26T06:20:59.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.340+0000 7f52697fa640 1 -- 172.21.15.37:0/1185746204 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f526c0617c0 con 0x7f52580a2800 2024-03-26T06:20:59.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.342+0000 7f5272a62640 1 -- 172.21.15.37:0/1185746204 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52580a2800 msgr2=0x7f52580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.342+0000 7f5272a62640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52580a2800 0x7f52580a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f525402c240 tx=0x7f525402df90 comp rx=0 tx=0).stop 2024-03-26T06:20:59.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.342+0000 7f5272a62640 1 -- 172.21.15.37:0/1185746204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c10bd00 msgr2=0x7f526c1a20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.342+0000 7f5272a62640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c10bd00 0x7f526c1a20a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f525c00d770 tx=0x7f525c00dc30 comp rx=0 tx=0).stop 2024-03-26T06:20:59.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.343+0000 7f5272a62640 1 -- 172.21.15.37:0/1185746204 shutdown_connections 2024-03-26T06:20:59.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.343+0000 7f5272a62640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52580a2800 0x7f52580a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.343+0000 7f5272a62640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f526c10d8b0 0x7f526c1a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.343+0000 7f5272a62640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f526c10cef0 0x7f526c1a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.343+0000 7f5272a62640 1 --2- 172.21.15.37:0/1185746204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c10bd00 0x7f526c1a20a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.343+0000 7f5272a62640 1 -- 172.21.15.37:0/1185746204 >> 172.21.15.37:0/1185746204 conn(0x7f526c06bcb0 msgr2=0x7f526c1101f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:59.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.343+0000 7f5272a62640 1 -- 172.21.15.37:0/1185746204 shutdown_connections 2024-03-26T06:20:59.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.343+0000 7f5272a62640 1 -- 172.21.15.37:0/1185746204 wait complete. 2024-03-26T06:20:59.353 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:20:58.961+0000 7f67584c5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:20:59.354 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:20:58.961+0000 7f67584c5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:20:59.354 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:20:58.961+0000 7f67584c5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:20:59.354 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:20:58.961+0000 7f67584c5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:20:59.354 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:20:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:20:58.961+0000 7f67584c5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:20:59.355 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-26T06:20:59.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.461+0000 7fc8352dc640 1 -- 172.21.15.37:0/2180711276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc830100ed0 msgr2=0x7fc830101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.461+0000 7fc8352dc640 1 --2- 172.21.15.37:0/2180711276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc830100ed0 0x7fc830101350 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc820009ca0 tx=0x7fc82002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.461+0000 7fc8352dc640 1 -- 172.21.15.37:0/2180711276 shutdown_connections 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.461+0000 7fc8352dc640 1 --2- 172.21.15.37:0/2180711276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc830100ed0 0x7fc830101350 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.461+0000 7fc8352dc640 1 --2- 172.21.15.37:0/2180711276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc830100530 0x7fc830100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.461+0000 7fc8352dc640 1 --2- 172.21.15.37:0/2180711276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8300ffbd0 0x7fc8300ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.461+0000 7fc8352dc640 1 -- 172.21.15.37:0/2180711276 >> 172.21.15.37:0/2180711276 conn(0x7fc8300fb770 msgr2=0x7fc8300fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.461+0000 7fc8352dc640 1 -- 172.21.15.37:0/2180711276 shutdown_connections 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.461+0000 7fc8352dc640 1 -- 172.21.15.37:0/2180711276 wait complete. 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc8352dc640 1 Processor -- start 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc8352dc640 1 -- start start 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc8352dc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8300ffbd0 0x7fc830199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc8352dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc830100530 0x7fc830199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc8352dc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc830100ed0 0x7fc8301a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc8352dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83010f1a0 con 0x7fc830100530 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc8352dc640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83010f020 con 0x7fc8300ffbd0 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc8352dc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83010f320 con 0x7fc830100ed0 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc82e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc830100530 0x7fc830199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc82e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc830100530 0x7fc830199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46654/0 (socket says 172.21.15.37:46654) 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc82e7fc640 1 -- 172.21.15.37:0/3828297833 learned_addr learned my addr 172.21.15.37:0/3828297833 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc82effd640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8300ffbd0 0x7fc830199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc82f7fe640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc830100ed0 0x7fc8301a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc82e7fc640 1 -- 172.21.15.37:0/3828297833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc830100ed0 msgr2=0x7fc8301a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc82e7fc640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc830100ed0 0x7fc8301a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc82e7fc640 1 -- 172.21.15.37:0/3828297833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8300ffbd0 msgr2=0x7fc830199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.462+0000 7fc82e7fc640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8300ffbd0 0x7fc830199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.463+0000 7fc82e7fc640 1 -- 172.21.15.37:0/3828297833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8301a13a0 con 0x7fc830100530 2024-03-26T06:20:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.463+0000 7fc82e7fc640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc830100530 0x7fc830199cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc81800e9f0 tx=0x7fc81800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.463+0000 7fc827fff640 1 -- 172.21.15.37:0/3828297833 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800cd80 con 0x7fc830100530 2024-03-26T06:20:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.463+0000 7fc827fff640 1 -- 172.21.15.37:0/3828297833 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc818010910 con 0x7fc830100530 2024-03-26T06:20:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.463+0000 7fc8352dc640 1 -- 172.21.15.37:0/3828297833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8301a1690 con 0x7fc830100530 2024-03-26T06:20:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.463+0000 7fc827fff640 1 -- 172.21.15.37:0/3828297833 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818018990 con 0x7fc830100530 2024-03-26T06:20:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.463+0000 7fc8352dc640 1 -- 172.21.15.37:0/3828297833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8301a1be0 con 0x7fc830100530 2024-03-26T06:20:59.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.464+0000 7fc8352dc640 1 -- 172.21.15.37:0/3828297833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83010aab0 con 0x7fc830100530 2024-03-26T06:20:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.465+0000 7fc827fff640 1 -- 172.21.15.37:0/3828297833 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc818010ac0 con 0x7fc830100530 2024-03-26T06:20:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.465+0000 7fc827fff640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc81c0a2800 0x7fc81c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.466+0000 7fc82effd640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc81c0a2800 0x7fc81c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.466+0000 7fc827fff640 1 -- 172.21.15.37:0/3828297833 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc8180cf1d0 con 0x7fc830100530 2024-03-26T06:20:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.466+0000 7fc82effd640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc81c0a2800 0x7fc81c0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc828005d00 tx=0x7fc828005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.469+0000 7fc827fff640 1 -- 172.21.15.37:0/3828297833 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc81809c320 con 0x7fc830100530 2024-03-26T06:20:59.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.629+0000 7fc827fff640 1 -- 172.21.15.37:0/3828297833 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fc818098290 con 0x7fc830100530 2024-03-26T06:20:59.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.703+0000 7fc8352dc640 1 -- 172.21.15.37:0/3828297833 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8300617c0 con 0x7fc81c0a2800 2024-03-26T06:20:59.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.707+0000 7fc827fff640 1 -- 172.21.15.37:0/3828297833 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fc8300617c0 con 0x7fc81c0a2800 2024-03-26T06:20:59.708 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_0/8329bfdf-3f9c-4a0d-8d09-f147880d7842 2024-03-26T06:20:59.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 -- 172.21.15.37:0/3828297833 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc81c0a2800 msgr2=0x7fc81c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc81c0a2800 0x7fc81c0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc828005d00 tx=0x7fc828005c50 comp rx=0 tx=0).stop 2024-03-26T06:20:59.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 -- 172.21.15.37:0/3828297833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc830100530 msgr2=0x7fc830199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc830100530 0x7fc830199cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc81800e9f0 tx=0x7fc81800eeb0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 -- 172.21.15.37:0/3828297833 shutdown_connections 2024-03-26T06:20:59.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc81c0a2800 0x7fc81c0a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc830100ed0 0x7fc8301a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc830100530 0x7fc830199cc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 --2- 172.21.15.37:0/3828297833 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8300ffbd0 0x7fc830199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 -- 172.21.15.37:0/3828297833 >> 172.21.15.37:0/3828297833 conn(0x7fc8300fb770 msgr2=0x7fc8300fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 -- 172.21.15.37:0/3828297833 shutdown_connections 2024-03-26T06:20:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.709+0000 7fc8352dc640 1 -- 172.21.15.37:0/3828297833 wait complete. 2024-03-26T06:20:59.721 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-26T06:20:59.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.827+0000 7f63acf09640 1 -- 172.21.15.37:0/785379321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63a8101da0 msgr2=0x7f63a8108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.827+0000 7f63acf09640 1 --2- 172.21.15.37:0/785379321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63a8101da0 0x7f63a8108630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009ca0 tx=0x7f63a002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 -- 172.21.15.37:0/785379321 shutdown_connections 2024-03-26T06:20:59.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 --2- 172.21.15.37:0/785379321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63a8101da0 0x7f63a8108630 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 --2- 172.21.15.37:0/785379321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63a81013e0 0x7f63a8101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 --2- 172.21.15.37:0/785379321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63a81001f0 0x7f63a8100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 -- 172.21.15.37:0/785379321 >> 172.21.15.37:0/785379321 conn(0x7f63a80fb780 msgr2=0x7f63a80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:20:59.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 -- 172.21.15.37:0/785379321 shutdown_connections 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 -- 172.21.15.37:0/785379321 wait complete. 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 Processor -- start 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 -- start start 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63a81001f0 0x7f63a8068fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63a81013e0 0x7f63a819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.828+0000 7f63acf09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63a8101da0 0x7f63a819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63acf09640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a810be80 con 0x7f63a8101da0 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63acf09640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a810bd00 con 0x7f63a81001f0 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63acf09640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a810c000 con 0x7f63a81013e0 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63a8101da0 0x7f63a819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63a8101da0 0x7f63a819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46682/0 (socket says 172.21.15.37:46682) 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63a6d9d640 1 -- 172.21.15.37:0/143134590 learned_addr learned my addr 172.21.15.37:0/143134590 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63a5d9b640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63a81013e0 0x7f63a819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63a659c640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63a81001f0 0x7f63a8068fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63a6d9d640 1 -- 172.21.15.37:0/143134590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63a81013e0 msgr2=0x7f63a819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63a6d9d640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63a81013e0 0x7f63a819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63a6d9d640 1 -- 172.21.15.37:0/143134590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63a81001f0 msgr2=0x7f63a8068fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:20:59.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63a6d9d640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63a81001f0 0x7f63a8068fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:20:59.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63a6d9d640 1 -- 172.21.15.37:0/143134590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a819f1b0 con 0x7f63a8101da0 2024-03-26T06:20:59.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63a6d9d640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63a8101da0 0x7f63a819e8f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f63a002c700 tx=0x7f63a0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:59.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63977fe640 1 -- 172.21.15.37:0/143134590 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0039040 con 0x7f63a8101da0 2024-03-26T06:20:59.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63977fe640 1 -- 172.21.15.37:0/143134590 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63a0044070 con 0x7f63a8101da0 2024-03-26T06:20:59.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63acf09640 1 -- 172.21.15.37:0/143134590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a819f440 con 0x7f63a8101da0 2024-03-26T06:20:59.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63977fe640 1 -- 172.21.15.37:0/143134590 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a00042a0 con 0x7f63a8101da0 2024-03-26T06:20:59.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.829+0000 7f63acf09640 1 -- 172.21.15.37:0/143134590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a819f930 con 0x7f63a8101da0 2024-03-26T06:20:59.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.830+0000 7f63acf09640 1 -- 172.21.15.37:0/143134590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a81074b0 con 0x7f63a8101da0 2024-03-26T06:20:59.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.831+0000 7f63977fe640 1 -- 172.21.15.37:0/143134590 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f63a0003970 con 0x7f63a8101da0 2024-03-26T06:20:59.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.832+0000 7f63977fe640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63900a2890 0x7f63900a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:20:59.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.832+0000 7f63a659c640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63900a2890 0x7f63900a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:20:59.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.832+0000 7f63977fe640 1 -- 172.21.15.37:0/143134590 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f63a00f0fa0 con 0x7f63a8101da0 2024-03-26T06:20:59.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.832+0000 7f63a659c640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63900a2890 0x7f63900a4d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f639c005fd0 tx=0x7f639c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:20:59.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:20:59.836+0000 7f63977fe640 1 -- 172.21.15.37:0/143134590 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f63a00be130 con 0x7f63a8101da0 2024-03-26T06:20:59.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2752979110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T06:20:59.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] up:active 2024-03-26T06:20:59.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T06:20:59.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-26T06:20:59.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:creating} 8 up:standby 2024-03-26T06:20:59.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-26T06:20:59.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] up:active 2024-03-26T06:20:59.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T06:20:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T06:20:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: Cluster is now healthy 2024-03-26T06:20:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:active} 4 up:standby-replay 4 up:standby 2024-03-26T06:20:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-26T06:20:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:20:59 smithi149 ceph-mon[36681]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-26T06:20:59.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2752979110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T06:20:59.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] up:active 2024-03-26T06:20:59.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T06:20:59.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-26T06:20:59.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:creating} 8 up:standby 2024-03-26T06:20:59.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-26T06:20:59.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] up:active 2024-03-26T06:20:59.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T06:20:59.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T06:20:59.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: Cluster is now healthy 2024-03-26T06:20:59.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:active} 4 up:standby-replay 4 up:standby 2024-03-26T06:20:59.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-26T06:20:59.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:20:59 smithi037 ceph-mon[31974]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-26T06:21:00.000 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b[67275]: 2024-03-26T06:20:59.745+0000 7f1a96d60700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-26T06:21:00.000 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2752979110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T06:21:00.000 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] up:active 2024-03-26T06:21:00.000 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T06:21:00.000 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-26T06:21:00.000 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:creating} 8 up:standby 2024-03-26T06:21:00.000 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-26T06:21:00.001 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] up:active 2024-03-26T06:21:00.001 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T06:21:00.001 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T06:21:00.001 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: Cluster is now healthy 2024-03-26T06:21:00.001 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=i=up:active} 4 up:standby-replay 4 up:standby 2024-03-26T06:21:00.001 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-26T06:21:00.001 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:20:59 smithi066 ceph-mon[36504]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-26T06:21:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.046+0000 7f63acf09640 1 -- 172.21.15.37:0/143134590 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f63a80617c0 con 0x7f63900a2890 2024-03-26T06:21:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.071+0000 7f63977fe640 1 -- 172.21.15.37:0/143134590 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f63a80617c0 con 0x7f63900a2890 2024-03-26T06:21:00.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 -- 172.21.15.37:0/143134590 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63900a2890 msgr2=0x7f63900a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63900a2890 0x7f63900a4d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f639c005fd0 tx=0x7f639c005850 comp rx=0 tx=0).stop 2024-03-26T06:21:00.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 -- 172.21.15.37:0/143134590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63a8101da0 msgr2=0x7f63a819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63a8101da0 0x7f63a819e8f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f63a002c700 tx=0x7f63a0032f20 comp rx=0 tx=0).stop 2024-03-26T06:21:00.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 -- 172.21.15.37:0/143134590 shutdown_connections 2024-03-26T06:21:00.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63900a2890 0x7f63900a4d50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63a8101da0 0x7f63a819e8f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63a81013e0 0x7f63a819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 --2- 172.21.15.37:0/143134590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63a81001f0 0x7f63a8068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 -- 172.21.15.37:0/143134590 >> 172.21.15.37:0/143134590 conn(0x7f63a80fb780 msgr2=0x7f63a8104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:00.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 -- 172.21.15.37:0/143134590 shutdown_connections 2024-03-26T06:21:00.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.073+0000 7f63acf09640 1 -- 172.21.15.37:0/143134590 wait complete. 2024-03-26T06:21:00.085 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-26T06:21:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.191+0000 7f2b5ed96640 1 -- 172.21.15.37:0/2775443424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b58102ea0 msgr2=0x7f2b58109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.191+0000 7f2b5ed96640 1 --2- 172.21.15.37:0/2775443424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b58102ea0 0x7f2b58109730 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009ca0 tx=0x7f2b5002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.191+0000 7f2b5ed96640 1 -- 172.21.15.37:0/2775443424 shutdown_connections 2024-03-26T06:21:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.191+0000 7f2b5ed96640 1 --2- 172.21.15.37:0/2775443424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b58102ea0 0x7f2b58109730 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.191+0000 7f2b5ed96640 1 --2- 172.21.15.37:0/2775443424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b581024e0 0x7f2b58102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.191+0000 7f2b5ed96640 1 --2- 172.21.15.37:0/2775443424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b581012f0 0x7f2b58101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.191+0000 7f2b5ed96640 1 -- 172.21.15.37:0/2775443424 >> 172.21.15.37:0/2775443424 conn(0x7f2b580fc860 msgr2=0x7f2b580feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5ed96640 1 -- 172.21.15.37:0/2775443424 shutdown_connections 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5ed96640 1 -- 172.21.15.37:0/2775443424 wait complete. 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5ed96640 1 Processor -- start 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5ed96640 1 -- start start 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5ed96640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b581012f0 0x7f2b58197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5ed96640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b581024e0 0x7f2b58197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5ed96640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b58102ea0 0x7f2b5819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5ed96640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5810cf80 con 0x7f2b581024e0 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5ed96640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5810ce00 con 0x7f2b581012f0 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5ed96640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5810d100 con 0x7f2b58102ea0 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b581024e0 0x7f2b58197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b581024e0 0x7f2b58197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46710/0 (socket says 172.21.15.37:46710) 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b57fff640 1 -- 172.21.15.37:0/1584439703 learned_addr learned my addr 172.21.15.37:0/1584439703 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5cb32640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b581012f0 0x7f2b58197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.192+0000 7f2b5d333640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b58102ea0 0x7f2b5819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b57fff640 1 -- 172.21.15.37:0/1584439703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b58102ea0 msgr2=0x7f2b5819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b57fff640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b58102ea0 0x7f2b5819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b57fff640 1 -- 172.21.15.37:0/1584439703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b581012f0 msgr2=0x7f2b58197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b57fff640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b581012f0 0x7f2b58197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b57fff640 1 -- 172.21.15.37:0/1584439703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5819f1a0 con 0x7f2b581024e0 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b5d333640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b58102ea0 0x7f2b5819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b57fff640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b581024e0 0x7f2b58197ac0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400c970 tx=0x7f2b4400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b55ffb640 1 -- 172.21.15.37:0/1584439703 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44007bb0 con 0x7f2b581024e0 2024-03-26T06:21:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b5ed96640 1 -- 172.21.15.37:0/1584439703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5819f490 con 0x7f2b581024e0 2024-03-26T06:21:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b55ffb640 1 -- 172.21.15.37:0/1584439703 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b44007d50 con 0x7f2b581024e0 2024-03-26T06:21:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b55ffb640 1 -- 172.21.15.37:0/1584439703 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4400f450 con 0x7f2b581024e0 2024-03-26T06:21:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.193+0000 7f2b5ed96640 1 -- 172.21.15.37:0/1584439703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5819f9e0 con 0x7f2b581024e0 2024-03-26T06:21:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.194+0000 7f2b5ed96640 1 -- 172.21.15.37:0/1584439703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b58108610 con 0x7f2b581024e0 2024-03-26T06:21:00.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.195+0000 7f2b55ffb640 1 -- 172.21.15.37:0/1584439703 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2b44005020 con 0x7f2b581024e0 2024-03-26T06:21:00.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.195+0000 7f2b55ffb640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2b480a2890 0x7f2b480a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.196+0000 7f2b5cb32640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2b480a2890 0x7f2b480a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.196+0000 7f2b55ffb640 1 -- 172.21.15.37:0/1584439703 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2b440ce910 con 0x7f2b581024e0 2024-03-26T06:21:00.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.196+0000 7f2b5cb32640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2b480a2890 0x7f2b480a4d50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2b58102340 tx=0x7f2b4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:00.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.200+0000 7f2b55ffb640 1 -- 172.21.15.37:0/1584439703 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2b44014030 con 0x7f2b581024e0 2024-03-26T06:21:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.412+0000 7f2b5ed96640 1 -- 172.21.15.37:0/1584439703 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b58061730 con 0x7f2b480a2890 2024-03-26T06:21:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.415+0000 7f2b55ffb640 1 -- 172.21.15.37:0/1584439703 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f2b58061730 con 0x7f2b480a2890 2024-03-26T06:21:00.416 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_1/c1037191-717f-4157-93f3-2e04e0bac5ca 2024-03-26T06:21:00.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 -- 172.21.15.37:0/1584439703 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2b480a2890 msgr2=0x7f2b480a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2b480a2890 0x7f2b480a4d50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2b58102340 tx=0x7f2b4c005f90 comp rx=0 tx=0).stop 2024-03-26T06:21:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 -- 172.21.15.37:0/1584439703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b581024e0 msgr2=0x7f2b58197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b581024e0 0x7f2b58197ac0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400c970 tx=0x7f2b4400ce30 comp rx=0 tx=0).stop 2024-03-26T06:21:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 -- 172.21.15.37:0/1584439703 shutdown_connections 2024-03-26T06:21:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2b480a2890 0x7f2b480a4d50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b58102ea0 0x7f2b5819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b581024e0 0x7f2b58197ac0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 --2- 172.21.15.37:0/1584439703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b581012f0 0x7f2b58197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 -- 172.21.15.37:0/1584439703 >> 172.21.15.37:0/1584439703 conn(0x7f2b580fc860 msgr2=0x7f2b581057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 -- 172.21.15.37:0/1584439703 shutdown_connections 2024-03-26T06:21:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.417+0000 7f2b5ed96640 1 -- 172.21.15.37:0/1584439703 wait complete. 2024-03-26T06:21:00.429 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:21:00.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.530+0000 7f0573661640 1 -- 172.21.15.37:0/2537842607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f056c100210 msgr2=0x7f056c100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.530+0000 7f0573661640 1 --2- 172.21.15.37:0/2537842607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f056c100210 0x7f056c100630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0568009900 tx=0x7f056802bc30 comp rx=0 tx=0).stop 2024-03-26T06:21:00.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.530+0000 7f0573661640 1 -- 172.21.15.37:0/2537842607 shutdown_connections 2024-03-26T06:21:00.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.530+0000 7f0573661640 1 --2- 172.21.15.37:0/2537842607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c101dc0 0x7f056c108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.530+0000 7f0573661640 1 --2- 172.21.15.37:0/2537842607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f056c101400 0x7f056c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.530+0000 7f0573661640 1 --2- 172.21.15.37:0/2537842607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f056c100210 0x7f056c100630 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.530+0000 7f0573661640 1 -- 172.21.15.37:0/2537842607 >> 172.21.15.37:0/2537842607 conn(0x7f056c0fb780 msgr2=0x7f056c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.530+0000 7f0573661640 1 -- 172.21.15.37:0/2537842607 shutdown_connections 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.530+0000 7f0573661640 1 -- 172.21.15.37:0/2537842607 wait complete. 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0573661640 1 Processor -- start 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0573661640 1 -- start start 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0573661640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f056c100210 0x7f056c197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0573661640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c101400 0x7f056c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0573661640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f056c101dc0 0x7f056c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0573661640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c10be70 con 0x7f056c101dc0 2024-03-26T06:21:00.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0573661640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c10bcf0 con 0x7f056c100210 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0573661640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c10bff0 con 0x7f056c101400 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0571bfe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f056c101dc0 0x7f056c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0571bfe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f056c101dc0 0x7f056c19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46730/0 (socket says 172.21.15.37:46730) 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0571bfe640 1 -- 172.21.15.37:0/3693879749 learned_addr learned my addr 172.21.15.37:0/3693879749 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f05713fd640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f056c100210 0x7f056c197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.531+0000 7f0570bfc640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c101400 0x7f056c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.532+0000 7f0571bfe640 1 -- 172.21.15.37:0/3693879749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c101400 msgr2=0x7f056c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.532+0000 7f0571bfe640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c101400 0x7f056c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.532+0000 7f0571bfe640 1 -- 172.21.15.37:0/3693879749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f056c100210 msgr2=0x7f056c197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.532+0000 7f0571bfe640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f056c100210 0x7f056c197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.532+0000 7f0571bfe640 1 -- 172.21.15.37:0/3693879749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c19f260 con 0x7f056c101dc0 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.532+0000 7f0571bfe640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f056c101dc0 0x7f056c19ebf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f056400cce0 tx=0x7f056400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.532+0000 7f05627fc640 1 -- 172.21.15.37:0/3693879749 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056400f450 con 0x7f056c101dc0 2024-03-26T06:21:00.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.532+0000 7f0573661640 1 -- 172.21.15.37:0/3693879749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c19f550 con 0x7f056c101dc0 2024-03-26T06:21:00.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.532+0000 7f05627fc640 1 -- 172.21.15.37:0/3693879749 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f056400fad0 con 0x7f056c101dc0 2024-03-26T06:21:00.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.532+0000 7f05627fc640 1 -- 172.21.15.37:0/3693879749 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564018c10 con 0x7f056c101dc0 2024-03-26T06:21:00.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.532+0000 7f0573661640 1 -- 172.21.15.37:0/3693879749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c19faa0 con 0x7f056c101dc0 2024-03-26T06:21:00.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.533+0000 7f0573661640 1 -- 172.21.15.37:0/3693879749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c19fdc0 con 0x7f056c101dc0 2024-03-26T06:21:00.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.534+0000 7f05627fc640 1 -- 172.21.15.37:0/3693879749 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f056400f5f0 con 0x7f056c101dc0 2024-03-26T06:21:00.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.534+0000 7f05627fc640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f05580a2890 0x7f05580a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.535+0000 7f05627fc640 1 -- 172.21.15.37:0/3693879749 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f05640cf140 con 0x7f056c101dc0 2024-03-26T06:21:00.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.535+0000 7f05713fd640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f05580a2890 0x7f05580a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.535+0000 7f05713fd640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f05580a2890 0x7f05580a4d50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0568000c00 tx=0x7f0568005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:00.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.539+0000 7f05627fc640 1 -- 172.21.15.37:0/3693879749 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0564014030 con 0x7f056c101dc0 2024-03-26T06:21:00.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:00 smithi037 ceph-mon[31974]: pgmap v291: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 12 op/s 2024-03-26T06:21:00.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:00 smithi037 ceph-mon[31974]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:00.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:00 smithi037 ceph-mon[31974]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:00.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:00 smithi037 ceph-mon[31974]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T06:21:00.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:00 smithi037 ceph-mon[31974]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:00.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:00 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] up:active 2024-03-26T06:21:00.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:00 smithi037 ceph-mon[31974]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:21:00.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:00 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:21:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.768+0000 7f0573661640 1 -- 172.21.15.37:0/3693879749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f056c19ff90 con 0x7f056c101dc0 2024-03-26T06:21:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.769+0000 7f05627fc640 1 -- 172.21.15.37:0/3693879749 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11385 (secure 0 0 0) 0x7f056409c230 con 0x7f056c101dc0 2024-03-26T06:21:00.770 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:21:00.770 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T06:20:53.631953+0000","modified":"2024-03-26T06:21:00.587013+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24494,"mds_2":24470,"mds_3":24469,"mds_4":14682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14670":{"gid":14670,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/1944002831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1944002831},{"type":"v1","addr":"172.21.15.37:6839","nonce":1944002831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14682":{"gid":14682,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6841/933565564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":933565564},{"type":"v1","addr":"172.21.15.37:6841","nonce":933565564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24394":{"gid":24394,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/1420321367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1420321367},{"type":"v1","addr":"172.21.15.66:6839","nonce":1420321367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-26T06:21:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-03-26T06:21:00.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 -- 172.21.15.37:0/3693879749 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f05580a2890 msgr2=0x7f05580a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f05580a2890 0x7f05580a4d50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0568000c00 tx=0x7f0568005fd0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 -- 172.21.15.37:0/3693879749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f056c101dc0 msgr2=0x7f056c19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f056c101dc0 0x7f056c19ebf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f056400cce0 tx=0x7f056400a620 comp rx=0 tx=0).stop 2024-03-26T06:21:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 -- 172.21.15.37:0/3693879749 shutdown_connections 2024-03-26T06:21:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f05580a2890 0x7f05580a4d50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f056c101dc0 0x7f056c19ebf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f056c101400 0x7f056c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 --2- 172.21.15.37:0/3693879749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f056c100210 0x7f056c197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 -- 172.21.15.37:0/3693879749 >> 172.21.15.37:0/3693879749 conn(0x7f056c0fb780 msgr2=0x7f056c0fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 -- 172.21.15.37:0/3693879749 shutdown_connections 2024-03-26T06:21:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.771+0000 7f0573661640 1 -- 172.21.15.37:0/3693879749 wait complete. 2024-03-26T06:21:00.783 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:21:00.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.885+0000 7f4308459640 1 -- 172.21.15.37:0/2785577186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4300102500 msgr2=0x7f4300102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.885+0000 7f4308459640 1 --2- 172.21.15.37:0/2785577186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4300102500 0x7f4300102980 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009900 tx=0x7f42fc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:21:00.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 -- 172.21.15.37:0/2785577186 shutdown_connections 2024-03-26T06:21:00.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 --2- 172.21.15.37:0/2785577186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300102ec0 0x7f4300109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 --2- 172.21.15.37:0/2785577186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4300102500 0x7f4300102980 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 --2- 172.21.15.37:0/2785577186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4300101310 0x7f4300101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 -- 172.21.15.37:0/2785577186 >> 172.21.15.37:0/2785577186 conn(0x7f43000fc8a0 msgr2=0x7f43000fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:00.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 -- 172.21.15.37:0/2785577186 shutdown_connections 2024-03-26T06:21:00.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 -- 172.21.15.37:0/2785577186 wait complete. 2024-03-26T06:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 Processor -- start 2024-03-26T06:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 -- start start 2024-03-26T06:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4300101310 0x7f4300197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300102500 0x7f4300197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4300102ec0 0x7f430019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430010cfa0 con 0x7f4300102ec0 2024-03-26T06:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430010ce20 con 0x7f4300101310 2024-03-26T06:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f4308459640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430010d120 con 0x7f4300102500 2024-03-26T06:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.886+0000 7f43069f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4300102ec0 0x7f430019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f43069f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4300102ec0 0x7f430019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46750/0 (socket says 172.21.15.37:46750) 2024-03-26T06:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f43069f6640 1 -- 172.21.15.37:0/3469159667 learned_addr learned my addr 172.21.15.37:0/3469159667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f43061f5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4300101310 0x7f4300197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f43059f4640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300102500 0x7f4300197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f43069f6640 1 -- 172.21.15.37:0/3469159667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300102500 msgr2=0x7f4300197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f43069f6640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300102500 0x7f4300197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f43069f6640 1 -- 172.21.15.37:0/3469159667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4300101310 msgr2=0x7f4300197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f43069f6640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4300101310 0x7f4300197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f43069f6640 1 -- 172.21.15.37:0/3469159667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430019f240 con 0x7f4300102ec0 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f43061f5640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4300101310 0x7f4300197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f43069f6640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4300102ec0 0x7f430019ebd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f42f800cad0 tx=0x7f42f800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f42f77fe640 1 -- 172.21.15.37:0/3469159667 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800ae20 con 0x7f4300102ec0 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f42f77fe640 1 -- 172.21.15.37:0/3469159667 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42f8005020 con 0x7f4300102ec0 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f4308459640 1 -- 172.21.15.37:0/3469159667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430019f530 con 0x7f4300102ec0 2024-03-26T06:21:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f42f77fe640 1 -- 172.21.15.37:0/3469159667 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800f6f0 con 0x7f4300102ec0 2024-03-26T06:21:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.887+0000 7f4308459640 1 -- 172.21.15.37:0/3469159667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430019faa0 con 0x7f4300102ec0 2024-03-26T06:21:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.888+0000 7f4308459640 1 -- 172.21.15.37:0/3469159667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4300108630 con 0x7f4300102ec0 2024-03-26T06:21:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.889+0000 7f42f77fe640 1 -- 172.21.15.37:0/3469159667 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f42f80051d0 con 0x7f4300102ec0 2024-03-26T06:21:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.890+0000 7f42f77fe640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42ec0a2910 0x7f42ec0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.890+0000 7f43061f5640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42ec0a2910 0x7f42ec0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:00.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.890+0000 7f42f77fe640 1 -- 172.21.15.37:0/3469159667 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f42f80cec90 con 0x7f4300102ec0 2024-03-26T06:21:00.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.890+0000 7f43061f5640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42ec0a2910 0x7f42ec0a4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4300102360 tx=0x7f42f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:00.894+0000 7f42f77fe640 1 -- 172.21.15.37:0/3469159667 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f42f8014030 con 0x7f4300102ec0 2024-03-26T06:21:00.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:00 smithi149 ceph-mon[36681]: pgmap v291: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 12 op/s 2024-03-26T06:21:00.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:00 smithi149 ceph-mon[36681]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:00.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:00 smithi149 ceph-mon[36681]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:00 smithi149 ceph-mon[36681]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T06:21:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:00 smithi149 ceph-mon[36681]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:00 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] up:active 2024-03-26T06:21:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:00 smithi149 ceph-mon[36681]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:21:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:00 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:21:01.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.076+0000 7f4308459640 1 -- 172.21.15.37:0/3469159667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4300061ef0 con 0x7f4300102ec0 2024-03-26T06:21:01.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.077+0000 7f42f77fe640 1 -- 172.21.15.37:0/3469159667 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22145 (secure 0 0 0) 0x7f42f809b730 con 0x7f4300102ec0 2024-03-26T06:21:01.079 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:21:01.079 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":85,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:20:55.279252+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:21:01.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 -- 172.21.15.37:0/3469159667 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42ec0a2910 msgr2=0x7f42ec0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42ec0a2910 0x7f42ec0a4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4300102360 tx=0x7f42f0005f90 comp rx=0 tx=0).stop 2024-03-26T06:21:01.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 -- 172.21.15.37:0/3469159667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4300102ec0 msgr2=0x7f430019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4300102ec0 0x7f430019ebd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f42f800cad0 tx=0x7f42f800cf90 comp rx=0 tx=0).stop 2024-03-26T06:21:01.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 -- 172.21.15.37:0/3469159667 shutdown_connections 2024-03-26T06:21:01.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42ec0a2910 0x7f42ec0a4dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4300102ec0 0x7f430019ebd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4300102500 0x7f4300197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 --2- 172.21.15.37:0/3469159667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4300101310 0x7f4300197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 -- 172.21.15.37:0/3469159667 >> 172.21.15.37:0/3469159667 conn(0x7f43000fc8a0 msgr2=0x7f4300105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:01.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 -- 172.21.15.37:0/3469159667 shutdown_connections 2024-03-26T06:21:01.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.079+0000 7f4308459640 1 -- 172.21.15.37:0/3469159667 wait complete. 2024-03-26T06:21:01.092 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-26T06:21:01.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T06:21:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:00 smithi066 ceph-mon[36504]: pgmap v291: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 12 op/s 2024-03-26T06:21:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:00 smithi066 ceph-mon[36504]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:00 smithi066 ceph-mon[36504]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:00 smithi066 ceph-mon[36504]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T06:21:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:00 smithi066 ceph-mon[36504]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:00 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] up:active 2024-03-26T06:21:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:00 smithi066 ceph-mon[36504]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:21:01.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:00 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:21:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.193+0000 7f1a3762b640 1 -- 172.21.15.37:0/3706787651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30101310 msgr2=0x7f1a30101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.193+0000 7f1a3762b640 1 --2- 172.21.15.37:0/3706787651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30101310 0x7f1a30101730 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009900 tx=0x7f1a2802bc30 comp rx=0 tx=0).stop 2024-03-26T06:21:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.193+0000 7f1a3762b640 1 -- 172.21.15.37:0/3706787651 shutdown_connections 2024-03-26T06:21:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.193+0000 7f1a3762b640 1 --2- 172.21.15.37:0/3706787651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a30102ec0 0x7f1a30109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.193+0000 7f1a3762b640 1 --2- 172.21.15.37:0/3706787651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a30102500 0x7f1a30102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.193+0000 7f1a3762b640 1 --2- 172.21.15.37:0/3706787651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30101310 0x7f1a30101730 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.193+0000 7f1a3762b640 1 -- 172.21.15.37:0/3706787651 >> 172.21.15.37:0/3706787651 conn(0x7f1a300fc8a0 msgr2=0x7f1a300fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.193+0000 7f1a3762b640 1 -- 172.21.15.37:0/3706787651 shutdown_connections 2024-03-26T06:21:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.193+0000 7f1a3762b640 1 -- 172.21.15.37:0/3706787651 wait complete. 2024-03-26T06:21:01.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a3762b640 1 Processor -- start 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a3762b640 1 -- start start 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a3762b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a30101310 0x7f1a300750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a3762b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30102500 0x7f1a300736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a3762b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a30102ec0 0x7f1a30073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a3762b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3010cfa0 con 0x7f1a30102500 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a3762b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3010ce20 con 0x7f1a30101310 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a3762b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3010d120 con 0x7f1a30102ec0 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a34bc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30102500 0x7f1a300736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a34bc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30102500 0x7f1a300736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46770/0 (socket says 172.21.15.37:46770) 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a34bc6640 1 -- 172.21.15.37:0/2213554931 learned_addr learned my addr 172.21.15.37:0/2213554931 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a353c7640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a30101310 0x7f1a300750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:01.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.194+0000 7f1a35bc8640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a30102ec0 0x7f1a30073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a34bc6640 1 -- 172.21.15.37:0/2213554931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a30102ec0 msgr2=0x7f1a30073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a34bc6640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a30102ec0 0x7f1a30073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a34bc6640 1 -- 172.21.15.37:0/2213554931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a30101310 msgr2=0x7f1a300750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a34bc6640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a30101310 0x7f1a300750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a34bc6640 1 -- 172.21.15.37:0/2213554931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a300744b0 con 0x7f1a30102500 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a353c7640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a30101310 0x7f1a300750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a34bc6640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30102500 0x7f1a300736e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400e9f0 tx=0x7f1a2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a2e7fc640 1 -- 172.21.15.37:0/2213554931 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400cd80 con 0x7f1a30102500 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a2e7fc640 1 -- 172.21.15.37:0/2213554931 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a24010910 con 0x7f1a30102500 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a3762b640 1 -- 172.21.15.37:0/2213554931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a301a1770 con 0x7f1a30102500 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a2e7fc640 1 -- 172.21.15.37:0/2213554931 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24018990 con 0x7f1a30102500 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.195+0000 7f1a3762b640 1 -- 172.21.15.37:0/2213554931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a301a1cc0 con 0x7f1a30102500 2024-03-26T06:21:01.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.196+0000 7f1a3762b640 1 -- 172.21.15.37:0/2213554931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a30108680 con 0x7f1a30102500 2024-03-26T06:21:01.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.197+0000 7f1a2e7fc640 1 -- 172.21.15.37:0/2213554931 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f1a24010ac0 con 0x7f1a30102500 2024-03-26T06:21:01.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.197+0000 7f1a2e7fc640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a180a2890 0x7f1a180a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:01.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.197+0000 7f1a353c7640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a180a2890 0x7f1a180a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:01.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.198+0000 7f1a2e7fc640 1 -- 172.21.15.37:0/2213554931 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1a240cee20 con 0x7f1a30102500 2024-03-26T06:21:01.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.198+0000 7f1a353c7640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a180a2890 0x7f1a180a4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a28000c00 tx=0x7f1a28005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:01.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.202+0000 7f1a2e7fc640 1 -- 172.21.15.37:0/2213554931 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1a2409bf10 con 0x7f1a30102500 2024-03-26T06:21:01.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.385+0000 7f1a3762b640 1 -- 172.21.15.37:0/2213554931 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a30061730 con 0x7f1a180a2890 2024-03-26T06:21:01.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.393+0000 7f1a2e7fc640 1 -- 172.21.15.37:0/2213554931 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258744 (secure 0 0 0) 0x7f1a30061730 con 0x7f1a180a2890 2024-03-26T06:21:01.397 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:21:01.399 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-26T06:21:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 -- 172.21.15.37:0/2213554931 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a180a2890 msgr2=0x7f1a180a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a180a2890 0x7f1a180a4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a28000c00 tx=0x7f1a28005fd0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 -- 172.21.15.37:0/2213554931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30102500 msgr2=0x7f1a300736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30102500 0x7f1a300736e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400e9f0 tx=0x7f1a2400eeb0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 -- 172.21.15.37:0/2213554931 shutdown_connections 2024-03-26T06:21:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1a180a2890 0x7f1a180a4d50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a30102ec0 0x7f1a30073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30102500 0x7f1a300736e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 --2- 172.21.15.37:0/2213554931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a30101310 0x7f1a300750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 -- 172.21.15.37:0/2213554931 >> 172.21.15.37:0/2213554931 conn(0x7f1a300fc8a0 msgr2=0x7f1a30105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:01.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 -- 172.21.15.37:0/2213554931 shutdown_connections 2024-03-26T06:21:01.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.396+0000 7f1a3762b640 1 -- 172.21.15.37:0/2213554931 wait complete. 2024-03-26T06:21:01.408 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-03-26T06:21:00.746155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594878,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":144,"num_read_kb":127,"num_write":291,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":221,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365968,"kb_used_data":12588,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575856,"statfs":{"total":1151940427776,"available":1151565676544,"internally_reserved":0,"allocated":12890112,"data_stored":7018780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4510,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":110,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.477874"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082986+0000","last_change":"2024-03-26T06:20:54.294436+0000","last_active":"2024-03-26T06:20:56.082986+0000","last_peered":"2024-03-26T06:20:56.082986+0000","last_clean":"2024-03-26T06:20:56.082986+0000","last_became_active":"2024-03-26T06:20:54.293049+0000","last_became_peered":"2024-03-26T06:20:54.293049+0000","last_unstale":"2024-03-26T06:20:56.082986+0000","last_undegraded":"2024-03-26T06:20:56.082986+0000","last_fullsized":"2024-03-26T06:20:56.082986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:44:02.729046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080469+0000","last_change":"2024-03-26T06:20:54.292560+0000","last_active":"2024-03-26T06:20:56.080469+0000","last_peered":"2024-03-26T06:20:56.080469+0000","last_clean":"2024-03-26T06:20:56.080469+0000","last_became_active":"2024-03-26T06:20:54.292166+0000","last_became_peered":"2024-03-26T06:20:54.292166+0000","last_unstale":"2024-03-26T06:20:56.080469+0000","last_undegraded":"2024-03-26T06:20:56.080469+0000","last_fullsized":"2024-03-26T06:20:56.080469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:45.255912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080688+0000","last_change":"2024-03-26T06:20:54.294350+0000","last_active":"2024-03-26T06:20:56.080688+0000","last_peered":"2024-03-26T06:20:56.080688+0000","last_clean":"2024-03-26T06:20:56.080688+0000","last_became_active":"2024-03-26T06:20:54.293813+0000","last_became_peered":"2024-03-26T06:20:54.293813+0000","last_unstale":"2024-03-26T06:20:56.080688+0000","last_undegraded":"2024-03-26T06:20:56.080688+0000","last_fullsized":"2024-03-26T06:20:56.080688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:44:31.126828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081978+0000","last_change":"2024-03-26T06:20:54.299151+0000","last_active":"2024-03-26T06:20:56.081978+0000","last_peered":"2024-03-26T06:20:56.081978+0000","last_clean":"2024-03-26T06:20:56.081978+0000","last_became_active":"2024-03-26T06:20:54.298820+0000","last_became_peered":"2024-03-26T06:20:54.298820+0000","last_unstale":"2024-03-26T06:20:56.081978+0000","last_undegraded":"2024-03-26T06:20:56.081978+0000","last_fullsized":"2024-03-26T06:20:56.081978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:47:25.503692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082042+0000","last_change":"2024-03-26T06:20:54.299977+0000","last_active":"2024-03-26T06:20:56.082042+0000","last_peered":"2024-03-26T06:20:56.082042+0000","last_clean":"2024-03-26T06:20:56.082042+0000","last_became_active":"2024-03-26T06:20:54.299641+0000","last_became_peered":"2024-03-26T06:20:54.299641+0000","last_unstale":"2024-03-26T06:20:56.082042+0000","last_undegraded":"2024-03-26T06:20:56.082042+0000","last_fullsized":"2024-03-26T06:20:56.082042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:42:32.478970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082317+0000","last_change":"2024-03-26T06:20:54.294308+0000","last_active":"2024-03-26T06:20:56.082317+0000","last_peered":"2024-03-26T06:20:56.082317+0000","last_clean":"2024-03-26T06:20:56.082317+0000","last_became_active":"2024-03-26T06:20:54.293882+0000","last_became_peered":"2024-03-26T06:20:54.293882+0000","last_unstale":"2024-03-26T06:20:56.082317+0000","last_undegraded":"2024-03-26T06:20:56.082317+0000","last_fullsized":"2024-03-26T06:20:56.082317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:26:26.014097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235824+0000","last_change":"2024-03-26T06:20:54.301644+0000","last_active":"2024-03-26T06:20:56.235824+0000","last_peered":"2024-03-26T06:20:56.235824+0000","last_clean":"2024-03-26T06:20:56.235824+0000","last_became_active":"2024-03-26T06:20:54.301257+0000","last_became_peered":"2024-03-26T06:20:54.301257+0000","last_unstale":"2024-03-26T06:20:56.235824+0000","last_undegraded":"2024-03-26T06:20:56.235824+0000","last_fullsized":"2024-03-26T06:20:56.235824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:43:27.602288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081228+0000","last_change":"2024-03-26T06:20:54.296421+0000","last_active":"2024-03-26T06:20:56.081228+0000","last_peered":"2024-03-26T06:20:56.081228+0000","last_clean":"2024-03-26T06:20:56.081228+0000","last_became_active":"2024-03-26T06:20:54.296086+0000","last_became_peered":"2024-03-26T06:20:54.296086+0000","last_unstale":"2024-03-26T06:20:56.081228+0000","last_undegraded":"2024-03-26T06:20:56.081228+0000","last_fullsized":"2024-03-26T06:20:56.081228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:22:25.532160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289603+0000","last_change":"2024-03-26T06:20:54.297206+0000","last_active":"2024-03-26T06:20:55.289603+0000","last_peered":"2024-03-26T06:20:55.289603+0000","last_clean":"2024-03-26T06:20:55.289603+0000","last_became_active":"2024-03-26T06:20:54.296849+0000","last_became_peered":"2024-03-26T06:20:54.296849+0000","last_unstale":"2024-03-26T06:20:55.289603+0000","last_undegraded":"2024-03-26T06:20:55.289603+0000","last_fullsized":"2024-03-26T06:20:55.289603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:24:49.972237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082381+0000","last_change":"2024-03-26T06:20:54.295048+0000","last_active":"2024-03-26T06:20:56.082381+0000","last_peered":"2024-03-26T06:20:56.082381+0000","last_clean":"2024-03-26T06:20:56.082381+0000","last_became_active":"2024-03-26T06:20:54.294161+0000","last_became_peered":"2024-03-26T06:20:54.294161+0000","last_unstale":"2024-03-26T06:20:56.082381+0000","last_undegraded":"2024-03-26T06:20:56.082381+0000","last_fullsized":"2024-03-26T06:20:56.082381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:41:16.779030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235997+0000","last_change":"2024-03-26T06:20:54.300235+0000","last_active":"2024-03-26T06:20:56.235997+0000","last_peered":"2024-03-26T06:20:56.235997+0000","last_clean":"2024-03-26T06:20:56.235997+0000","last_became_active":"2024-03-26T06:20:54.299677+0000","last_became_peered":"2024-03-26T06:20:54.299677+0000","last_unstale":"2024-03-26T06:20:56.235997+0000","last_undegraded":"2024-03-26T06:20:56.235997+0000","last_fullsized":"2024-03-26T06:20:56.235997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:18:57.028840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081952+0000","last_change":"2024-03-26T06:20:54.303953+0000","last_active":"2024-03-26T06:20:56.081952+0000","last_peered":"2024-03-26T06:20:56.081952+0000","last_clean":"2024-03-26T06:20:56.081952+0000","last_became_active":"2024-03-26T06:20:54.303339+0000","last_became_peered":"2024-03-26T06:20:54.303339+0000","last_unstale":"2024-03-26T06:20:56.081952+0000","last_undegraded":"2024-03-26T06:20:56.081952+0000","last_fullsized":"2024-03-26T06:20:56.081952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:54:38.511221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289015+0000","last_change":"2024-03-26T06:20:54.283535+0000","last_active":"2024-03-26T06:20:55.289015+0000","last_peered":"2024-03-26T06:20:55.289015+0000","last_clean":"2024-03-26T06:20:55.289015+0000","last_became_active":"2024-03-26T06:20:54.282820+0000","last_became_peered":"2024-03-26T06:20:54.282820+0000","last_unstale":"2024-03-26T06:20:55.289015+0000","last_undegraded":"2024-03-26T06:20:55.289015+0000","last_fullsized":"2024-03-26T06:20:55.289015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:13:07.906341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080505+0000","last_change":"2024-03-26T06:20:54.292841+0000","last_active":"2024-03-26T06:20:56.080505+0000","last_peered":"2024-03-26T06:20:56.080505+0000","last_clean":"2024-03-26T06:20:56.080505+0000","last_became_active":"2024-03-26T06:20:54.292530+0000","last_became_peered":"2024-03-26T06:20:54.292530+0000","last_unstale":"2024-03-26T06:20:56.080505+0000","last_undegraded":"2024-03-26T06:20:56.080505+0000","last_fullsized":"2024-03-26T06:20:56.080505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:35:54.592898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080252+0000","last_change":"2024-03-26T06:20:54.306753+0000","last_active":"2024-03-26T06:20:56.080252+0000","last_peered":"2024-03-26T06:20:56.080252+0000","last_clean":"2024-03-26T06:20:56.080252+0000","last_became_active":"2024-03-26T06:20:54.306379+0000","last_became_peered":"2024-03-26T06:20:54.306379+0000","last_unstale":"2024-03-26T06:20:56.080252+0000","last_undegraded":"2024-03-26T06:20:56.080252+0000","last_fullsized":"2024-03-26T06:20:56.080252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:04:19.879733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288996+0000","last_change":"2024-03-26T06:20:54.305719+0000","last_active":"2024-03-26T06:20:55.288996+0000","last_peered":"2024-03-26T06:20:55.288996+0000","last_clean":"2024-03-26T06:20:55.288996+0000","last_became_active":"2024-03-26T06:20:54.305336+0000","last_became_peered":"2024-03-26T06:20:54.305336+0000","last_unstale":"2024-03-26T06:20:55.288996+0000","last_undegraded":"2024-03-26T06:20:55.288996+0000","last_fullsized":"2024-03-26T06:20:55.288996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:22:15.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.083044+0000","last_change":"2024-03-26T06:20:54.293451+0000","last_active":"2024-03-26T06:20:56.083044+0000","last_peered":"2024-03-26T06:20:56.083044+0000","last_clean":"2024-03-26T06:20:56.083044+0000","last_became_active":"2024-03-26T06:20:54.293139+0000","last_became_peered":"2024-03-26T06:20:54.293139+0000","last_unstale":"2024-03-26T06:20:56.083044+0000","last_undegraded":"2024-03-26T06:20:56.083044+0000","last_fullsized":"2024-03-26T06:20:56.083044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:56:04.407687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081187+0000","last_change":"2024-03-26T06:20:54.290256+0000","last_active":"2024-03-26T06:20:56.081187+0000","last_peered":"2024-03-26T06:20:56.081187+0000","last_clean":"2024-03-26T06:20:56.081187+0000","last_became_active":"2024-03-26T06:20:54.289773+0000","last_became_peered":"2024-03-26T06:20:54.289773+0000","last_unstale":"2024-03-26T06:20:56.081187+0000","last_undegraded":"2024-03-26T06:20:56.081187+0000","last_fullsized":"2024-03-26T06:20:56.081187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:46:07.503946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293543+0000","last_change":"2024-03-26T06:20:54.302601+0000","last_active":"2024-03-26T06:20:55.293543+0000","last_peered":"2024-03-26T06:20:55.293543+0000","last_clean":"2024-03-26T06:20:55.293543+0000","last_became_active":"2024-03-26T06:20:54.301523+0000","last_became_peered":"2024-03-26T06:20:54.301523+0000","last_unstale":"2024-03-26T06:20:55.293543+0000","last_undegraded":"2024-03-26T06:20:55.293543+0000","last_fullsized":"2024-03-26T06:20:55.293543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:58:49.055958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:57.616517+0000","last_change":"2024-03-26T06:20:53.280275+0000","last_active":"2024-03-26T06:20:57.616517+0000","last_peered":"2024-03-26T06:20:57.616517+0000","last_clean":"2024-03-26T06:20:57.616517+0000","last_became_active":"2024-03-26T06:20:53.279916+0000","last_became_peered":"2024-03-26T06:20:53.279916+0000","last_unstale":"2024-03-26T06:20:57.616517+0000","last_undegraded":"2024-03-26T06:20:57.616517+0000","last_fullsized":"2024-03-26T06:20:57.616517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:50.844507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081204+0000","last_change":"2024-03-26T06:20:54.302720+0000","last_active":"2024-03-26T06:20:56.081204+0000","last_peered":"2024-03-26T06:20:56.081204+0000","last_clean":"2024-03-26T06:20:56.081204+0000","last_became_active":"2024-03-26T06:20:54.302171+0000","last_became_peered":"2024-03-26T06:20:54.302171+0000","last_unstale":"2024-03-26T06:20:56.081204+0000","last_undegraded":"2024-03-26T06:20:56.081204+0000","last_fullsized":"2024-03-26T06:20:56.081204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:23:12.887883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082078+0000","last_change":"2024-03-26T06:20:53.282355+0000","last_active":"2024-03-26T06:20:56.082078+0000","last_peered":"2024-03-26T06:20:56.082078+0000","last_clean":"2024-03-26T06:20:56.082078+0000","last_became_active":"2024-03-26T06:20:53.282062+0000","last_became_peered":"2024-03-26T06:20:53.282062+0000","last_unstale":"2024-03-26T06:20:56.082078+0000","last_undegraded":"2024-03-26T06:20:56.082078+0000","last_fullsized":"2024-03-26T06:20:56.082078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:58:33.398589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081056+0000","last_change":"2024-03-26T06:20:54.294924+0000","last_active":"2024-03-26T06:20:56.081056+0000","last_peered":"2024-03-26T06:20:56.081056+0000","last_clean":"2024-03-26T06:20:56.081056+0000","last_became_active":"2024-03-26T06:20:54.293971+0000","last_became_peered":"2024-03-26T06:20:54.293971+0000","last_unstale":"2024-03-26T06:20:56.081056+0000","last_undegraded":"2024-03-26T06:20:56.081056+0000","last_fullsized":"2024-03-26T06:20:56.081056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:54:15.457497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:59.594629+0000","last_change":"2024-03-26T06:20:53.286284+0000","last_active":"2024-03-26T06:20:59.594629+0000","last_peered":"2024-03-26T06:20:59.594629+0000","last_clean":"2024-03-26T06:20:59.594629+0000","last_became_active":"2024-03-26T06:20:53.285979+0000","last_became_peered":"2024-03-26T06:20:53.285979+0000","last_unstale":"2024-03-26T06:20:59.594629+0000","last_undegraded":"2024-03-26T06:20:59.594629+0000","last_fullsized":"2024-03-26T06:20:59.594629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:33:31.989055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289440+0000","last_change":"2024-03-26T06:20:54.290137+0000","last_active":"2024-03-26T06:20:55.289440+0000","last_peered":"2024-03-26T06:20:55.289440+0000","last_clean":"2024-03-26T06:20:55.289440+0000","last_became_active":"2024-03-26T06:20:54.289727+0000","last_became_peered":"2024-03-26T06:20:54.289727+0000","last_unstale":"2024-03-26T06:20:55.289440+0000","last_undegraded":"2024-03-26T06:20:55.289440+0000","last_fullsized":"2024-03-26T06:20:55.289440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:31:26.447358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:59.593998+0000","last_change":"2024-03-26T06:20:53.283774+0000","last_active":"2024-03-26T06:20:59.593998+0000","last_peered":"2024-03-26T06:20:59.593998+0000","last_clean":"2024-03-26T06:20:59.593998+0000","last_became_active":"2024-03-26T06:20:53.283430+0000","last_became_peered":"2024-03-26T06:20:53.283430+0000","last_unstale":"2024-03-26T06:20:59.593998+0000","last_undegraded":"2024-03-26T06:20:59.593998+0000","last_fullsized":"2024-03-26T06:20:59.593998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:33:34.747419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082152+0000","last_change":"2024-03-26T06:20:54.293961+0000","last_active":"2024-03-26T06:20:56.082152+0000","last_peered":"2024-03-26T06:20:56.082152+0000","last_clean":"2024-03-26T06:20:56.082152+0000","last_became_active":"2024-03-26T06:20:54.293602+0000","last_became_peered":"2024-03-26T06:20:54.293602+0000","last_unstale":"2024-03-26T06:20:56.082152+0000","last_undegraded":"2024-03-26T06:20:56.082152+0000","last_fullsized":"2024-03-26T06:20:56.082152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:21:08.995710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.605973+0000","last_change":"2024-03-26T06:20:53.287298+0000","last_active":"2024-03-26T06:20:56.605973+0000","last_peered":"2024-03-26T06:20:56.605973+0000","last_clean":"2024-03-26T06:20:56.605973+0000","last_became_active":"2024-03-26T06:20:53.286987+0000","last_became_peered":"2024-03-26T06:20:53.286987+0000","last_unstale":"2024-03-26T06:20:56.605973+0000","last_undegraded":"2024-03-26T06:20:56.605973+0000","last_fullsized":"2024-03-26T06:20:56.605973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:20.343750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081504+0000","last_change":"2024-03-26T06:20:54.291410+0000","last_active":"2024-03-26T06:20:56.081504+0000","last_peered":"2024-03-26T06:20:56.081504+0000","last_clean":"2024-03-26T06:20:56.081504+0000","last_became_active":"2024-03-26T06:20:54.291085+0000","last_became_peered":"2024-03-26T06:20:54.291085+0000","last_unstale":"2024-03-26T06:20:56.081504+0000","last_undegraded":"2024-03-26T06:20:56.081504+0000","last_fullsized":"2024-03-26T06:20:56.081504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:39.727359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:57.617246+0000","last_change":"2024-03-26T06:20:53.285476+0000","last_active":"2024-03-26T06:20:57.617246+0000","last_peered":"2024-03-26T06:20:57.617246+0000","last_clean":"2024-03-26T06:20:57.617246+0000","last_became_active":"2024-03-26T06:20:53.285009+0000","last_became_peered":"2024-03-26T06:20:53.285009+0000","last_unstale":"2024-03-26T06:20:57.617246+0000","last_undegraded":"2024-03-26T06:20:57.617246+0000","last_fullsized":"2024-03-26T06:20:57.617246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:40.006117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081879+0000","last_change":"2024-03-26T06:20:54.299909+0000","last_active":"2024-03-26T06:20:56.081879+0000","last_peered":"2024-03-26T06:20:56.081879+0000","last_clean":"2024-03-26T06:20:56.081879+0000","last_became_active":"2024-03-26T06:20:54.299582+0000","last_became_peered":"2024-03-26T06:20:54.299582+0000","last_unstale":"2024-03-26T06:20:56.081879+0000","last_undegraded":"2024-03-26T06:20:56.081879+0000","last_fullsized":"2024-03-26T06:20:56.081879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:16:41.110029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288640+0000","last_change":"2024-03-26T06:20:53.281619+0000","last_active":"2024-03-26T06:20:55.288640+0000","last_peered":"2024-03-26T06:20:55.288640+0000","last_clean":"2024-03-26T06:20:55.288640+0000","last_became_active":"2024-03-26T06:20:53.279792+0000","last_became_peered":"2024-03-26T06:20:53.279792+0000","last_unstale":"2024-03-26T06:20:55.288640+0000","last_undegraded":"2024-03-26T06:20:55.288640+0000","last_fullsized":"2024-03-26T06:20:55.288640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:17:17.892132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081457+0000","last_change":"2024-03-26T06:20:54.293418+0000","last_active":"2024-03-26T06:20:56.081457+0000","last_peered":"2024-03-26T06:20:56.081457+0000","last_clean":"2024-03-26T06:20:56.081457+0000","last_became_active":"2024-03-26T06:20:54.293109+0000","last_became_peered":"2024-03-26T06:20:54.293109+0000","last_unstale":"2024-03-26T06:20:56.081457+0000","last_undegraded":"2024-03-26T06:20:56.081457+0000","last_fullsized":"2024-03-26T06:20:56.081457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:55:13.764552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.602212+0000","last_change":"2024-03-26T06:20:53.285367+0000","last_active":"2024-03-26T06:20:56.602212+0000","last_peered":"2024-03-26T06:20:56.602212+0000","last_clean":"2024-03-26T06:20:56.602212+0000","last_became_active":"2024-03-26T06:20:53.285022+0000","last_became_peered":"2024-03-26T06:20:53.285022+0000","last_unstale":"2024-03-26T06:20:56.602212+0000","last_undegraded":"2024-03-26T06:20:56.602212+0000","last_fullsized":"2024-03-26T06:20:56.602212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:01:32.288961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081059+0000","last_change":"2024-03-26T06:20:54.290125+0000","last_active":"2024-03-26T06:20:56.081059+0000","last_peered":"2024-03-26T06:20:56.081059+0000","last_clean":"2024-03-26T06:20:56.081059+0000","last_became_active":"2024-03-26T06:20:54.289739+0000","last_became_peered":"2024-03-26T06:20:54.289739+0000","last_unstale":"2024-03-26T06:20:56.081059+0000","last_undegraded":"2024-03-26T06:20:56.081059+0000","last_fullsized":"2024-03-26T06:20:56.081059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:51:01.087526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.602850+0000","last_change":"2024-03-26T06:20:53.289898+0000","last_active":"2024-03-26T06:20:56.602850+0000","last_peered":"2024-03-26T06:20:56.602850+0000","last_clean":"2024-03-26T06:20:56.602850+0000","last_became_active":"2024-03-26T06:20:53.289592+0000","last_became_peered":"2024-03-26T06:20:53.289592+0000","last_unstale":"2024-03-26T06:20:56.602850+0000","last_undegraded":"2024-03-26T06:20:56.602850+0000","last_fullsized":"2024-03-26T06:20:56.602850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:11:41.249408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289546+0000","last_change":"2024-03-26T06:20:54.290970+0000","last_active":"2024-03-26T06:20:55.289546+0000","last_peered":"2024-03-26T06:20:55.289546+0000","last_clean":"2024-03-26T06:20:55.289546+0000","last_became_active":"2024-03-26T06:20:54.289686+0000","last_became_peered":"2024-03-26T06:20:54.289686+0000","last_unstale":"2024-03-26T06:20:55.289546+0000","last_undegraded":"2024-03-26T06:20:55.289546+0000","last_fullsized":"2024-03-26T06:20:55.289546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:39:42.552786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.605097+0000","last_change":"2024-03-26T06:20:53.279247+0000","last_active":"2024-03-26T06:20:56.605097+0000","last_peered":"2024-03-26T06:20:56.605097+0000","last_clean":"2024-03-26T06:20:56.605097+0000","last_became_active":"2024-03-26T06:20:53.278896+0000","last_became_peered":"2024-03-26T06:20:53.278896+0000","last_unstale":"2024-03-26T06:20:56.605097+0000","last_undegraded":"2024-03-26T06:20:56.605097+0000","last_fullsized":"2024-03-26T06:20:56.605097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:53:27.831637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080871+0000","last_change":"2024-03-26T06:20:54.299129+0000","last_active":"2024-03-26T06:20:56.080871+0000","last_peered":"2024-03-26T06:20:56.080871+0000","last_clean":"2024-03-26T06:20:56.080871+0000","last_became_active":"2024-03-26T06:20:54.298777+0000","last_became_peered":"2024-03-26T06:20:54.298777+0000","last_unstale":"2024-03-26T06:20:56.080871+0000","last_undegraded":"2024-03-26T06:20:56.080871+0000","last_fullsized":"2024-03-26T06:20:56.080871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:29:38.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.603979+0000","last_change":"2024-03-26T06:20:53.284988+0000","last_active":"2024-03-26T06:20:56.603979+0000","last_peered":"2024-03-26T06:20:56.603979+0000","last_clean":"2024-03-26T06:20:56.603979+0000","last_became_active":"2024-03-26T06:20:53.284684+0000","last_became_peered":"2024-03-26T06:20:53.284684+0000","last_unstale":"2024-03-26T06:20:56.603979+0000","last_undegraded":"2024-03-26T06:20:56.603979+0000","last_fullsized":"2024-03-26T06:20:56.603979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:31:45.004042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289450+0000","last_change":"2024-03-26T06:20:54.307147+0000","last_active":"2024-03-26T06:20:55.289450+0000","last_peered":"2024-03-26T06:20:55.289450+0000","last_clean":"2024-03-26T06:20:55.289450+0000","last_became_active":"2024-03-26T06:20:54.306797+0000","last_became_peered":"2024-03-26T06:20:54.306797+0000","last_unstale":"2024-03-26T06:20:55.289450+0000","last_undegraded":"2024-03-26T06:20:55.289450+0000","last_fullsized":"2024-03-26T06:20:55.289450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:20:58.904884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.603293+0000","last_change":"2024-03-26T06:20:53.284577+0000","last_active":"2024-03-26T06:20:56.603293+0000","last_peered":"2024-03-26T06:20:56.603293+0000","last_clean":"2024-03-26T06:20:56.603293+0000","last_became_active":"2024-03-26T06:20:53.284161+0000","last_became_peered":"2024-03-26T06:20:53.284161+0000","last_unstale":"2024-03-26T06:20:56.603293+0000","last_undegraded":"2024-03-26T06:20:56.603293+0000","last_fullsized":"2024-03-26T06:20:56.603293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:57.765950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081398+0000","last_change":"2024-03-26T06:20:54.301794+0000","last_active":"2024-03-26T06:20:56.081398+0000","last_peered":"2024-03-26T06:20:56.081398+0000","last_clean":"2024-03-26T06:20:56.081398+0000","last_became_active":"2024-03-26T06:20:54.301445+0000","last_became_peered":"2024-03-26T06:20:54.301445+0000","last_unstale":"2024-03-26T06:20:56.081398+0000","last_undegraded":"2024-03-26T06:20:56.081398+0000","last_fullsized":"2024-03-26T06:20:56.081398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:32:18.280992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293225+0000","last_change":"2024-03-26T06:20:53.282394+0000","last_active":"2024-03-26T06:20:55.293225+0000","last_peered":"2024-03-26T06:20:55.293225+0000","last_clean":"2024-03-26T06:20:55.293225+0000","last_became_active":"2024-03-26T06:20:53.282068+0000","last_became_peered":"2024-03-26T06:20:53.282068+0000","last_unstale":"2024-03-26T06:20:55.293225+0000","last_undegraded":"2024-03-26T06:20:55.293225+0000","last_fullsized":"2024-03-26T06:20:55.293225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:04:00.503682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288758+0000","last_change":"2024-03-26T06:20:54.303294+0000","last_active":"2024-03-26T06:20:55.288758+0000","last_peered":"2024-03-26T06:20:55.288758+0000","last_clean":"2024-03-26T06:20:55.288758+0000","last_became_active":"2024-03-26T06:20:54.301342+0000","last_became_peered":"2024-03-26T06:20:54.301342+0000","last_unstale":"2024-03-26T06:20:55.288758+0000","last_undegraded":"2024-03-26T06:20:55.288758+0000","last_fullsized":"2024-03-26T06:20:55.288758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:53:51.012761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.603651+0000","last_change":"2024-03-26T06:20:53.286119+0000","last_active":"2024-03-26T06:20:56.603651+0000","last_peered":"2024-03-26T06:20:56.603651+0000","last_clean":"2024-03-26T06:20:56.603651+0000","last_became_active":"2024-03-26T06:20:53.285784+0000","last_became_peered":"2024-03-26T06:20:53.285784+0000","last_unstale":"2024-03-26T06:20:56.603651+0000","last_undegraded":"2024-03-26T06:20:56.603651+0000","last_fullsized":"2024-03-26T06:20:56.603651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:54:36.786966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.236156+0000","last_change":"2024-03-26T06:20:54.309706+0000","last_active":"2024-03-26T06:20:56.236156+0000","last_peered":"2024-03-26T06:20:56.236156+0000","last_clean":"2024-03-26T06:20:56.236156+0000","last_became_active":"2024-03-26T06:20:54.309291+0000","last_became_peered":"2024-03-26T06:20:54.309291+0000","last_unstale":"2024-03-26T06:20:56.236156+0000","last_undegraded":"2024-03-26T06:20:56.236156+0000","last_fullsized":"2024-03-26T06:20:56.236156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:51:52.150636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082182+0000","last_change":"2024-03-26T06:20:53.278569+0000","last_active":"2024-03-26T06:20:56.082182+0000","last_peered":"2024-03-26T06:20:56.082182+0000","last_clean":"2024-03-26T06:20:56.082182+0000","last_became_active":"2024-03-26T06:20:53.278207+0000","last_became_peered":"2024-03-26T06:20:53.278207+0000","last_unstale":"2024-03-26T06:20:56.082182+0000","last_undegraded":"2024-03-26T06:20:56.082182+0000","last_fullsized":"2024-03-26T06:20:56.082182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:59:12.345429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235604+0000","last_change":"2024-03-26T06:20:54.308325+0000","last_active":"2024-03-26T06:20:56.235604+0000","last_peered":"2024-03-26T06:20:56.235604+0000","last_clean":"2024-03-26T06:20:56.235604+0000","last_became_active":"2024-03-26T06:20:54.307970+0000","last_became_peered":"2024-03-26T06:20:54.307970+0000","last_unstale":"2024-03-26T06:20:56.235604+0000","last_undegraded":"2024-03-26T06:20:56.235604+0000","last_fullsized":"2024-03-26T06:20:56.235604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:18:26.597073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:59.581783+0000","last_change":"2024-03-26T06:20:53.284675+0000","last_active":"2024-03-26T06:20:59.581783+0000","last_peered":"2024-03-26T06:20:59.581783+0000","last_clean":"2024-03-26T06:20:59.581783+0000","last_became_active":"2024-03-26T06:20:53.284368+0000","last_became_peered":"2024-03-26T06:20:53.284368+0000","last_unstale":"2024-03-26T06:20:59.581783+0000","last_undegraded":"2024-03-26T06:20:59.581783+0000","last_fullsized":"2024-03-26T06:20:59.581783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:15:51.733268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081147+0000","last_change":"2024-03-26T06:20:54.296348+0000","last_active":"2024-03-26T06:20:56.081147+0000","last_peered":"2024-03-26T06:20:56.081147+0000","last_clean":"2024-03-26T06:20:56.081147+0000","last_became_active":"2024-03-26T06:20:54.296019+0000","last_became_peered":"2024-03-26T06:20:54.296019+0000","last_unstale":"2024-03-26T06:20:56.081147+0000","last_undegraded":"2024-03-26T06:20:56.081147+0000","last_fullsized":"2024-03-26T06:20:56.081147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:25:28.498887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:58.610803+0000","last_change":"2024-03-26T06:20:53.281314+0000","last_active":"2024-03-26T06:20:58.610803+0000","last_peered":"2024-03-26T06:20:58.610803+0000","last_clean":"2024-03-26T06:20:58.610803+0000","last_became_active":"2024-03-26T06:20:53.281007+0000","last_became_peered":"2024-03-26T06:20:53.281007+0000","last_unstale":"2024-03-26T06:20:58.610803+0000","last_undegraded":"2024-03-26T06:20:58.610803+0000","last_fullsized":"2024-03-26T06:20:58.610803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:39:25.840470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082652+0000","last_change":"2024-03-26T06:20:54.290011+0000","last_active":"2024-03-26T06:20:56.082652+0000","last_peered":"2024-03-26T06:20:56.082652+0000","last_clean":"2024-03-26T06:20:56.082652+0000","last_became_active":"2024-03-26T06:20:54.289492+0000","last_became_peered":"2024-03-26T06:20:54.289492+0000","last_unstale":"2024-03-26T06:20:56.082652+0000","last_undegraded":"2024-03-26T06:20:56.082652+0000","last_fullsized":"2024-03-26T06:20:56.082652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:39:16.960414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080900+0000","last_change":"2024-03-26T06:20:53.276374+0000","last_active":"2024-03-26T06:20:56.080900+0000","last_peered":"2024-03-26T06:20:56.080900+0000","last_clean":"2024-03-26T06:20:56.080900+0000","last_became_active":"2024-03-26T06:20:53.276031+0000","last_became_peered":"2024-03-26T06:20:53.276031+0000","last_unstale":"2024-03-26T06:20:56.080900+0000","last_undegraded":"2024-03-26T06:20:56.080900+0000","last_fullsized":"2024-03-26T06:20:56.080900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:14.601841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082140+0000","last_change":"2024-03-26T06:20:54.304279+0000","last_active":"2024-03-26T06:20:56.082140+0000","last_peered":"2024-03-26T06:20:56.082140+0000","last_clean":"2024-03-26T06:20:56.082140+0000","last_became_active":"2024-03-26T06:20:54.303859+0000","last_became_peered":"2024-03-26T06:20:54.303859+0000","last_unstale":"2024-03-26T06:20:56.082140+0000","last_undegraded":"2024-03-26T06:20:56.082140+0000","last_fullsized":"2024-03-26T06:20:56.082140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:32:37.969077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235618+0000","last_change":"2024-03-26T06:20:53.286038+0000","last_active":"2024-03-26T06:20:56.235618+0000","last_peered":"2024-03-26T06:20:56.235618+0000","last_clean":"2024-03-26T06:20:56.235618+0000","last_became_active":"2024-03-26T06:20:53.285353+0000","last_became_peered":"2024-03-26T06:20:53.285353+0000","last_unstale":"2024-03-26T06:20:56.235618+0000","last_undegraded":"2024-03-26T06:20:56.235618+0000","last_fullsized":"2024-03-26T06:20:56.235618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:43:53.970393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081851+0000","last_change":"2024-03-26T06:20:54.294425+0000","last_active":"2024-03-26T06:20:56.081851+0000","last_peered":"2024-03-26T06:20:56.081851+0000","last_clean":"2024-03-26T06:20:56.081851+0000","last_became_active":"2024-03-26T06:20:54.293607+0000","last_became_peered":"2024-03-26T06:20:54.293607+0000","last_unstale":"2024-03-26T06:20:56.081851+0000","last_undegraded":"2024-03-26T06:20:56.081851+0000","last_fullsized":"2024-03-26T06:20:56.081851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:12:49.118427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081642+0000","last_change":"2024-03-26T06:20:54.291038+0000","last_active":"2024-03-26T06:20:56.081642+0000","last_peered":"2024-03-26T06:20:56.081642+0000","last_clean":"2024-03-26T06:20:56.081642+0000","last_became_active":"2024-03-26T06:20:54.290678+0000","last_became_peered":"2024-03-26T06:20:54.290678+0000","last_unstale":"2024-03-26T06:20:56.081642+0000","last_undegraded":"2024-03-26T06:20:56.081642+0000","last_fullsized":"2024-03-26T06:20:56.081642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:29:30.528408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288580+0000","last_change":"2024-03-26T06:20:53.288795+0000","last_active":"2024-03-26T06:20:55.288580+0000","last_peered":"2024-03-26T06:20:55.288580+0000","last_clean":"2024-03-26T06:20:55.288580+0000","last_became_active":"2024-03-26T06:20:53.288468+0000","last_became_peered":"2024-03-26T06:20:53.288468+0000","last_unstale":"2024-03-26T06:20:55.288580+0000","last_undegraded":"2024-03-26T06:20:55.288580+0000","last_fullsized":"2024-03-26T06:20:55.288580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:16:05.569407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081864+0000","last_change":"2024-03-26T06:20:54.296706+0000","last_active":"2024-03-26T06:20:56.081864+0000","last_peered":"2024-03-26T06:20:56.081864+0000","last_clean":"2024-03-26T06:20:56.081864+0000","last_became_active":"2024-03-26T06:20:54.296387+0000","last_became_peered":"2024-03-26T06:20:54.296387+0000","last_unstale":"2024-03-26T06:20:56.081864+0000","last_undegraded":"2024-03-26T06:20:56.081864+0000","last_fullsized":"2024-03-26T06:20:56.081864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:23:23.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082176+0000","last_change":"2024-03-26T06:20:53.282790+0000","last_active":"2024-03-26T06:20:56.082176+0000","last_peered":"2024-03-26T06:20:56.082176+0000","last_clean":"2024-03-26T06:20:56.082176+0000","last_became_active":"2024-03-26T06:20:53.282451+0000","last_became_peered":"2024-03-26T06:20:53.282451+0000","last_unstale":"2024-03-26T06:20:56.082176+0000","last_undegraded":"2024-03-26T06:20:56.082176+0000","last_fullsized":"2024-03-26T06:20:56.082176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:47:56.328354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235480+0000","last_change":"2024-03-26T06:19:42.185355+0000","last_active":"2024-03-26T06:20:56.235480+0000","last_peered":"2024-03-26T06:20:56.235480+0000","last_clean":"2024-03-26T06:20:56.235480+0000","last_became_active":"2024-03-26T06:19:42.184672+0000","last_became_peered":"2024-03-26T06:19:42.184672+0000","last_unstale":"2024-03-26T06:20:56.235480+0000","last_undegraded":"2024-03-26T06:20:56.235480+0000","last_fullsized":"2024-03-26T06:20:56.235480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:16:17.700523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:16:17.700523+0000","last_clean_scrub_stamp":"2024-03-26T06:16:17.700523+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:14:25.637431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288668+0000","last_change":"2024-03-26T06:20:54.295599+0000","last_active":"2024-03-26T06:20:55.288668+0000","last_peered":"2024-03-26T06:20:55.288668+0000","last_clean":"2024-03-26T06:20:55.288668+0000","last_became_active":"2024-03-26T06:20:54.295270+0000","last_became_peered":"2024-03-26T06:20:54.295270+0000","last_unstale":"2024-03-26T06:20:55.288668+0000","last_undegraded":"2024-03-26T06:20:55.288668+0000","last_fullsized":"2024-03-26T06:20:55.288668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:47.406455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235947+0000","last_change":"2024-03-26T06:20:53.284859+0000","last_active":"2024-03-26T06:20:56.235947+0000","last_peered":"2024-03-26T06:20:56.235947+0000","last_clean":"2024-03-26T06:20:56.235947+0000","last_became_active":"2024-03-26T06:20:53.283671+0000","last_became_peered":"2024-03-26T06:20:53.283671+0000","last_unstale":"2024-03-26T06:20:56.235947+0000","last_undegraded":"2024-03-26T06:20:56.235947+0000","last_fullsized":"2024-03-26T06:20:56.235947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:06:23.676648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:59.582900+0000","last_change":"2024-03-26T06:20:53.288792+0000","last_active":"2024-03-26T06:20:59.582900+0000","last_peered":"2024-03-26T06:20:59.582900+0000","last_clean":"2024-03-26T06:20:59.582900+0000","last_became_active":"2024-03-26T06:20:53.288463+0000","last_became_peered":"2024-03-26T06:20:53.288463+0000","last_unstale":"2024-03-26T06:20:59.582900+0000","last_undegraded":"2024-03-26T06:20:59.582900+0000","last_fullsized":"2024-03-26T06:20:59.582900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:55:55.014266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289046+0000","last_change":"2024-03-26T06:20:54.288478+0000","last_active":"2024-03-26T06:20:55.289046+0000","last_peered":"2024-03-26T06:20:55.289046+0000","last_clean":"2024-03-26T06:20:55.289046+0000","last_became_active":"2024-03-26T06:20:54.288014+0000","last_became_peered":"2024-03-26T06:20:54.288014+0000","last_unstale":"2024-03-26T06:20:55.289046+0000","last_undegraded":"2024-03-26T06:20:55.289046+0000","last_fullsized":"2024-03-26T06:20:55.289046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:36:16.252703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:57.615440+0000","last_change":"2024-03-26T06:20:53.280055+0000","last_active":"2024-03-26T06:20:57.615440+0000","last_peered":"2024-03-26T06:20:57.615440+0000","last_clean":"2024-03-26T06:20:57.615440+0000","last_became_active":"2024-03-26T06:20:53.279751+0000","last_became_peered":"2024-03-26T06:20:53.279751+0000","last_unstale":"2024-03-26T06:20:57.615440+0000","last_undegraded":"2024-03-26T06:20:57.615440+0000","last_fullsized":"2024-03-26T06:20:57.615440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:32:48.407054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082157+0000","last_change":"2024-03-26T06:20:54.302001+0000","last_active":"2024-03-26T06:20:56.082157+0000","last_peered":"2024-03-26T06:20:56.082157+0000","last_clean":"2024-03-26T06:20:56.082157+0000","last_became_active":"2024-03-26T06:20:54.301659+0000","last_became_peered":"2024-03-26T06:20:54.301659+0000","last_unstale":"2024-03-26T06:20:56.082157+0000","last_undegraded":"2024-03-26T06:20:56.082157+0000","last_fullsized":"2024-03-26T06:20:56.082157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:38:08.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081778+0000","last_change":"2024-03-26T06:20:53.285561+0000","last_active":"2024-03-26T06:20:56.081778+0000","last_peered":"2024-03-26T06:20:56.081778+0000","last_clean":"2024-03-26T06:20:56.081778+0000","last_became_active":"2024-03-26T06:20:53.285189+0000","last_became_peered":"2024-03-26T06:20:53.285189+0000","last_unstale":"2024-03-26T06:20:56.081778+0000","last_undegraded":"2024-03-26T06:20:56.081778+0000","last_fullsized":"2024-03-26T06:20:56.081778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:53:54.266188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080654+0000","last_change":"2024-03-26T06:20:54.300421+0000","last_active":"2024-03-26T06:20:56.080654+0000","last_peered":"2024-03-26T06:20:56.080654+0000","last_clean":"2024-03-26T06:20:56.080654+0000","last_became_active":"2024-03-26T06:20:54.299292+0000","last_became_peered":"2024-03-26T06:20:54.299292+0000","last_unstale":"2024-03-26T06:20:56.080654+0000","last_undegraded":"2024-03-26T06:20:56.080654+0000","last_fullsized":"2024-03-26T06:20:56.080654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:29:12.685903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.605209+0000","last_change":"2024-03-26T06:20:53.275182+0000","last_active":"2024-03-26T06:20:56.605209+0000","last_peered":"2024-03-26T06:20:56.605209+0000","last_clean":"2024-03-26T06:20:56.605209+0000","last_became_active":"2024-03-26T06:20:53.274853+0000","last_became_peered":"2024-03-26T06:20:53.274853+0000","last_unstale":"2024-03-26T06:20:56.605209+0000","last_undegraded":"2024-03-26T06:20:56.605209+0000","last_fullsized":"2024-03-26T06:20:56.605209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:57:40.972303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081159+0000","last_change":"2024-03-26T06:20:54.295448+0000","last_active":"2024-03-26T06:20:56.081159+0000","last_peered":"2024-03-26T06:20:56.081159+0000","last_clean":"2024-03-26T06:20:56.081159+0000","last_became_active":"2024-03-26T06:20:54.295116+0000","last_became_peered":"2024-03-26T06:20:54.295116+0000","last_unstale":"2024-03-26T06:20:56.081159+0000","last_undegraded":"2024-03-26T06:20:56.081159+0000","last_fullsized":"2024-03-26T06:20:56.081159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:22:07.474387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081562+0000","last_change":"2024-03-26T06:20:53.281317+0000","last_active":"2024-03-26T06:20:56.081562+0000","last_peered":"2024-03-26T06:20:56.081562+0000","last_clean":"2024-03-26T06:20:56.081562+0000","last_became_active":"2024-03-26T06:20:53.280979+0000","last_became_peered":"2024-03-26T06:20:53.280979+0000","last_unstale":"2024-03-26T06:20:56.081562+0000","last_undegraded":"2024-03-26T06:20:56.081562+0000","last_fullsized":"2024-03-26T06:20:56.081562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:02:29.850564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081209+0000","last_change":"2024-03-26T06:20:54.288963+0000","last_active":"2024-03-26T06:20:56.081209+0000","last_peered":"2024-03-26T06:20:56.081209+0000","last_clean":"2024-03-26T06:20:56.081209+0000","last_became_active":"2024-03-26T06:20:54.288433+0000","last_became_peered":"2024-03-26T06:20:54.288433+0000","last_unstale":"2024-03-26T06:20:56.081209+0000","last_undegraded":"2024-03-26T06:20:56.081209+0000","last_fullsized":"2024-03-26T06:20:56.081209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:33.713471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293107+0000","last_change":"2024-03-26T06:20:53.279874+0000","last_active":"2024-03-26T06:20:55.293107+0000","last_peered":"2024-03-26T06:20:55.293107+0000","last_clean":"2024-03-26T06:20:55.293107+0000","last_became_active":"2024-03-26T06:20:53.279548+0000","last_became_peered":"2024-03-26T06:20:53.279548+0000","last_unstale":"2024-03-26T06:20:55.293107+0000","last_undegraded":"2024-03-26T06:20:55.293107+0000","last_fullsized":"2024-03-26T06:20:55.293107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:14.807650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081742+0000","last_change":"2024-03-26T06:20:54.305353+0000","last_active":"2024-03-26T06:20:56.081742+0000","last_peered":"2024-03-26T06:20:56.081742+0000","last_clean":"2024-03-26T06:20:56.081742+0000","last_became_active":"2024-03-26T06:20:54.304871+0000","last_became_peered":"2024-03-26T06:20:54.304871+0000","last_unstale":"2024-03-26T06:20:56.081742+0000","last_undegraded":"2024-03-26T06:20:56.081742+0000","last_fullsized":"2024-03-26T06:20:56.081742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:24:48.679669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.605147+0000","last_change":"2024-03-26T06:20:53.286291+0000","last_active":"2024-03-26T06:20:56.605147+0000","last_peered":"2024-03-26T06:20:56.605147+0000","last_clean":"2024-03-26T06:20:56.605147+0000","last_became_active":"2024-03-26T06:20:53.285913+0000","last_became_peered":"2024-03-26T06:20:53.285913+0000","last_unstale":"2024-03-26T06:20:56.605147+0000","last_undegraded":"2024-03-26T06:20:56.605147+0000","last_fullsized":"2024-03-26T06:20:56.605147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:33.013108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235999+0000","last_change":"2024-03-26T06:20:54.302788+0000","last_active":"2024-03-26T06:20:56.235999+0000","last_peered":"2024-03-26T06:20:56.235999+0000","last_clean":"2024-03-26T06:20:56.235999+0000","last_became_active":"2024-03-26T06:20:54.302299+0000","last_became_peered":"2024-03-26T06:20:54.302299+0000","last_unstale":"2024-03-26T06:20:56.235999+0000","last_undegraded":"2024-03-26T06:20:56.235999+0000","last_fullsized":"2024-03-26T06:20:56.235999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:17:54.683294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293498+0000","last_change":"2024-03-26T06:20:53.280496+0000","last_active":"2024-03-26T06:20:55.293498+0000","last_peered":"2024-03-26T06:20:55.293498+0000","last_clean":"2024-03-26T06:20:55.293498+0000","last_became_active":"2024-03-26T06:20:53.280178+0000","last_became_peered":"2024-03-26T06:20:53.280178+0000","last_unstale":"2024-03-26T06:20:55.293498+0000","last_undegraded":"2024-03-26T06:20:55.293498+0000","last_fullsized":"2024-03-26T06:20:55.293498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.095552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.236108+0000","last_change":"2024-03-26T06:20:54.299702+0000","last_active":"2024-03-26T06:20:56.236108+0000","last_peered":"2024-03-26T06:20:56.236108+0000","last_clean":"2024-03-26T06:20:56.236108+0000","last_became_active":"2024-03-26T06:20:54.299069+0000","last_became_peered":"2024-03-26T06:20:54.299069+0000","last_unstale":"2024-03-26T06:20:56.236108+0000","last_undegraded":"2024-03-26T06:20:56.236108+0000","last_fullsized":"2024-03-26T06:20:56.236108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:20:56.019540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081951+0000","last_change":"2024-03-26T06:20:53.284304+0000","last_active":"2024-03-26T06:20:56.081951+0000","last_peered":"2024-03-26T06:20:56.081951+0000","last_clean":"2024-03-26T06:20:56.081951+0000","last_became_active":"2024-03-26T06:20:53.283988+0000","last_became_peered":"2024-03-26T06:20:53.283988+0000","last_unstale":"2024-03-26T06:20:56.081951+0000","last_undegraded":"2024-03-26T06:20:56.081951+0000","last_fullsized":"2024-03-26T06:20:56.081951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:07:21.495880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235883+0000","last_change":"2024-03-26T06:20:54.305751+0000","last_active":"2024-03-26T06:20:56.235883+0000","last_peered":"2024-03-26T06:20:56.235883+0000","last_clean":"2024-03-26T06:20:56.235883+0000","last_became_active":"2024-03-26T06:20:54.303904+0000","last_became_peered":"2024-03-26T06:20:54.303904+0000","last_unstale":"2024-03-26T06:20:56.235883+0000","last_undegraded":"2024-03-26T06:20:56.235883+0000","last_fullsized":"2024-03-26T06:20:56.235883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:28:43.597279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288909+0000","last_change":"2024-03-26T06:20:53.283452+0000","last_active":"2024-03-26T06:20:55.288909+0000","last_peered":"2024-03-26T06:20:55.288909+0000","last_clean":"2024-03-26T06:20:55.288909+0000","last_became_active":"2024-03-26T06:20:53.282950+0000","last_became_peered":"2024-03-26T06:20:53.282950+0000","last_unstale":"2024-03-26T06:20:55.288909+0000","last_undegraded":"2024-03-26T06:20:55.288909+0000","last_fullsized":"2024-03-26T06:20:55.288909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:59:15.161087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.236075+0000","last_change":"2024-03-26T06:20:54.300176+0000","last_active":"2024-03-26T06:20:56.236075+0000","last_peered":"2024-03-26T06:20:56.236075+0000","last_clean":"2024-03-26T06:20:56.236075+0000","last_became_active":"2024-03-26T06:20:54.299785+0000","last_became_peered":"2024-03-26T06:20:54.299785+0000","last_unstale":"2024-03-26T06:20:56.236075+0000","last_undegraded":"2024-03-26T06:20:56.236075+0000","last_fullsized":"2024-03-26T06:20:56.236075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:10.091612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081368+0000","last_change":"2024-03-26T06:20:54.288774+0000","last_active":"2024-03-26T06:20:56.081368+0000","last_peered":"2024-03-26T06:20:56.081368+0000","last_clean":"2024-03-26T06:20:56.081368+0000","last_became_active":"2024-03-26T06:20:54.287986+0000","last_became_peered":"2024-03-26T06:20:54.287986+0000","last_unstale":"2024-03-26T06:20:56.081368+0000","last_undegraded":"2024-03-26T06:20:56.081368+0000","last_fullsized":"2024-03-26T06:20:56.081368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:49:04.418838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293321+0000","last_change":"2024-03-26T06:20:54.306650+0000","last_active":"2024-03-26T06:20:55.293321+0000","last_peered":"2024-03-26T06:20:55.293321+0000","last_clean":"2024-03-26T06:20:55.293321+0000","last_became_active":"2024-03-26T06:20:54.306246+0000","last_became_peered":"2024-03-26T06:20:54.306246+0000","last_unstale":"2024-03-26T06:20:55.293321+0000","last_undegraded":"2024-03-26T06:20:55.293321+0000","last_fullsized":"2024-03-26T06:20:55.293321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:12:08.377446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293185+0000","last_change":"2024-03-26T06:20:54.287935+0000","last_active":"2024-03-26T06:20:55.293185+0000","last_peered":"2024-03-26T06:20:55.293185+0000","last_clean":"2024-03-26T06:20:55.293185+0000","last_became_active":"2024-03-26T06:20:54.287575+0000","last_became_peered":"2024-03-26T06:20:54.287575+0000","last_unstale":"2024-03-26T06:20:55.293185+0000","last_undegraded":"2024-03-26T06:20:55.293185+0000","last_fullsized":"2024-03-26T06:20:55.293185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:41:00.110749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080404+0000","last_change":"2024-03-26T06:20:54.299188+0000","last_active":"2024-03-26T06:20:56.080404+0000","last_peered":"2024-03-26T06:20:56.080404+0000","last_clean":"2024-03-26T06:20:56.080404+0000","last_became_active":"2024-03-26T06:20:54.298822+0000","last_became_peered":"2024-03-26T06:20:54.298822+0000","last_unstale":"2024-03-26T06:20:56.080404+0000","last_undegraded":"2024-03-26T06:20:56.080404+0000","last_fullsized":"2024-03-26T06:20:56.080404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:03:17.694808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289017+0000","last_change":"2024-03-26T06:20:54.295974+0000","last_active":"2024-03-26T06:20:55.289017+0000","last_peered":"2024-03-26T06:20:55.289017+0000","last_clean":"2024-03-26T06:20:55.289017+0000","last_became_active":"2024-03-26T06:20:54.295111+0000","last_became_peered":"2024-03-26T06:20:54.295111+0000","last_unstale":"2024-03-26T06:20:55.289017+0000","last_undegraded":"2024-03-26T06:20:55.289017+0000","last_fullsized":"2024-03-26T06:20:55.289017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:53:45.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082595+0000","last_change":"2024-03-26T06:20:54.282386+0000","last_active":"2024-03-26T06:20:56.082595+0000","last_peered":"2024-03-26T06:20:56.082595+0000","last_clean":"2024-03-26T06:20:56.082595+0000","last_became_active":"2024-03-26T06:20:54.282017+0000","last_became_peered":"2024-03-26T06:20:54.282017+0000","last_unstale":"2024-03-26T06:20:56.082595+0000","last_undegraded":"2024-03-26T06:20:56.082595+0000","last_fullsized":"2024-03-26T06:20:56.082595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:09:14.663629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293627+0000","last_change":"2024-03-26T06:20:54.297876+0000","last_active":"2024-03-26T06:20:55.293627+0000","last_peered":"2024-03-26T06:20:55.293627+0000","last_clean":"2024-03-26T06:20:55.293627+0000","last_became_active":"2024-03-26T06:20:54.297545+0000","last_became_peered":"2024-03-26T06:20:54.297545+0000","last_unstale":"2024-03-26T06:20:55.293627+0000","last_undegraded":"2024-03-26T06:20:55.293627+0000","last_fullsized":"2024-03-26T06:20:55.293627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:15:51.786210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082180+0000","last_change":"2024-03-26T06:20:54.304631+0000","last_active":"2024-03-26T06:20:56.082180+0000","last_peered":"2024-03-26T06:20:56.082180+0000","last_clean":"2024-03-26T06:20:56.082180+0000","last_became_active":"2024-03-26T06:20:54.304159+0000","last_became_peered":"2024-03-26T06:20:54.304159+0000","last_unstale":"2024-03-26T06:20:56.082180+0000","last_undegraded":"2024-03-26T06:20:56.082180+0000","last_fullsized":"2024-03-26T06:20:56.082180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:54:15.896127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081470+0000","last_change":"2024-03-26T06:20:54.289849+0000","last_active":"2024-03-26T06:20:56.081470+0000","last_peered":"2024-03-26T06:20:56.081470+0000","last_clean":"2024-03-26T06:20:56.081470+0000","last_became_active":"2024-03-26T06:20:54.287372+0000","last_became_peered":"2024-03-26T06:20:54.287372+0000","last_unstale":"2024-03-26T06:20:56.081470+0000","last_undegraded":"2024-03-26T06:20:56.081470+0000","last_fullsized":"2024-03-26T06:20:56.081470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:48.537975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081591+0000","last_change":"2024-03-26T06:20:54.285737+0000","last_active":"2024-03-26T06:20:56.081591+0000","last_peered":"2024-03-26T06:20:56.081591+0000","last_clean":"2024-03-26T06:20:56.081591+0000","last_became_active":"2024-03-26T06:20:54.285423+0000","last_became_peered":"2024-03-26T06:20:54.285423+0000","last_unstale":"2024-03-26T06:20:56.081591+0000","last_undegraded":"2024-03-26T06:20:56.081591+0000","last_fullsized":"2024-03-26T06:20:56.081591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:55:59.095785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289158+0000","last_change":"2024-03-26T06:20:54.301849+0000","last_active":"2024-03-26T06:20:55.289158+0000","last_peered":"2024-03-26T06:20:55.289158+0000","last_clean":"2024-03-26T06:20:55.289158+0000","last_became_active":"2024-03-26T06:20:54.301444+0000","last_became_peered":"2024-03-26T06:20:54.301444+0000","last_unstale":"2024-03-26T06:20:55.289158+0000","last_undegraded":"2024-03-26T06:20:55.289158+0000","last_fullsized":"2024-03-26T06:20:55.289158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:57:32.365781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288951+0000","last_change":"2024-03-26T06:20:54.302979+0000","last_active":"2024-03-26T06:20:55.288951+0000","last_peered":"2024-03-26T06:20:55.288951+0000","last_clean":"2024-03-26T06:20:55.288951+0000","last_became_active":"2024-03-26T06:20:54.301428+0000","last_became_peered":"2024-03-26T06:20:54.301428+0000","last_unstale":"2024-03-26T06:20:55.288951+0000","last_undegraded":"2024-03-26T06:20:55.288951+0000","last_fullsized":"2024-03-26T06:20:55.288951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:49:29.519202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082319+0000","last_change":"2024-03-26T06:20:54.290284+0000","last_active":"2024-03-26T06:20:56.082319+0000","last_peered":"2024-03-26T06:20:56.082319+0000","last_clean":"2024-03-26T06:20:56.082319+0000","last_became_active":"2024-03-26T06:20:54.289556+0000","last_became_peered":"2024-03-26T06:20:54.289556+0000","last_unstale":"2024-03-26T06:20:56.082319+0000","last_undegraded":"2024-03-26T06:20:56.082319+0000","last_fullsized":"2024-03-26T06:20:56.082319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:32:29.694950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4510,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":110,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":14220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383692,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1544192,"data_stored":1031457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070493,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":440021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1519616,"data_stored":1030093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":439999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":440167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":430055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":429973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1507328,"data_stored":1020501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T06:21:01.413 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T06:21:01.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.550+0000 7fe4d73f1640 1 -- 172.21.15.37:0/4059618279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4d0100230 msgr2=0x7fe4d0100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.550+0000 7fe4d73f1640 1 --2- 172.21.15.37:0/4059618279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4d0100230 0x7fe4d0100650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009900 tx=0x7fe4cc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:21:01.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.550+0000 7fe4d73f1640 1 -- 172.21.15.37:0/4059618279 shutdown_connections 2024-03-26T06:21:01.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.550+0000 7fe4d73f1640 1 --2- 172.21.15.37:0/4059618279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0101de0 0x7fe4d0108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.550+0000 7fe4d73f1640 1 --2- 172.21.15.37:0/4059618279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d0101420 0x7fe4d01018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.550+0000 7fe4d73f1640 1 --2- 172.21.15.37:0/4059618279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4d0100230 0x7fe4d0100650 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.550+0000 7fe4d73f1640 1 -- 172.21.15.37:0/4059618279 >> 172.21.15.37:0/4059618279 conn(0x7fe4d00fb7a0 msgr2=0x7fe4d00fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:01.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.550+0000 7fe4d73f1640 1 -- 172.21.15.37:0/4059618279 shutdown_connections 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.550+0000 7fe4d73f1640 1 -- 172.21.15.37:0/4059618279 wait complete. 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.550+0000 7fe4d73f1640 1 Processor -- start 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.550+0000 7fe4d73f1640 1 -- start start 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d73f1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0100230 0x7fe4d006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d73f1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4d0101420 0x7fe4d019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d73f1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d0101de0 0x7fe4d019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d73f1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d010bf00 con 0x7fe4d0101420 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d73f1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d010bd80 con 0x7fe4d0100230 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d73f1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d010c080 con 0x7fe4d0101de0 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d498c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4d0101420 0x7fe4d019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d498c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4d0101420 0x7fe4d019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46788/0 (socket says 172.21.15.37:46788) 2024-03-26T06:21:01.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d498c640 1 -- 172.21.15.37:0/1391137160 learned_addr learned my addr 172.21.15.37:0/1391137160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d518d640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0100230 0x7fe4d006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d498c640 1 -- 172.21.15.37:0/1391137160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d0101de0 msgr2=0x7fe4d019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d498c640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d0101de0 0x7fe4d019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d498c640 1 -- 172.21.15.37:0/1391137160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0100230 msgr2=0x7fe4d006a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d498c640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0100230 0x7fe4d006a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d498c640 1 -- 172.21.15.37:0/1391137160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d019f310 con 0x7fe4d0101420 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d498c640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4d0101420 0x7fe4d019c410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000b4a0 tx=0x7fe4c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4c67fc640 1 -- 172.21.15.37:0/1391137160 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0014070 con 0x7fe4d0101420 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4c67fc640 1 -- 172.21.15.37:0/1391137160 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4c0010af0 con 0x7fe4d0101420 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d73f1640 1 -- 172.21.15.37:0/1391137160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d019f600 con 0x7fe4d0101420 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4c67fc640 1 -- 172.21.15.37:0/1391137160 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0017d10 con 0x7fe4d0101420 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.551+0000 7fe4d73f1640 1 -- 172.21.15.37:0/1391137160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d019fba0 con 0x7fe4d0101420 2024-03-26T06:21:01.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.552+0000 7fe4d73f1640 1 -- 172.21.15.37:0/1391137160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d01075d0 con 0x7fe4d0101420 2024-03-26T06:21:01.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.553+0000 7fe4c67fc640 1 -- 172.21.15.37:0/1391137160 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fe4c0017070 con 0x7fe4d0101420 2024-03-26T06:21:01.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.554+0000 7fe4c67fc640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe4bc0a2910 0x7fe4bc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:01.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.554+0000 7fe4d518d640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe4bc0a2910 0x7fe4bc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:01.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.554+0000 7fe4c67fc640 1 -- 172.21.15.37:0/1391137160 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe4c00cdd00 con 0x7fe4d0101420 2024-03-26T06:21:01.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.554+0000 7fe4d518d640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe4bc0a2910 0x7fe4bc0a4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc000c00 tx=0x7fe4cc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:01.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.558+0000 7fe4c67fc640 1 -- 172.21.15.37:0/1391137160 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe4c009ae00 con 0x7fe4d0101420 2024-03-26T06:21:01.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:01 smithi037 ceph-mon[31974]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:01.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:01 smithi037 ceph-mon[31974]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:01.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:01 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3693879749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:21:01.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:01 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3469159667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:21:01.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.739+0000 7fe4d73f1640 1 -- 172.21.15.37:0/1391137160 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4d0061730 con 0x7fe4bc0a2910 2024-03-26T06:21:01.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.746+0000 7fe4c67fc640 1 -- 172.21.15.37:0/1391137160 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258744 (secure 0 0 0) 0x7fe4d0061730 con 0x7fe4bc0a2910 2024-03-26T06:21:01.750 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:21:01.751 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 -- 172.21.15.37:0/1391137160 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe4bc0a2910 msgr2=0x7fe4bc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe4bc0a2910 0x7fe4bc0a4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc000c00 tx=0x7fe4cc005fd0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 -- 172.21.15.37:0/1391137160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4d0101420 msgr2=0x7fe4d019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4d0101420 0x7fe4d019c410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000b4a0 tx=0x7fe4c000b960 comp rx=0 tx=0).stop 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 -- 172.21.15.37:0/1391137160 shutdown_connections 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe4bc0a2910 0x7fe4bc0a4dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4d0101de0 0x7fe4d019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4d0101420 0x7fe4d019c410 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 --2- 172.21.15.37:0/1391137160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0100230 0x7fe4d006a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 -- 172.21.15.37:0/1391137160 >> 172.21.15.37:0/1391137160 conn(0x7fe4d00fb7a0 msgr2=0x7fe4d01047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 -- 172.21.15.37:0/1391137160 shutdown_connections 2024-03-26T06:21:01.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:01.749+0000 7fe4d73f1640 1 -- 172.21.15.37:0/1391137160 wait complete. 2024-03-26T06:21:01.761 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-03-26T06:21:00.746155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594878,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":144,"num_read_kb":127,"num_write":291,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":221,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365968,"kb_used_data":12588,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575856,"statfs":{"total":1151940427776,"available":1151565676544,"internally_reserved":0,"allocated":12890112,"data_stored":7018780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4510,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":110,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.477874"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082986+0000","last_change":"2024-03-26T06:20:54.294436+0000","last_active":"2024-03-26T06:20:56.082986+0000","last_peered":"2024-03-26T06:20:56.082986+0000","last_clean":"2024-03-26T06:20:56.082986+0000","last_became_active":"2024-03-26T06:20:54.293049+0000","last_became_peered":"2024-03-26T06:20:54.293049+0000","last_unstale":"2024-03-26T06:20:56.082986+0000","last_undegraded":"2024-03-26T06:20:56.082986+0000","last_fullsized":"2024-03-26T06:20:56.082986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:44:02.729046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080469+0000","last_change":"2024-03-26T06:20:54.292560+0000","last_active":"2024-03-26T06:20:56.080469+0000","last_peered":"2024-03-26T06:20:56.080469+0000","last_clean":"2024-03-26T06:20:56.080469+0000","last_became_active":"2024-03-26T06:20:54.292166+0000","last_became_peered":"2024-03-26T06:20:54.292166+0000","last_unstale":"2024-03-26T06:20:56.080469+0000","last_undegraded":"2024-03-26T06:20:56.080469+0000","last_fullsized":"2024-03-26T06:20:56.080469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:45.255912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080688+0000","last_change":"2024-03-26T06:20:54.294350+0000","last_active":"2024-03-26T06:20:56.080688+0000","last_peered":"2024-03-26T06:20:56.080688+0000","last_clean":"2024-03-26T06:20:56.080688+0000","last_became_active":"2024-03-26T06:20:54.293813+0000","last_became_peered":"2024-03-26T06:20:54.293813+0000","last_unstale":"2024-03-26T06:20:56.080688+0000","last_undegraded":"2024-03-26T06:20:56.080688+0000","last_fullsized":"2024-03-26T06:20:56.080688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:44:31.126828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081978+0000","last_change":"2024-03-26T06:20:54.299151+0000","last_active":"2024-03-26T06:20:56.081978+0000","last_peered":"2024-03-26T06:20:56.081978+0000","last_clean":"2024-03-26T06:20:56.081978+0000","last_became_active":"2024-03-26T06:20:54.298820+0000","last_became_peered":"2024-03-26T06:20:54.298820+0000","last_unstale":"2024-03-26T06:20:56.081978+0000","last_undegraded":"2024-03-26T06:20:56.081978+0000","last_fullsized":"2024-03-26T06:20:56.081978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:47:25.503692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082042+0000","last_change":"2024-03-26T06:20:54.299977+0000","last_active":"2024-03-26T06:20:56.082042+0000","last_peered":"2024-03-26T06:20:56.082042+0000","last_clean":"2024-03-26T06:20:56.082042+0000","last_became_active":"2024-03-26T06:20:54.299641+0000","last_became_peered":"2024-03-26T06:20:54.299641+0000","last_unstale":"2024-03-26T06:20:56.082042+0000","last_undegraded":"2024-03-26T06:20:56.082042+0000","last_fullsized":"2024-03-26T06:20:56.082042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:42:32.478970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082317+0000","last_change":"2024-03-26T06:20:54.294308+0000","last_active":"2024-03-26T06:20:56.082317+0000","last_peered":"2024-03-26T06:20:56.082317+0000","last_clean":"2024-03-26T06:20:56.082317+0000","last_became_active":"2024-03-26T06:20:54.293882+0000","last_became_peered":"2024-03-26T06:20:54.293882+0000","last_unstale":"2024-03-26T06:20:56.082317+0000","last_undegraded":"2024-03-26T06:20:56.082317+0000","last_fullsized":"2024-03-26T06:20:56.082317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:26:26.014097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235824+0000","last_change":"2024-03-26T06:20:54.301644+0000","last_active":"2024-03-26T06:20:56.235824+0000","last_peered":"2024-03-26T06:20:56.235824+0000","last_clean":"2024-03-26T06:20:56.235824+0000","last_became_active":"2024-03-26T06:20:54.301257+0000","last_became_peered":"2024-03-26T06:20:54.301257+0000","last_unstale":"2024-03-26T06:20:56.235824+0000","last_undegraded":"2024-03-26T06:20:56.235824+0000","last_fullsized":"2024-03-26T06:20:56.235824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:43:27.602288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081228+0000","last_change":"2024-03-26T06:20:54.296421+0000","last_active":"2024-03-26T06:20:56.081228+0000","last_peered":"2024-03-26T06:20:56.081228+0000","last_clean":"2024-03-26T06:20:56.081228+0000","last_became_active":"2024-03-26T06:20:54.296086+0000","last_became_peered":"2024-03-26T06:20:54.296086+0000","last_unstale":"2024-03-26T06:20:56.081228+0000","last_undegraded":"2024-03-26T06:20:56.081228+0000","last_fullsized":"2024-03-26T06:20:56.081228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:22:25.532160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289603+0000","last_change":"2024-03-26T06:20:54.297206+0000","last_active":"2024-03-26T06:20:55.289603+0000","last_peered":"2024-03-26T06:20:55.289603+0000","last_clean":"2024-03-26T06:20:55.289603+0000","last_became_active":"2024-03-26T06:20:54.296849+0000","last_became_peered":"2024-03-26T06:20:54.296849+0000","last_unstale":"2024-03-26T06:20:55.289603+0000","last_undegraded":"2024-03-26T06:20:55.289603+0000","last_fullsized":"2024-03-26T06:20:55.289603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:24:49.972237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082381+0000","last_change":"2024-03-26T06:20:54.295048+0000","last_active":"2024-03-26T06:20:56.082381+0000","last_peered":"2024-03-26T06:20:56.082381+0000","last_clean":"2024-03-26T06:20:56.082381+0000","last_became_active":"2024-03-26T06:20:54.294161+0000","last_became_peered":"2024-03-26T06:20:54.294161+0000","last_unstale":"2024-03-26T06:20:56.082381+0000","last_undegraded":"2024-03-26T06:20:56.082381+0000","last_fullsized":"2024-03-26T06:20:56.082381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:41:16.779030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235997+0000","last_change":"2024-03-26T06:20:54.300235+0000","last_active":"2024-03-26T06:20:56.235997+0000","last_peered":"2024-03-26T06:20:56.235997+0000","last_clean":"2024-03-26T06:20:56.235997+0000","last_became_active":"2024-03-26T06:20:54.299677+0000","last_became_peered":"2024-03-26T06:20:54.299677+0000","last_unstale":"2024-03-26T06:20:56.235997+0000","last_undegraded":"2024-03-26T06:20:56.235997+0000","last_fullsized":"2024-03-26T06:20:56.235997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:18:57.028840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081952+0000","last_change":"2024-03-26T06:20:54.303953+0000","last_active":"2024-03-26T06:20:56.081952+0000","last_peered":"2024-03-26T06:20:56.081952+0000","last_clean":"2024-03-26T06:20:56.081952+0000","last_became_active":"2024-03-26T06:20:54.303339+0000","last_became_peered":"2024-03-26T06:20:54.303339+0000","last_unstale":"2024-03-26T06:20:56.081952+0000","last_undegraded":"2024-03-26T06:20:56.081952+0000","last_fullsized":"2024-03-26T06:20:56.081952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:54:38.511221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289015+0000","last_change":"2024-03-26T06:20:54.283535+0000","last_active":"2024-03-26T06:20:55.289015+0000","last_peered":"2024-03-26T06:20:55.289015+0000","last_clean":"2024-03-26T06:20:55.289015+0000","last_became_active":"2024-03-26T06:20:54.282820+0000","last_became_peered":"2024-03-26T06:20:54.282820+0000","last_unstale":"2024-03-26T06:20:55.289015+0000","last_undegraded":"2024-03-26T06:20:55.289015+0000","last_fullsized":"2024-03-26T06:20:55.289015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:13:07.906341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080505+0000","last_change":"2024-03-26T06:20:54.292841+0000","last_active":"2024-03-26T06:20:56.080505+0000","last_peered":"2024-03-26T06:20:56.080505+0000","last_clean":"2024-03-26T06:20:56.080505+0000","last_became_active":"2024-03-26T06:20:54.292530+0000","last_became_peered":"2024-03-26T06:20:54.292530+0000","last_unstale":"2024-03-26T06:20:56.080505+0000","last_undegraded":"2024-03-26T06:20:56.080505+0000","last_fullsized":"2024-03-26T06:20:56.080505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:35:54.592898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080252+0000","last_change":"2024-03-26T06:20:54.306753+0000","last_active":"2024-03-26T06:20:56.080252+0000","last_peered":"2024-03-26T06:20:56.080252+0000","last_clean":"2024-03-26T06:20:56.080252+0000","last_became_active":"2024-03-26T06:20:54.306379+0000","last_became_peered":"2024-03-26T06:20:54.306379+0000","last_unstale":"2024-03-26T06:20:56.080252+0000","last_undegraded":"2024-03-26T06:20:56.080252+0000","last_fullsized":"2024-03-26T06:20:56.080252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:04:19.879733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288996+0000","last_change":"2024-03-26T06:20:54.305719+0000","last_active":"2024-03-26T06:20:55.288996+0000","last_peered":"2024-03-26T06:20:55.288996+0000","last_clean":"2024-03-26T06:20:55.288996+0000","last_became_active":"2024-03-26T06:20:54.305336+0000","last_became_peered":"2024-03-26T06:20:54.305336+0000","last_unstale":"2024-03-26T06:20:55.288996+0000","last_undegraded":"2024-03-26T06:20:55.288996+0000","last_fullsized":"2024-03-26T06:20:55.288996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:22:15.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.083044+0000","last_change":"2024-03-26T06:20:54.293451+0000","last_active":"2024-03-26T06:20:56.083044+0000","last_peered":"2024-03-26T06:20:56.083044+0000","last_clean":"2024-03-26T06:20:56.083044+0000","last_became_active":"2024-03-26T06:20:54.293139+0000","last_became_peered":"2024-03-26T06:20:54.293139+0000","last_unstale":"2024-03-26T06:20:56.083044+0000","last_undegraded":"2024-03-26T06:20:56.083044+0000","last_fullsized":"2024-03-26T06:20:56.083044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:56:04.407687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081187+0000","last_change":"2024-03-26T06:20:54.290256+0000","last_active":"2024-03-26T06:20:56.081187+0000","last_peered":"2024-03-26T06:20:56.081187+0000","last_clean":"2024-03-26T06:20:56.081187+0000","last_became_active":"2024-03-26T06:20:54.289773+0000","last_became_peered":"2024-03-26T06:20:54.289773+0000","last_unstale":"2024-03-26T06:20:56.081187+0000","last_undegraded":"2024-03-26T06:20:56.081187+0000","last_fullsized":"2024-03-26T06:20:56.081187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:46:07.503946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293543+0000","last_change":"2024-03-26T06:20:54.302601+0000","last_active":"2024-03-26T06:20:55.293543+0000","last_peered":"2024-03-26T06:20:55.293543+0000","last_clean":"2024-03-26T06:20:55.293543+0000","last_became_active":"2024-03-26T06:20:54.301523+0000","last_became_peered":"2024-03-26T06:20:54.301523+0000","last_unstale":"2024-03-26T06:20:55.293543+0000","last_undegraded":"2024-03-26T06:20:55.293543+0000","last_fullsized":"2024-03-26T06:20:55.293543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:58:49.055958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:57.616517+0000","last_change":"2024-03-26T06:20:53.280275+0000","last_active":"2024-03-26T06:20:57.616517+0000","last_peered":"2024-03-26T06:20:57.616517+0000","last_clean":"2024-03-26T06:20:57.616517+0000","last_became_active":"2024-03-26T06:20:53.279916+0000","last_became_peered":"2024-03-26T06:20:53.279916+0000","last_unstale":"2024-03-26T06:20:57.616517+0000","last_undegraded":"2024-03-26T06:20:57.616517+0000","last_fullsized":"2024-03-26T06:20:57.616517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:50.844507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081204+0000","last_change":"2024-03-26T06:20:54.302720+0000","last_active":"2024-03-26T06:20:56.081204+0000","last_peered":"2024-03-26T06:20:56.081204+0000","last_clean":"2024-03-26T06:20:56.081204+0000","last_became_active":"2024-03-26T06:20:54.302171+0000","last_became_peered":"2024-03-26T06:20:54.302171+0000","last_unstale":"2024-03-26T06:20:56.081204+0000","last_undegraded":"2024-03-26T06:20:56.081204+0000","last_fullsized":"2024-03-26T06:20:56.081204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:23:12.887883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082078+0000","last_change":"2024-03-26T06:20:53.282355+0000","last_active":"2024-03-26T06:20:56.082078+0000","last_peered":"2024-03-26T06:20:56.082078+0000","last_clean":"2024-03-26T06:20:56.082078+0000","last_became_active":"2024-03-26T06:20:53.282062+0000","last_became_peered":"2024-03-26T06:20:53.282062+0000","last_unstale":"2024-03-26T06:20:56.082078+0000","last_undegraded":"2024-03-26T06:20:56.082078+0000","last_fullsized":"2024-03-26T06:20:56.082078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:58:33.398589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081056+0000","last_change":"2024-03-26T06:20:54.294924+0000","last_active":"2024-03-26T06:20:56.081056+0000","last_peered":"2024-03-26T06:20:56.081056+0000","last_clean":"2024-03-26T06:20:56.081056+0000","last_became_active":"2024-03-26T06:20:54.293971+0000","last_became_peered":"2024-03-26T06:20:54.293971+0000","last_unstale":"2024-03-26T06:20:56.081056+0000","last_undegraded":"2024-03-26T06:20:56.081056+0000","last_fullsized":"2024-03-26T06:20:56.081056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:54:15.457497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:59.594629+0000","last_change":"2024-03-26T06:20:53.286284+0000","last_active":"2024-03-26T06:20:59.594629+0000","last_peered":"2024-03-26T06:20:59.594629+0000","last_clean":"2024-03-26T06:20:59.594629+0000","last_became_active":"2024-03-26T06:20:53.285979+0000","last_became_peered":"2024-03-26T06:20:53.285979+0000","last_unstale":"2024-03-26T06:20:59.594629+0000","last_undegraded":"2024-03-26T06:20:59.594629+0000","last_fullsized":"2024-03-26T06:20:59.594629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:33:31.989055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289440+0000","last_change":"2024-03-26T06:20:54.290137+0000","last_active":"2024-03-26T06:20:55.289440+0000","last_peered":"2024-03-26T06:20:55.289440+0000","last_clean":"2024-03-26T06:20:55.289440+0000","last_became_active":"2024-03-26T06:20:54.289727+0000","last_became_peered":"2024-03-26T06:20:54.289727+0000","last_unstale":"2024-03-26T06:20:55.289440+0000","last_undegraded":"2024-03-26T06:20:55.289440+0000","last_fullsized":"2024-03-26T06:20:55.289440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:31:26.447358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:59.593998+0000","last_change":"2024-03-26T06:20:53.283774+0000","last_active":"2024-03-26T06:20:59.593998+0000","last_peered":"2024-03-26T06:20:59.593998+0000","last_clean":"2024-03-26T06:20:59.593998+0000","last_became_active":"2024-03-26T06:20:53.283430+0000","last_became_peered":"2024-03-26T06:20:53.283430+0000","last_unstale":"2024-03-26T06:20:59.593998+0000","last_undegraded":"2024-03-26T06:20:59.593998+0000","last_fullsized":"2024-03-26T06:20:59.593998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:33:34.747419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082152+0000","last_change":"2024-03-26T06:20:54.293961+0000","last_active":"2024-03-26T06:20:56.082152+0000","last_peered":"2024-03-26T06:20:56.082152+0000","last_clean":"2024-03-26T06:20:56.082152+0000","last_became_active":"2024-03-26T06:20:54.293602+0000","last_became_peered":"2024-03-26T06:20:54.293602+0000","last_unstale":"2024-03-26T06:20:56.082152+0000","last_undegraded":"2024-03-26T06:20:56.082152+0000","last_fullsized":"2024-03-26T06:20:56.082152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:21:08.995710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.605973+0000","last_change":"2024-03-26T06:20:53.287298+0000","last_active":"2024-03-26T06:20:56.605973+0000","last_peered":"2024-03-26T06:20:56.605973+0000","last_clean":"2024-03-26T06:20:56.605973+0000","last_became_active":"2024-03-26T06:20:53.286987+0000","last_became_peered":"2024-03-26T06:20:53.286987+0000","last_unstale":"2024-03-26T06:20:56.605973+0000","last_undegraded":"2024-03-26T06:20:56.605973+0000","last_fullsized":"2024-03-26T06:20:56.605973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:20.343750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081504+0000","last_change":"2024-03-26T06:20:54.291410+0000","last_active":"2024-03-26T06:20:56.081504+0000","last_peered":"2024-03-26T06:20:56.081504+0000","last_clean":"2024-03-26T06:20:56.081504+0000","last_became_active":"2024-03-26T06:20:54.291085+0000","last_became_peered":"2024-03-26T06:20:54.291085+0000","last_unstale":"2024-03-26T06:20:56.081504+0000","last_undegraded":"2024-03-26T06:20:56.081504+0000","last_fullsized":"2024-03-26T06:20:56.081504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:39.727359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:57.617246+0000","last_change":"2024-03-26T06:20:53.285476+0000","last_active":"2024-03-26T06:20:57.617246+0000","last_peered":"2024-03-26T06:20:57.617246+0000","last_clean":"2024-03-26T06:20:57.617246+0000","last_became_active":"2024-03-26T06:20:53.285009+0000","last_became_peered":"2024-03-26T06:20:53.285009+0000","last_unstale":"2024-03-26T06:20:57.617246+0000","last_undegraded":"2024-03-26T06:20:57.617246+0000","last_fullsized":"2024-03-26T06:20:57.617246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:40.006117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081879+0000","last_change":"2024-03-26T06:20:54.299909+0000","last_active":"2024-03-26T06:20:56.081879+0000","last_peered":"2024-03-26T06:20:56.081879+0000","last_clean":"2024-03-26T06:20:56.081879+0000","last_became_active":"2024-03-26T06:20:54.299582+0000","last_became_peered":"2024-03-26T06:20:54.299582+0000","last_unstale":"2024-03-26T06:20:56.081879+0000","last_undegraded":"2024-03-26T06:20:56.081879+0000","last_fullsized":"2024-03-26T06:20:56.081879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:16:41.110029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288640+0000","last_change":"2024-03-26T06:20:53.281619+0000","last_active":"2024-03-26T06:20:55.288640+0000","last_peered":"2024-03-26T06:20:55.288640+0000","last_clean":"2024-03-26T06:20:55.288640+0000","last_became_active":"2024-03-26T06:20:53.279792+0000","last_became_peered":"2024-03-26T06:20:53.279792+0000","last_unstale":"2024-03-26T06:20:55.288640+0000","last_undegraded":"2024-03-26T06:20:55.288640+0000","last_fullsized":"2024-03-26T06:20:55.288640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:17:17.892132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081457+0000","last_change":"2024-03-26T06:20:54.293418+0000","last_active":"2024-03-26T06:20:56.081457+0000","last_peered":"2024-03-26T06:20:56.081457+0000","last_clean":"2024-03-26T06:20:56.081457+0000","last_became_active":"2024-03-26T06:20:54.293109+0000","last_became_peered":"2024-03-26T06:20:54.293109+0000","last_unstale":"2024-03-26T06:20:56.081457+0000","last_undegraded":"2024-03-26T06:20:56.081457+0000","last_fullsized":"2024-03-26T06:20:56.081457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:55:13.764552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.602212+0000","last_change":"2024-03-26T06:20:53.285367+0000","last_active":"2024-03-26T06:20:56.602212+0000","last_peered":"2024-03-26T06:20:56.602212+0000","last_clean":"2024-03-26T06:20:56.602212+0000","last_became_active":"2024-03-26T06:20:53.285022+0000","last_became_peered":"2024-03-26T06:20:53.285022+0000","last_unstale":"2024-03-26T06:20:56.602212+0000","last_undegraded":"2024-03-26T06:20:56.602212+0000","last_fullsized":"2024-03-26T06:20:56.602212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:01:32.288961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081059+0000","last_change":"2024-03-26T06:20:54.290125+0000","last_active":"2024-03-26T06:20:56.081059+0000","last_peered":"2024-03-26T06:20:56.081059+0000","last_clean":"2024-03-26T06:20:56.081059+0000","last_became_active":"2024-03-26T06:20:54.289739+0000","last_became_peered":"2024-03-26T06:20:54.289739+0000","last_unstale":"2024-03-26T06:20:56.081059+0000","last_undegraded":"2024-03-26T06:20:56.081059+0000","last_fullsized":"2024-03-26T06:20:56.081059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:51:01.087526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.602850+0000","last_change":"2024-03-26T06:20:53.289898+0000","last_active":"2024-03-26T06:20:56.602850+0000","last_peered":"2024-03-26T06:20:56.602850+0000","last_clean":"2024-03-26T06:20:56.602850+0000","last_became_active":"2024-03-26T06:20:53.289592+0000","last_became_peered":"2024-03-26T06:20:53.289592+0000","last_unstale":"2024-03-26T06:20:56.602850+0000","last_undegraded":"2024-03-26T06:20:56.602850+0000","last_fullsized":"2024-03-26T06:20:56.602850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:11:41.249408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289546+0000","last_change":"2024-03-26T06:20:54.290970+0000","last_active":"2024-03-26T06:20:55.289546+0000","last_peered":"2024-03-26T06:20:55.289546+0000","last_clean":"2024-03-26T06:20:55.289546+0000","last_became_active":"2024-03-26T06:20:54.289686+0000","last_became_peered":"2024-03-26T06:20:54.289686+0000","last_unstale":"2024-03-26T06:20:55.289546+0000","last_undegraded":"2024-03-26T06:20:55.289546+0000","last_fullsized":"2024-03-26T06:20:55.289546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:39:42.552786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.605097+0000","last_change":"2024-03-26T06:20:53.279247+0000","last_active":"2024-03-26T06:20:56.605097+0000","last_peered":"2024-03-26T06:20:56.605097+0000","last_clean":"2024-03-26T06:20:56.605097+0000","last_became_active":"2024-03-26T06:20:53.278896+0000","last_became_peered":"2024-03-26T06:20:53.278896+0000","last_unstale":"2024-03-26T06:20:56.605097+0000","last_undegraded":"2024-03-26T06:20:56.605097+0000","last_fullsized":"2024-03-26T06:20:56.605097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:53:27.831637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080871+0000","last_change":"2024-03-26T06:20:54.299129+0000","last_active":"2024-03-26T06:20:56.080871+0000","last_peered":"2024-03-26T06:20:56.080871+0000","last_clean":"2024-03-26T06:20:56.080871+0000","last_became_active":"2024-03-26T06:20:54.298777+0000","last_became_peered":"2024-03-26T06:20:54.298777+0000","last_unstale":"2024-03-26T06:20:56.080871+0000","last_undegraded":"2024-03-26T06:20:56.080871+0000","last_fullsized":"2024-03-26T06:20:56.080871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:29:38.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.603979+0000","last_change":"2024-03-26T06:20:53.284988+0000","last_active":"2024-03-26T06:20:56.603979+0000","last_peered":"2024-03-26T06:20:56.603979+0000","last_clean":"2024-03-26T06:20:56.603979+0000","last_became_active":"2024-03-26T06:20:53.284684+0000","last_became_peered":"2024-03-26T06:20:53.284684+0000","last_unstale":"2024-03-26T06:20:56.603979+0000","last_undegraded":"2024-03-26T06:20:56.603979+0000","last_fullsized":"2024-03-26T06:20:56.603979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:31:45.004042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289450+0000","last_change":"2024-03-26T06:20:54.307147+0000","last_active":"2024-03-26T06:20:55.289450+0000","last_peered":"2024-03-26T06:20:55.289450+0000","last_clean":"2024-03-26T06:20:55.289450+0000","last_became_active":"2024-03-26T06:20:54.306797+0000","last_became_peered":"2024-03-26T06:20:54.306797+0000","last_unstale":"2024-03-26T06:20:55.289450+0000","last_undegraded":"2024-03-26T06:20:55.289450+0000","last_fullsized":"2024-03-26T06:20:55.289450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:20:58.904884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.603293+0000","last_change":"2024-03-26T06:20:53.284577+0000","last_active":"2024-03-26T06:20:56.603293+0000","last_peered":"2024-03-26T06:20:56.603293+0000","last_clean":"2024-03-26T06:20:56.603293+0000","last_became_active":"2024-03-26T06:20:53.284161+0000","last_became_peered":"2024-03-26T06:20:53.284161+0000","last_unstale":"2024-03-26T06:20:56.603293+0000","last_undegraded":"2024-03-26T06:20:56.603293+0000","last_fullsized":"2024-03-26T06:20:56.603293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:57.765950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081398+0000","last_change":"2024-03-26T06:20:54.301794+0000","last_active":"2024-03-26T06:20:56.081398+0000","last_peered":"2024-03-26T06:20:56.081398+0000","last_clean":"2024-03-26T06:20:56.081398+0000","last_became_active":"2024-03-26T06:20:54.301445+0000","last_became_peered":"2024-03-26T06:20:54.301445+0000","last_unstale":"2024-03-26T06:20:56.081398+0000","last_undegraded":"2024-03-26T06:20:56.081398+0000","last_fullsized":"2024-03-26T06:20:56.081398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:32:18.280992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293225+0000","last_change":"2024-03-26T06:20:53.282394+0000","last_active":"2024-03-26T06:20:55.293225+0000","last_peered":"2024-03-26T06:20:55.293225+0000","last_clean":"2024-03-26T06:20:55.293225+0000","last_became_active":"2024-03-26T06:20:53.282068+0000","last_became_peered":"2024-03-26T06:20:53.282068+0000","last_unstale":"2024-03-26T06:20:55.293225+0000","last_undegraded":"2024-03-26T06:20:55.293225+0000","last_fullsized":"2024-03-26T06:20:55.293225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:04:00.503682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288758+0000","last_change":"2024-03-26T06:20:54.303294+0000","last_active":"2024-03-26T06:20:55.288758+0000","last_peered":"2024-03-26T06:20:55.288758+0000","last_clean":"2024-03-26T06:20:55.288758+0000","last_became_active":"2024-03-26T06:20:54.301342+0000","last_became_peered":"2024-03-26T06:20:54.301342+0000","last_unstale":"2024-03-26T06:20:55.288758+0000","last_undegraded":"2024-03-26T06:20:55.288758+0000","last_fullsized":"2024-03-26T06:20:55.288758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:53:51.012761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.603651+0000","last_change":"2024-03-26T06:20:53.286119+0000","last_active":"2024-03-26T06:20:56.603651+0000","last_peered":"2024-03-26T06:20:56.603651+0000","last_clean":"2024-03-26T06:20:56.603651+0000","last_became_active":"2024-03-26T06:20:53.285784+0000","last_became_peered":"2024-03-26T06:20:53.285784+0000","last_unstale":"2024-03-26T06:20:56.603651+0000","last_undegraded":"2024-03-26T06:20:56.603651+0000","last_fullsized":"2024-03-26T06:20:56.603651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:54:36.786966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.236156+0000","last_change":"2024-03-26T06:20:54.309706+0000","last_active":"2024-03-26T06:20:56.236156+0000","last_peered":"2024-03-26T06:20:56.236156+0000","last_clean":"2024-03-26T06:20:56.236156+0000","last_became_active":"2024-03-26T06:20:54.309291+0000","last_became_peered":"2024-03-26T06:20:54.309291+0000","last_unstale":"2024-03-26T06:20:56.236156+0000","last_undegraded":"2024-03-26T06:20:56.236156+0000","last_fullsized":"2024-03-26T06:20:56.236156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:51:52.150636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082182+0000","last_change":"2024-03-26T06:20:53.278569+0000","last_active":"2024-03-26T06:20:56.082182+0000","last_peered":"2024-03-26T06:20:56.082182+0000","last_clean":"2024-03-26T06:20:56.082182+0000","last_became_active":"2024-03-26T06:20:53.278207+0000","last_became_peered":"2024-03-26T06:20:53.278207+0000","last_unstale":"2024-03-26T06:20:56.082182+0000","last_undegraded":"2024-03-26T06:20:56.082182+0000","last_fullsized":"2024-03-26T06:20:56.082182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:59:12.345429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235604+0000","last_change":"2024-03-26T06:20:54.308325+0000","last_active":"2024-03-26T06:20:56.235604+0000","last_peered":"2024-03-26T06:20:56.235604+0000","last_clean":"2024-03-26T06:20:56.235604+0000","last_became_active":"2024-03-26T06:20:54.307970+0000","last_became_peered":"2024-03-26T06:20:54.307970+0000","last_unstale":"2024-03-26T06:20:56.235604+0000","last_undegraded":"2024-03-26T06:20:56.235604+0000","last_fullsized":"2024-03-26T06:20:56.235604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:18:26.597073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:59.581783+0000","last_change":"2024-03-26T06:20:53.284675+0000","last_active":"2024-03-26T06:20:59.581783+0000","last_peered":"2024-03-26T06:20:59.581783+0000","last_clean":"2024-03-26T06:20:59.581783+0000","last_became_active":"2024-03-26T06:20:53.284368+0000","last_became_peered":"2024-03-26T06:20:53.284368+0000","last_unstale":"2024-03-26T06:20:59.581783+0000","last_undegraded":"2024-03-26T06:20:59.581783+0000","last_fullsized":"2024-03-26T06:20:59.581783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:15:51.733268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081147+0000","last_change":"2024-03-26T06:20:54.296348+0000","last_active":"2024-03-26T06:20:56.081147+0000","last_peered":"2024-03-26T06:20:56.081147+0000","last_clean":"2024-03-26T06:20:56.081147+0000","last_became_active":"2024-03-26T06:20:54.296019+0000","last_became_peered":"2024-03-26T06:20:54.296019+0000","last_unstale":"2024-03-26T06:20:56.081147+0000","last_undegraded":"2024-03-26T06:20:56.081147+0000","last_fullsized":"2024-03-26T06:20:56.081147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:25:28.498887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:58.610803+0000","last_change":"2024-03-26T06:20:53.281314+0000","last_active":"2024-03-26T06:20:58.610803+0000","last_peered":"2024-03-26T06:20:58.610803+0000","last_clean":"2024-03-26T06:20:58.610803+0000","last_became_active":"2024-03-26T06:20:53.281007+0000","last_became_peered":"2024-03-26T06:20:53.281007+0000","last_unstale":"2024-03-26T06:20:58.610803+0000","last_undegraded":"2024-03-26T06:20:58.610803+0000","last_fullsized":"2024-03-26T06:20:58.610803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:39:25.840470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082652+0000","last_change":"2024-03-26T06:20:54.290011+0000","last_active":"2024-03-26T06:20:56.082652+0000","last_peered":"2024-03-26T06:20:56.082652+0000","last_clean":"2024-03-26T06:20:56.082652+0000","last_became_active":"2024-03-26T06:20:54.289492+0000","last_became_peered":"2024-03-26T06:20:54.289492+0000","last_unstale":"2024-03-26T06:20:56.082652+0000","last_undegraded":"2024-03-26T06:20:56.082652+0000","last_fullsized":"2024-03-26T06:20:56.082652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:39:16.960414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080900+0000","last_change":"2024-03-26T06:20:53.276374+0000","last_active":"2024-03-26T06:20:56.080900+0000","last_peered":"2024-03-26T06:20:56.080900+0000","last_clean":"2024-03-26T06:20:56.080900+0000","last_became_active":"2024-03-26T06:20:53.276031+0000","last_became_peered":"2024-03-26T06:20:53.276031+0000","last_unstale":"2024-03-26T06:20:56.080900+0000","last_undegraded":"2024-03-26T06:20:56.080900+0000","last_fullsized":"2024-03-26T06:20:56.080900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:14.601841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082140+0000","last_change":"2024-03-26T06:20:54.304279+0000","last_active":"2024-03-26T06:20:56.082140+0000","last_peered":"2024-03-26T06:20:56.082140+0000","last_clean":"2024-03-26T06:20:56.082140+0000","last_became_active":"2024-03-26T06:20:54.303859+0000","last_became_peered":"2024-03-26T06:20:54.303859+0000","last_unstale":"2024-03-26T06:20:56.082140+0000","last_undegraded":"2024-03-26T06:20:56.082140+0000","last_fullsized":"2024-03-26T06:20:56.082140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:32:37.969077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235618+0000","last_change":"2024-03-26T06:20:53.286038+0000","last_active":"2024-03-26T06:20:56.235618+0000","last_peered":"2024-03-26T06:20:56.235618+0000","last_clean":"2024-03-26T06:20:56.235618+0000","last_became_active":"2024-03-26T06:20:53.285353+0000","last_became_peered":"2024-03-26T06:20:53.285353+0000","last_unstale":"2024-03-26T06:20:56.235618+0000","last_undegraded":"2024-03-26T06:20:56.235618+0000","last_fullsized":"2024-03-26T06:20:56.235618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:43:53.970393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081851+0000","last_change":"2024-03-26T06:20:54.294425+0000","last_active":"2024-03-26T06:20:56.081851+0000","last_peered":"2024-03-26T06:20:56.081851+0000","last_clean":"2024-03-26T06:20:56.081851+0000","last_became_active":"2024-03-26T06:20:54.293607+0000","last_became_peered":"2024-03-26T06:20:54.293607+0000","last_unstale":"2024-03-26T06:20:56.081851+0000","last_undegraded":"2024-03-26T06:20:56.081851+0000","last_fullsized":"2024-03-26T06:20:56.081851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:12:49.118427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081642+0000","last_change":"2024-03-26T06:20:54.291038+0000","last_active":"2024-03-26T06:20:56.081642+0000","last_peered":"2024-03-26T06:20:56.081642+0000","last_clean":"2024-03-26T06:20:56.081642+0000","last_became_active":"2024-03-26T06:20:54.290678+0000","last_became_peered":"2024-03-26T06:20:54.290678+0000","last_unstale":"2024-03-26T06:20:56.081642+0000","last_undegraded":"2024-03-26T06:20:56.081642+0000","last_fullsized":"2024-03-26T06:20:56.081642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:29:30.528408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288580+0000","last_change":"2024-03-26T06:20:53.288795+0000","last_active":"2024-03-26T06:20:55.288580+0000","last_peered":"2024-03-26T06:20:55.288580+0000","last_clean":"2024-03-26T06:20:55.288580+0000","last_became_active":"2024-03-26T06:20:53.288468+0000","last_became_peered":"2024-03-26T06:20:53.288468+0000","last_unstale":"2024-03-26T06:20:55.288580+0000","last_undegraded":"2024-03-26T06:20:55.288580+0000","last_fullsized":"2024-03-26T06:20:55.288580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:16:05.569407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081864+0000","last_change":"2024-03-26T06:20:54.296706+0000","last_active":"2024-03-26T06:20:56.081864+0000","last_peered":"2024-03-26T06:20:56.081864+0000","last_clean":"2024-03-26T06:20:56.081864+0000","last_became_active":"2024-03-26T06:20:54.296387+0000","last_became_peered":"2024-03-26T06:20:54.296387+0000","last_unstale":"2024-03-26T06:20:56.081864+0000","last_undegraded":"2024-03-26T06:20:56.081864+0000","last_fullsized":"2024-03-26T06:20:56.081864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:23:23.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082176+0000","last_change":"2024-03-26T06:20:53.282790+0000","last_active":"2024-03-26T06:20:56.082176+0000","last_peered":"2024-03-26T06:20:56.082176+0000","last_clean":"2024-03-26T06:20:56.082176+0000","last_became_active":"2024-03-26T06:20:53.282451+0000","last_became_peered":"2024-03-26T06:20:53.282451+0000","last_unstale":"2024-03-26T06:20:56.082176+0000","last_undegraded":"2024-03-26T06:20:56.082176+0000","last_fullsized":"2024-03-26T06:20:56.082176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:47:56.328354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235480+0000","last_change":"2024-03-26T06:19:42.185355+0000","last_active":"2024-03-26T06:20:56.235480+0000","last_peered":"2024-03-26T06:20:56.235480+0000","last_clean":"2024-03-26T06:20:56.235480+0000","last_became_active":"2024-03-26T06:19:42.184672+0000","last_became_peered":"2024-03-26T06:19:42.184672+0000","last_unstale":"2024-03-26T06:20:56.235480+0000","last_undegraded":"2024-03-26T06:20:56.235480+0000","last_fullsized":"2024-03-26T06:20:56.235480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:16:17.700523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:16:17.700523+0000","last_clean_scrub_stamp":"2024-03-26T06:16:17.700523+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:14:25.637431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288668+0000","last_change":"2024-03-26T06:20:54.295599+0000","last_active":"2024-03-26T06:20:55.288668+0000","last_peered":"2024-03-26T06:20:55.288668+0000","last_clean":"2024-03-26T06:20:55.288668+0000","last_became_active":"2024-03-26T06:20:54.295270+0000","last_became_peered":"2024-03-26T06:20:54.295270+0000","last_unstale":"2024-03-26T06:20:55.288668+0000","last_undegraded":"2024-03-26T06:20:55.288668+0000","last_fullsized":"2024-03-26T06:20:55.288668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:47.406455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235947+0000","last_change":"2024-03-26T06:20:53.284859+0000","last_active":"2024-03-26T06:20:56.235947+0000","last_peered":"2024-03-26T06:20:56.235947+0000","last_clean":"2024-03-26T06:20:56.235947+0000","last_became_active":"2024-03-26T06:20:53.283671+0000","last_became_peered":"2024-03-26T06:20:53.283671+0000","last_unstale":"2024-03-26T06:20:56.235947+0000","last_undegraded":"2024-03-26T06:20:56.235947+0000","last_fullsized":"2024-03-26T06:20:56.235947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:06:23.676648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:59.582900+0000","last_change":"2024-03-26T06:20:53.288792+0000","last_active":"2024-03-26T06:20:59.582900+0000","last_peered":"2024-03-26T06:20:59.582900+0000","last_clean":"2024-03-26T06:20:59.582900+0000","last_became_active":"2024-03-26T06:20:53.288463+0000","last_became_peered":"2024-03-26T06:20:53.288463+0000","last_unstale":"2024-03-26T06:20:59.582900+0000","last_undegraded":"2024-03-26T06:20:59.582900+0000","last_fullsized":"2024-03-26T06:20:59.582900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:55:55.014266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289046+0000","last_change":"2024-03-26T06:20:54.288478+0000","last_active":"2024-03-26T06:20:55.289046+0000","last_peered":"2024-03-26T06:20:55.289046+0000","last_clean":"2024-03-26T06:20:55.289046+0000","last_became_active":"2024-03-26T06:20:54.288014+0000","last_became_peered":"2024-03-26T06:20:54.288014+0000","last_unstale":"2024-03-26T06:20:55.289046+0000","last_undegraded":"2024-03-26T06:20:55.289046+0000","last_fullsized":"2024-03-26T06:20:55.289046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:36:16.252703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:57.615440+0000","last_change":"2024-03-26T06:20:53.280055+0000","last_active":"2024-03-26T06:20:57.615440+0000","last_peered":"2024-03-26T06:20:57.615440+0000","last_clean":"2024-03-26T06:20:57.615440+0000","last_became_active":"2024-03-26T06:20:53.279751+0000","last_became_peered":"2024-03-26T06:20:53.279751+0000","last_unstale":"2024-03-26T06:20:57.615440+0000","last_undegraded":"2024-03-26T06:20:57.615440+0000","last_fullsized":"2024-03-26T06:20:57.615440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:32:48.407054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082157+0000","last_change":"2024-03-26T06:20:54.302001+0000","last_active":"2024-03-26T06:20:56.082157+0000","last_peered":"2024-03-26T06:20:56.082157+0000","last_clean":"2024-03-26T06:20:56.082157+0000","last_became_active":"2024-03-26T06:20:54.301659+0000","last_became_peered":"2024-03-26T06:20:54.301659+0000","last_unstale":"2024-03-26T06:20:56.082157+0000","last_undegraded":"2024-03-26T06:20:56.082157+0000","last_fullsized":"2024-03-26T06:20:56.082157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:38:08.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081778+0000","last_change":"2024-03-26T06:20:53.285561+0000","last_active":"2024-03-26T06:20:56.081778+0000","last_peered":"2024-03-26T06:20:56.081778+0000","last_clean":"2024-03-26T06:20:56.081778+0000","last_became_active":"2024-03-26T06:20:53.285189+0000","last_became_peered":"2024-03-26T06:20:53.285189+0000","last_unstale":"2024-03-26T06:20:56.081778+0000","last_undegraded":"2024-03-26T06:20:56.081778+0000","last_fullsized":"2024-03-26T06:20:56.081778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:53:54.266188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080654+0000","last_change":"2024-03-26T06:20:54.300421+0000","last_active":"2024-03-26T06:20:56.080654+0000","last_peered":"2024-03-26T06:20:56.080654+0000","last_clean":"2024-03-26T06:20:56.080654+0000","last_became_active":"2024-03-26T06:20:54.299292+0000","last_became_peered":"2024-03-26T06:20:54.299292+0000","last_unstale":"2024-03-26T06:20:56.080654+0000","last_undegraded":"2024-03-26T06:20:56.080654+0000","last_fullsized":"2024-03-26T06:20:56.080654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:29:12.685903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.605209+0000","last_change":"2024-03-26T06:20:53.275182+0000","last_active":"2024-03-26T06:20:56.605209+0000","last_peered":"2024-03-26T06:20:56.605209+0000","last_clean":"2024-03-26T06:20:56.605209+0000","last_became_active":"2024-03-26T06:20:53.274853+0000","last_became_peered":"2024-03-26T06:20:53.274853+0000","last_unstale":"2024-03-26T06:20:56.605209+0000","last_undegraded":"2024-03-26T06:20:56.605209+0000","last_fullsized":"2024-03-26T06:20:56.605209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:57:40.972303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081159+0000","last_change":"2024-03-26T06:20:54.295448+0000","last_active":"2024-03-26T06:20:56.081159+0000","last_peered":"2024-03-26T06:20:56.081159+0000","last_clean":"2024-03-26T06:20:56.081159+0000","last_became_active":"2024-03-26T06:20:54.295116+0000","last_became_peered":"2024-03-26T06:20:54.295116+0000","last_unstale":"2024-03-26T06:20:56.081159+0000","last_undegraded":"2024-03-26T06:20:56.081159+0000","last_fullsized":"2024-03-26T06:20:56.081159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:22:07.474387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081562+0000","last_change":"2024-03-26T06:20:53.281317+0000","last_active":"2024-03-26T06:20:56.081562+0000","last_peered":"2024-03-26T06:20:56.081562+0000","last_clean":"2024-03-26T06:20:56.081562+0000","last_became_active":"2024-03-26T06:20:53.280979+0000","last_became_peered":"2024-03-26T06:20:53.280979+0000","last_unstale":"2024-03-26T06:20:56.081562+0000","last_undegraded":"2024-03-26T06:20:56.081562+0000","last_fullsized":"2024-03-26T06:20:56.081562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:02:29.850564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081209+0000","last_change":"2024-03-26T06:20:54.288963+0000","last_active":"2024-03-26T06:20:56.081209+0000","last_peered":"2024-03-26T06:20:56.081209+0000","last_clean":"2024-03-26T06:20:56.081209+0000","last_became_active":"2024-03-26T06:20:54.288433+0000","last_became_peered":"2024-03-26T06:20:54.288433+0000","last_unstale":"2024-03-26T06:20:56.081209+0000","last_undegraded":"2024-03-26T06:20:56.081209+0000","last_fullsized":"2024-03-26T06:20:56.081209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:33.713471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293107+0000","last_change":"2024-03-26T06:20:53.279874+0000","last_active":"2024-03-26T06:20:55.293107+0000","last_peered":"2024-03-26T06:20:55.293107+0000","last_clean":"2024-03-26T06:20:55.293107+0000","last_became_active":"2024-03-26T06:20:53.279548+0000","last_became_peered":"2024-03-26T06:20:53.279548+0000","last_unstale":"2024-03-26T06:20:55.293107+0000","last_undegraded":"2024-03-26T06:20:55.293107+0000","last_fullsized":"2024-03-26T06:20:55.293107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:14.807650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081742+0000","last_change":"2024-03-26T06:20:54.305353+0000","last_active":"2024-03-26T06:20:56.081742+0000","last_peered":"2024-03-26T06:20:56.081742+0000","last_clean":"2024-03-26T06:20:56.081742+0000","last_became_active":"2024-03-26T06:20:54.304871+0000","last_became_peered":"2024-03-26T06:20:54.304871+0000","last_unstale":"2024-03-26T06:20:56.081742+0000","last_undegraded":"2024-03-26T06:20:56.081742+0000","last_fullsized":"2024-03-26T06:20:56.081742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:24:48.679669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.605147+0000","last_change":"2024-03-26T06:20:53.286291+0000","last_active":"2024-03-26T06:20:56.605147+0000","last_peered":"2024-03-26T06:20:56.605147+0000","last_clean":"2024-03-26T06:20:56.605147+0000","last_became_active":"2024-03-26T06:20:53.285913+0000","last_became_peered":"2024-03-26T06:20:53.285913+0000","last_unstale":"2024-03-26T06:20:56.605147+0000","last_undegraded":"2024-03-26T06:20:56.605147+0000","last_fullsized":"2024-03-26T06:20:56.605147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:33.013108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235999+0000","last_change":"2024-03-26T06:20:54.302788+0000","last_active":"2024-03-26T06:20:56.235999+0000","last_peered":"2024-03-26T06:20:56.235999+0000","last_clean":"2024-03-26T06:20:56.235999+0000","last_became_active":"2024-03-26T06:20:54.302299+0000","last_became_peered":"2024-03-26T06:20:54.302299+0000","last_unstale":"2024-03-26T06:20:56.235999+0000","last_undegraded":"2024-03-26T06:20:56.235999+0000","last_fullsized":"2024-03-26T06:20:56.235999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:17:54.683294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293498+0000","last_change":"2024-03-26T06:20:53.280496+0000","last_active":"2024-03-26T06:20:55.293498+0000","last_peered":"2024-03-26T06:20:55.293498+0000","last_clean":"2024-03-26T06:20:55.293498+0000","last_became_active":"2024-03-26T06:20:53.280178+0000","last_became_peered":"2024-03-26T06:20:53.280178+0000","last_unstale":"2024-03-26T06:20:55.293498+0000","last_undegraded":"2024-03-26T06:20:55.293498+0000","last_fullsized":"2024-03-26T06:20:55.293498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.095552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.236108+0000","last_change":"2024-03-26T06:20:54.299702+0000","last_active":"2024-03-26T06:20:56.236108+0000","last_peered":"2024-03-26T06:20:56.236108+0000","last_clean":"2024-03-26T06:20:56.236108+0000","last_became_active":"2024-03-26T06:20:54.299069+0000","last_became_peered":"2024-03-26T06:20:54.299069+0000","last_unstale":"2024-03-26T06:20:56.236108+0000","last_undegraded":"2024-03-26T06:20:56.236108+0000","last_fullsized":"2024-03-26T06:20:56.236108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:20:56.019540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081951+0000","last_change":"2024-03-26T06:20:53.284304+0000","last_active":"2024-03-26T06:20:56.081951+0000","last_peered":"2024-03-26T06:20:56.081951+0000","last_clean":"2024-03-26T06:20:56.081951+0000","last_became_active":"2024-03-26T06:20:53.283988+0000","last_became_peered":"2024-03-26T06:20:53.283988+0000","last_unstale":"2024-03-26T06:20:56.081951+0000","last_undegraded":"2024-03-26T06:20:56.081951+0000","last_fullsized":"2024-03-26T06:20:56.081951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:07:21.495880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.235883+0000","last_change":"2024-03-26T06:20:54.305751+0000","last_active":"2024-03-26T06:20:56.235883+0000","last_peered":"2024-03-26T06:20:56.235883+0000","last_clean":"2024-03-26T06:20:56.235883+0000","last_became_active":"2024-03-26T06:20:54.303904+0000","last_became_peered":"2024-03-26T06:20:54.303904+0000","last_unstale":"2024-03-26T06:20:56.235883+0000","last_undegraded":"2024-03-26T06:20:56.235883+0000","last_fullsized":"2024-03-26T06:20:56.235883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:28:43.597279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288909+0000","last_change":"2024-03-26T06:20:53.283452+0000","last_active":"2024-03-26T06:20:55.288909+0000","last_peered":"2024-03-26T06:20:55.288909+0000","last_clean":"2024-03-26T06:20:55.288909+0000","last_became_active":"2024-03-26T06:20:53.282950+0000","last_became_peered":"2024-03-26T06:20:53.282950+0000","last_unstale":"2024-03-26T06:20:55.288909+0000","last_undegraded":"2024-03-26T06:20:55.288909+0000","last_fullsized":"2024-03-26T06:20:55.288909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:59:15.161087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.236075+0000","last_change":"2024-03-26T06:20:54.300176+0000","last_active":"2024-03-26T06:20:56.236075+0000","last_peered":"2024-03-26T06:20:56.236075+0000","last_clean":"2024-03-26T06:20:56.236075+0000","last_became_active":"2024-03-26T06:20:54.299785+0000","last_became_peered":"2024-03-26T06:20:54.299785+0000","last_unstale":"2024-03-26T06:20:56.236075+0000","last_undegraded":"2024-03-26T06:20:56.236075+0000","last_fullsized":"2024-03-26T06:20:56.236075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:10.091612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081368+0000","last_change":"2024-03-26T06:20:54.288774+0000","last_active":"2024-03-26T06:20:56.081368+0000","last_peered":"2024-03-26T06:20:56.081368+0000","last_clean":"2024-03-26T06:20:56.081368+0000","last_became_active":"2024-03-26T06:20:54.287986+0000","last_became_peered":"2024-03-26T06:20:54.287986+0000","last_unstale":"2024-03-26T06:20:56.081368+0000","last_undegraded":"2024-03-26T06:20:56.081368+0000","last_fullsized":"2024-03-26T06:20:56.081368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:49:04.418838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293321+0000","last_change":"2024-03-26T06:20:54.306650+0000","last_active":"2024-03-26T06:20:55.293321+0000","last_peered":"2024-03-26T06:20:55.293321+0000","last_clean":"2024-03-26T06:20:55.293321+0000","last_became_active":"2024-03-26T06:20:54.306246+0000","last_became_peered":"2024-03-26T06:20:54.306246+0000","last_unstale":"2024-03-26T06:20:55.293321+0000","last_undegraded":"2024-03-26T06:20:55.293321+0000","last_fullsized":"2024-03-26T06:20:55.293321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:12:08.377446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293185+0000","last_change":"2024-03-26T06:20:54.287935+0000","last_active":"2024-03-26T06:20:55.293185+0000","last_peered":"2024-03-26T06:20:55.293185+0000","last_clean":"2024-03-26T06:20:55.293185+0000","last_became_active":"2024-03-26T06:20:54.287575+0000","last_became_peered":"2024-03-26T06:20:54.287575+0000","last_unstale":"2024-03-26T06:20:55.293185+0000","last_undegraded":"2024-03-26T06:20:55.293185+0000","last_fullsized":"2024-03-26T06:20:55.293185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:41:00.110749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.080404+0000","last_change":"2024-03-26T06:20:54.299188+0000","last_active":"2024-03-26T06:20:56.080404+0000","last_peered":"2024-03-26T06:20:56.080404+0000","last_clean":"2024-03-26T06:20:56.080404+0000","last_became_active":"2024-03-26T06:20:54.298822+0000","last_became_peered":"2024-03-26T06:20:54.298822+0000","last_unstale":"2024-03-26T06:20:56.080404+0000","last_undegraded":"2024-03-26T06:20:56.080404+0000","last_fullsized":"2024-03-26T06:20:56.080404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:03:17.694808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289017+0000","last_change":"2024-03-26T06:20:54.295974+0000","last_active":"2024-03-26T06:20:55.289017+0000","last_peered":"2024-03-26T06:20:55.289017+0000","last_clean":"2024-03-26T06:20:55.289017+0000","last_became_active":"2024-03-26T06:20:54.295111+0000","last_became_peered":"2024-03-26T06:20:54.295111+0000","last_unstale":"2024-03-26T06:20:55.289017+0000","last_undegraded":"2024-03-26T06:20:55.289017+0000","last_fullsized":"2024-03-26T06:20:55.289017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:53:45.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082595+0000","last_change":"2024-03-26T06:20:54.282386+0000","last_active":"2024-03-26T06:20:56.082595+0000","last_peered":"2024-03-26T06:20:56.082595+0000","last_clean":"2024-03-26T06:20:56.082595+0000","last_became_active":"2024-03-26T06:20:54.282017+0000","last_became_peered":"2024-03-26T06:20:54.282017+0000","last_unstale":"2024-03-26T06:20:56.082595+0000","last_undegraded":"2024-03-26T06:20:56.082595+0000","last_fullsized":"2024-03-26T06:20:56.082595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:09:14.663629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.293627+0000","last_change":"2024-03-26T06:20:54.297876+0000","last_active":"2024-03-26T06:20:55.293627+0000","last_peered":"2024-03-26T06:20:55.293627+0000","last_clean":"2024-03-26T06:20:55.293627+0000","last_became_active":"2024-03-26T06:20:54.297545+0000","last_became_peered":"2024-03-26T06:20:54.297545+0000","last_unstale":"2024-03-26T06:20:55.293627+0000","last_undegraded":"2024-03-26T06:20:55.293627+0000","last_fullsized":"2024-03-26T06:20:55.293627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:15:51.786210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082180+0000","last_change":"2024-03-26T06:20:54.304631+0000","last_active":"2024-03-26T06:20:56.082180+0000","last_peered":"2024-03-26T06:20:56.082180+0000","last_clean":"2024-03-26T06:20:56.082180+0000","last_became_active":"2024-03-26T06:20:54.304159+0000","last_became_peered":"2024-03-26T06:20:54.304159+0000","last_unstale":"2024-03-26T06:20:56.082180+0000","last_undegraded":"2024-03-26T06:20:56.082180+0000","last_fullsized":"2024-03-26T06:20:56.082180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:54:15.896127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081470+0000","last_change":"2024-03-26T06:20:54.289849+0000","last_active":"2024-03-26T06:20:56.081470+0000","last_peered":"2024-03-26T06:20:56.081470+0000","last_clean":"2024-03-26T06:20:56.081470+0000","last_became_active":"2024-03-26T06:20:54.287372+0000","last_became_peered":"2024-03-26T06:20:54.287372+0000","last_unstale":"2024-03-26T06:20:56.081470+0000","last_undegraded":"2024-03-26T06:20:56.081470+0000","last_fullsized":"2024-03-26T06:20:56.081470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:48.537975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.081591+0000","last_change":"2024-03-26T06:20:54.285737+0000","last_active":"2024-03-26T06:20:56.081591+0000","last_peered":"2024-03-26T06:20:56.081591+0000","last_clean":"2024-03-26T06:20:56.081591+0000","last_became_active":"2024-03-26T06:20:54.285423+0000","last_became_peered":"2024-03-26T06:20:54.285423+0000","last_unstale":"2024-03-26T06:20:56.081591+0000","last_undegraded":"2024-03-26T06:20:56.081591+0000","last_fullsized":"2024-03-26T06:20:56.081591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:55:59.095785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.289158+0000","last_change":"2024-03-26T06:20:54.301849+0000","last_active":"2024-03-26T06:20:55.289158+0000","last_peered":"2024-03-26T06:20:55.289158+0000","last_clean":"2024-03-26T06:20:55.289158+0000","last_became_active":"2024-03-26T06:20:54.301444+0000","last_became_peered":"2024-03-26T06:20:54.301444+0000","last_unstale":"2024-03-26T06:20:55.289158+0000","last_undegraded":"2024-03-26T06:20:55.289158+0000","last_fullsized":"2024-03-26T06:20:55.289158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:57:32.365781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:55.288951+0000","last_change":"2024-03-26T06:20:54.302979+0000","last_active":"2024-03-26T06:20:55.288951+0000","last_peered":"2024-03-26T06:20:55.288951+0000","last_clean":"2024-03-26T06:20:55.288951+0000","last_became_active":"2024-03-26T06:20:54.301428+0000","last_became_peered":"2024-03-26T06:20:54.301428+0000","last_unstale":"2024-03-26T06:20:55.288951+0000","last_undegraded":"2024-03-26T06:20:55.288951+0000","last_fullsized":"2024-03-26T06:20:55.288951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:49:29.519202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T06:20:56.082319+0000","last_change":"2024-03-26T06:20:54.290284+0000","last_active":"2024-03-26T06:20:56.082319+0000","last_peered":"2024-03-26T06:20:56.082319+0000","last_clean":"2024-03-26T06:20:56.082319+0000","last_became_active":"2024-03-26T06:20:54.289556+0000","last_became_peered":"2024-03-26T06:20:54.289556+0000","last_unstale":"2024-03-26T06:20:56.082319+0000","last_undegraded":"2024-03-26T06:20:56.082319+0000","last_fullsized":"2024-03-26T06:20:56.082319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:32:29.694950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4510,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":110,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":14220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383692,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1544192,"data_stored":1031457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070493,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":440021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1519616,"data_stored":1030093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":439999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":440167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":430055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":429973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1507328,"data_stored":1020501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T06:21:01.767 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-26T06:21:01.767 INFO:tasks.cephadm:Setting up client nodes... 2024-03-26T06:21:01.768 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-26T06:21:01.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:01 smithi149 ceph-mon[36681]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:01.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:01 smithi149 ceph-mon[36681]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:01.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:01 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3693879749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:21:01.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:01 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3469159667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:21:02.039 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:02.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:01 smithi066 ceph-mon[36504]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:02.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:01 smithi066 ceph-mon[36504]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:21:02.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:01 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3693879749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:21:02.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:01 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3469159667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:21:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:02 smithi037 ceph-mon[31974]: pgmap v292: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-03-26T06:21:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:02 smithi037 ceph-mon[31974]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:21:02.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:02 smithi149 ceph-mon[36681]: pgmap v292: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-03-26T06:21:02.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:02 smithi149 ceph-mon[36681]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:21:03.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:02 smithi066 ceph-mon[36504]: pgmap v292: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-03-26T06:21:03.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:02 smithi066 ceph-mon[36504]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:21:03.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:03 smithi037 ceph-mon[31974]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:21:03.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.751+0000 7fd6885b6700 1 -- 172.21.15.37:0/3162759288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd680101e30 msgr2=0x7fd680102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:03.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.751+0000 7fd6885b6700 1 --2- 172.21.15.37:0/3162759288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd680101e30 0x7fd680102250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009990 tx=0x7fd67c009ca0 comp rx=0 tx=0).stop 2024-03-26T06:21:03.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.751+0000 7fd6885b6700 1 -- 172.21.15.37:0/3162759288 shutdown_connections 2024-03-26T06:21:03.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.751+0000 7fd6885b6700 1 --2- 172.21.15.37:0/3162759288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd680101e30 0x7fd680102250 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:03.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.751+0000 7fd6885b6700 1 --2- 172.21.15.37:0/3162759288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd680103950 0x7fd68010a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:03.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.751+0000 7fd6885b6700 1 --2- 172.21.15.37:0/3162759288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd680102f90 0x7fd680103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:03.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.751+0000 7fd6885b6700 1 -- 172.21.15.37:0/3162759288 >> 172.21.15.37:0/3162759288 conn(0x7fd6800fd3f0 msgr2=0x7fd6800ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:03.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.751+0000 7fd6885b6700 1 -- 172.21.15.37:0/3162759288 shutdown_connections 2024-03-26T06:21:03.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.751+0000 7fd6885b6700 1 -- 172.21.15.37:0/3162759288 wait complete. 2024-03-26T06:21:03.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd6885b6700 1 Processor -- start 2024-03-26T06:21:03.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd6885b6700 1 -- start start 2024-03-26T06:21:03.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd6885b6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd680101e30 0x7fd680197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:03.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd6885b6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd680102f90 0x7fd6801984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:03.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd6885b6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd680103950 0x7fd68019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:03.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd6885b6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68010da20 con 0x7fd680103950 2024-03-26T06:21:03.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd6885b6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68010d8a0 con 0x7fd680101e30 2024-03-26T06:21:03.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd6885b6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68010dba0 con 0x7fd680102f90 2024-03-26T06:21:03.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd686b53700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd680103950 0x7fd68019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:03.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd686b53700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd680103950 0x7fd68019f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46818/0 (socket says 172.21.15.37:46818) 2024-03-26T06:21:03.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd686b53700 1 -- 172.21.15.37:0/3318585148 learned_addr learned my addr 172.21.15.37:0/3318585148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:03.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd686352700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd680101e30 0x7fd680197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:03.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.752+0000 7fd685b51700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd680102f90 0x7fd6801984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:03.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.753+0000 7fd686b53700 1 -- 172.21.15.37:0/3318585148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd680102f90 msgr2=0x7fd6801984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:03.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.753+0000 7fd686b53700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd680102f90 0x7fd6801984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.753+0000 7fd686b53700 1 -- 172.21.15.37:0/3318585148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd680101e30 msgr2=0x7fd680197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.753+0000 7fd686b53700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd680101e30 0x7fd680197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.753+0000 7fd686b53700 1 -- 172.21.15.37:0/3318585148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68019fc10 con 0x7fd680103950 2024-03-26T06:21:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.753+0000 7fd686b53700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd680103950 0x7fd68019f570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd678007f10 tx=0x7fd67800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.753+0000 7fd6777fe700 1 -- 172.21.15.37:0/3318585148 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678004bb0 con 0x7fd680103950 2024-03-26T06:21:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.753+0000 7fd6777fe700 1 -- 172.21.15.37:0/3318585148 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd678004d50 con 0x7fd680103950 2024-03-26T06:21:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.753+0000 7fd6885b6700 1 -- 172.21.15.37:0/3318585148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68019ff00 con 0x7fd680103950 2024-03-26T06:21:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.753+0000 7fd6777fe700 1 -- 172.21.15.37:0/3318585148 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6780059f0 con 0x7fd680103950 2024-03-26T06:21:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.753+0000 7fd6885b6700 1 -- 172.21.15.37:0/3318585148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6801a03b0 con 0x7fd680103950 2024-03-26T06:21:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.754+0000 7fd6885b6700 1 -- 172.21.15.37:0/3318585148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6800502f0 con 0x7fd680103950 2024-03-26T06:21:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.755+0000 7fd6777fe700 1 -- 172.21.15.37:0/3318585148 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fd67801a020 con 0x7fd680103950 2024-03-26T06:21:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.756+0000 7fd6777fe700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd66c0a29d0 0x7fd66c0a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.756+0000 7fd686352700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd66c0a29d0 0x7fd66c0a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.756+0000 7fd6777fe700 1 -- 172.21.15.37:0/3318585148 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd6780ce6f0 con 0x7fd680103950 2024-03-26T06:21:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.756+0000 7fd686352700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd66c0a29d0 0x7fd66c0a4e90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00b5c0 tx=0x7fd67c0038f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:03.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:03.759+0000 7fd6777fe700 1 -- 172.21.15.37:0/3318585148 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd678010070 con 0x7fd680103950 2024-03-26T06:21:03.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:03 smithi149 ceph-mon[36681]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:21:04.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.050+0000 7fd6885b6700 1 -- 172.21.15.37:0/3318585148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fd68002d830 con 0x7fd680103950 2024-03-26T06:21:04.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.052+0000 7fd6777fe700 1 -- 172.21.15.37:0/3318585148 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7fd68002d830 con 0x7fd680103950 2024-03-26T06:21:04.053 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-03-26T06:21:04.053 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBQaQJmTmAbAxAAc8e6WnR3UaQIbGiRG7tdJg== 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.054+0000 7fd6885b6700 1 -- 172.21.15.37:0/3318585148 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd66c0a29d0 msgr2=0x7fd66c0a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.054+0000 7fd6885b6700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd66c0a29d0 0x7fd66c0a4e90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00b5c0 tx=0x7fd67c0038f0 comp rx=0 tx=0).stop 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.055+0000 7fd6885b6700 1 -- 172.21.15.37:0/3318585148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd680103950 msgr2=0x7fd68019f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.055+0000 7fd6885b6700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd680103950 0x7fd68019f570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd678007f10 tx=0x7fd67800d3b0 comp rx=0 tx=0).stop 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.055+0000 7fd6885b6700 1 -- 172.21.15.37:0/3318585148 shutdown_connections 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.055+0000 7fd6885b6700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd680103950 0x7fd68019f570 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.055+0000 7fd6885b6700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd66c0a29d0 0x7fd66c0a4e90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.055+0000 7fd6885b6700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd680101e30 0x7fd680197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.055+0000 7fd6885b6700 1 --2- 172.21.15.37:0/3318585148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd680102f90 0x7fd6801984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.055+0000 7fd6885b6700 1 -- 172.21.15.37:0/3318585148 >> 172.21.15.37:0/3318585148 conn(0x7fd6800fd3f0 msgr2=0x7fd6801062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.055+0000 7fd6885b6700 1 -- 172.21.15.37:0/3318585148 shutdown_connections 2024-03-26T06:21:04.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:04.055+0000 7fd6885b6700 1 -- 172.21.15.37:0/3318585148 wait complete. 2024-03-26T06:21:04.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:03 smithi066 ceph-mon[36504]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:21:04.689 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:21:04.689 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-26T06:21:04.689 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-26T06:21:04.733 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-26T06:21:04.734 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-26T06:21:04.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-26T06:21:04.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:04 smithi149 ceph-mon[36681]: pgmap v293: 97 pgs: 97 active+clean; 585 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7.3 KiB/s wr, 26 op/s 2024-03-26T06:21:04.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:04 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3318585148' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T06:21:04.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:04 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3318585148' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T06:21:04.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:04 smithi037 ceph-mon[31974]: pgmap v293: 97 pgs: 97 active+clean; 585 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7.3 KiB/s wr, 26 op/s 2024-03-26T06:21:04.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:04 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3318585148' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T06:21:04.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:04 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3318585148' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T06:21:05.014 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:05.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:04 smithi066 ceph-mon[36504]: pgmap v293: 97 pgs: 97 active+clean; 585 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7.3 KiB/s wr, 26 op/s 2024-03-26T06:21:05.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:04 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3318585148' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T06:21:05.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:04 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3318585148' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T06:21:06.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.547+0000 7f402472b700 1 -- 172.21.15.37:0/1872052520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c06af30 msgr2=0x7f401c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:06.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.547+0000 7f402472b700 1 --2- 172.21.15.37:0/1872052520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c06af30 0x7f401c106a10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f400c009990 tx=0x7f400c009ca0 comp rx=0 tx=0).stop 2024-03-26T06:21:06.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.547+0000 7f402472b700 1 -- 172.21.15.37:0/1872052520 shutdown_connections 2024-03-26T06:21:06.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.547+0000 7f402472b700 1 --2- 172.21.15.37:0/1872052520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f401c1070f0 0x7f401c10eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:06.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.547+0000 7f402472b700 1 --2- 172.21.15.37:0/1872052520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f401c06a580 0x7f401c06a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:06.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.547+0000 7f402472b700 1 --2- 172.21.15.37:0/1872052520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c06af30 0x7f401c106a10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:06.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.547+0000 7f402472b700 1 -- 172.21.15.37:0/1872052520 >> 172.21.15.37:0/1872052520 conn(0x7f401c076d30 msgr2=0x7f401c077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:06.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.547+0000 7f402472b700 1 -- 172.21.15.37:0/1872052520 shutdown_connections 2024-03-26T06:21:06.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.547+0000 7f402472b700 1 -- 172.21.15.37:0/1872052520 wait complete. 2024-03-26T06:21:06.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.547+0000 7f402472b700 1 Processor -- start 2024-03-26T06:21:06.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.547+0000 7f402472b700 1 -- start start 2024-03-26T06:21:06.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f402472b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f401c06a580 0x7f401c100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:06.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f402472b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f401c06af30 0x7f401c101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f402472b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1070f0 0x7f401c101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f402472b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c112070 con 0x7f401c1070f0 2024-03-26T06:21:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f402472b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c111ef0 con 0x7f401c06a580 2024-03-26T06:21:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f402472b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c1121f0 con 0x7f401c06af30 2024-03-26T06:21:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f4022cc8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1070f0 0x7f401c101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f4022cc8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1070f0 0x7f401c101860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46836/0 (socket says 172.21.15.37:46836) 2024-03-26T06:21:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f4022cc8700 1 -- 172.21.15.37:0/1377651581 learned_addr learned my addr 172.21.15.37:0/1377651581 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f40224c7700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f401c06a580 0x7f401c100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f4021cc6700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f401c06af30 0x7f401c101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f4022cc8700 1 -- 172.21.15.37:0/1377651581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f401c06af30 msgr2=0x7f401c101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f4022cc8700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f401c06af30 0x7f401c101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f4022cc8700 1 -- 172.21.15.37:0/1377651581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f401c06a580 msgr2=0x7f401c100de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f4022cc8700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f401c06a580 0x7f401c100de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f4022cc8700 1 -- 172.21.15.37:0/1377651581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c1a6a10 con 0x7f401c1070f0 2024-03-26T06:21:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f4022cc8700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1070f0 0x7f401c101860 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4014007ee0 tx=0x7f401400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f40137fe700 1 -- 172.21.15.37:0/1377651581 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401400dbd0 con 0x7f401c1070f0 2024-03-26T06:21:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f402472b700 1 -- 172.21.15.37:0/1377651581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c1a6d00 con 0x7f401c1070f0 2024-03-26T06:21:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f40137fe700 1 -- 172.21.15.37:0/1377651581 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f401400dd70 con 0x7f401c1070f0 2024-03-26T06:21:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f40137fe700 1 -- 172.21.15.37:0/1377651581 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014005a60 con 0x7f401c1070f0 2024-03-26T06:21:06.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.548+0000 7f402472b700 1 -- 172.21.15.37:0/1377651581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c1a71f0 con 0x7f401c1070f0 2024-03-26T06:21:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.549+0000 7f402472b700 1 -- 172.21.15.37:0/1377651581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c068aa0 con 0x7f401c1070f0 2024-03-26T06:21:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.550+0000 7f40137fe700 1 -- 172.21.15.37:0/1377651581 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f401401a020 con 0x7f401c1070f0 2024-03-26T06:21:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.551+0000 7f40137fe700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f40080a2920 0x7f40080a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.551+0000 7f40224c7700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f40080a2920 0x7f40080a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.551+0000 7f40137fe700 1 -- 172.21.15.37:0/1377651581 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f40140ce7f0 con 0x7f401c1070f0 2024-03-26T06:21:06.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.552+0000 7f40224c7700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f40080a2920 0x7f40080a4de0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4018005950 tx=0x7f401800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:06.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.555+0000 7f40137fe700 1 -- 172.21.15.37:0/1377651581 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4014010070 con 0x7f401c1070f0 2024-03-26T06:21:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.824+0000 7f402472b700 1 -- 172.21.15.37:0/1377651581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f401c102310 con 0x7f401c1070f0 2024-03-26T06:21:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.829+0000 7f40137fe700 1 -- 172.21.15.37:0/1377651581 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+262656 (secure 0 0 0) 0x7f401c102310 con 0x7f401c1070f0 2024-03-26T06:21:06.834 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:21:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 -- 172.21.15.37:0/1377651581 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f40080a2920 msgr2=0x7f40080a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f40080a2920 0x7f40080a4de0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4018005950 tx=0x7f401800b410 comp rx=0 tx=0).stop 2024-03-26T06:21:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 -- 172.21.15.37:0/1377651581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1070f0 msgr2=0x7f401c101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1070f0 0x7f401c101860 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4014007ee0 tx=0x7f401400d3b0 comp rx=0 tx=0).stop 2024-03-26T06:21:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 -- 172.21.15.37:0/1377651581 shutdown_connections 2024-03-26T06:21:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1070f0 0x7f401c101860 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f40080a2920 0x7f40080a4de0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f401c06a580 0x7f401c100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 --2- 172.21.15.37:0/1377651581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f401c06af30 0x7f401c101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 -- 172.21.15.37:0/1377651581 >> 172.21.15.37:0/1377651581 conn(0x7f401c076d30 msgr2=0x7f401c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 -- 172.21.15.37:0/1377651581 shutdown_connections 2024-03-26T06:21:06.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:06.832+0000 7f402472b700 1 -- 172.21.15.37:0/1377651581 wait complete. 2024-03-26T06:21:06.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:06 smithi149 ceph-mon[36681]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-03-26T06:21:06.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:06 smithi037 ceph-mon[31974]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-03-26T06:21:07.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:06 smithi066 ceph-mon[36504]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-03-26T06:21:07.472 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":641545212},{"type":"v1","addr":"172.21.15.37:6801","nonce":641545212}]},"active_addr":"172.21.15.37:6801/641545212","active_change":"2024-03-26T06:13:46.665918+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24122,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.37:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2221744469}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":374271722}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":3212150316}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":656518427}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":3606343279}]}]} 2024-03-26T06:21:07.478 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-26T06:21:07.478 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-26T06:21:07.478 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T06:21:07.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:07 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1377651581' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T06:21:07.729 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:07.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:07 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1377651581' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T06:21:08.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:07 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1377651581' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T06:21:08.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:08 smithi149 ceph-mon[36681]: pgmap v295: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2024-03-26T06:21:08.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:08 smithi037 ceph-mon[31974]: pgmap v295: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2024-03-26T06:21:09.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:08 smithi066 ceph-mon[36504]: pgmap v295: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2024-03-26T06:21:09.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.421+0000 7f7ee4f16700 1 -- 172.21.15.37:0/2855307936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ee01027f0 msgr2=0x7f7ee010ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:09.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.421+0000 7f7ee4f16700 1 --2- 172.21.15.37:0/2855307936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ee01027f0 0x7f7ee010ec90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800b0a0 tx=0x7f7ed800b3b0 comp rx=0 tx=0).stop 2024-03-26T06:21:09.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.421+0000 7f7ee4f16700 1 -- 172.21.15.37:0/2855307936 shutdown_connections 2024-03-26T06:21:09.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.421+0000 7f7ee4f16700 1 --2- 172.21.15.37:0/2855307936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ee01027f0 0x7f7ee010ec90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:09.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.421+0000 7f7ee4f16700 1 --2- 172.21.15.37:0/2855307936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ee0107ee0 0x7f7ee01082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:09.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.421+0000 7f7ee4f16700 1 --2- 172.21.15.37:0/2855307936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee0101e30 0x7f7ee01022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:09.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.421+0000 7f7ee4f16700 1 -- 172.21.15.37:0/2855307936 >> 172.21.15.37:0/2855307936 conn(0x7f7ee0078f10 msgr2=0x7f7ee0079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:09.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.421+0000 7f7ee4f16700 1 -- 172.21.15.37:0/2855307936 shutdown_connections 2024-03-26T06:21:09.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.421+0000 7f7ee4f16700 1 -- 172.21.15.37:0/2855307936 wait complete. 2024-03-26T06:21:09.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.422+0000 7f7ee4f16700 1 Processor -- start 2024-03-26T06:21:09.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.422+0000 7f7ee4f16700 1 -- start start 2024-03-26T06:21:09.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ee4f16700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ee0101e30 0x7f7ee00740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:09.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ee4f16700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee01027f0 0x7f7ee0074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:09.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ee4f16700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ee0107ee0 0x7f7ee0074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:09.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ee4f16700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee01121f0 con 0x7f7ee0101e30 2024-03-26T06:21:09.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ede59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ee0101e30 0x7f7ee00740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:09.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ede59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ee0101e30 0x7f7ee00740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39048/0 (socket says 172.21.15.37:39048) 2024-03-26T06:21:09.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ede59c700 1 -- 172.21.15.37:0/1413211870 learned_addr learned my addr 172.21.15.37:0/1413211870 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ee4f16700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee0112070 con 0x7f7ee01027f0 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ee4f16700 1 -- 172.21.15.37:0/1413211870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee0112370 con 0x7f7ee0107ee0 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7eddd9b700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee01027f0 0x7f7ee0074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7eded9d700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ee0107ee0 0x7f7ee0074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ede59c700 1 -- 172.21.15.37:0/1413211870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ee0107ee0 msgr2=0x7f7ee0074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ede59c700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ee0107ee0 0x7f7ee0074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ede59c700 1 -- 172.21.15.37:0/1413211870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee01027f0 msgr2=0x7f7ee0074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ede59c700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee01027f0 0x7f7ee0074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ede59c700 1 -- 172.21.15.37:0/1413211870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee01a6af0 con 0x7f7ee0101e30 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7eded9d700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ee0107ee0 0x7f7ee0074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ede59c700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ee0101e30 0x7f7ee00740e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800b530 tx=0x7f7ec800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:09.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ed77fe700 1 -- 172.21.15.37:0/1413211870 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8015020 con 0x7f7ee0101e30 2024-03-26T06:21:09.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ed77fe700 1 -- 172.21.15.37:0/1413211870 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ec8011d10 con 0x7f7ee0101e30 2024-03-26T06:21:09.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.423+0000 7f7ee4f16700 1 -- 172.21.15.37:0/1413211870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee01a6de0 con 0x7f7ee0101e30 2024-03-26T06:21:09.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.424+0000 7f7ed77fe700 1 -- 172.21.15.37:0/1413211870 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8010e50 con 0x7f7ee0101e30 2024-03-26T06:21:09.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.424+0000 7f7ee4f16700 1 -- 172.21.15.37:0/1413211870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee01a72e0 con 0x7f7ee0101e30 2024-03-26T06:21:09.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.425+0000 7f7ee4f16700 1 -- 172.21.15.37:0/1413211870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee0068aa0 con 0x7f7ee0101e30 2024-03-26T06:21:09.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.426+0000 7f7ed77fe700 1 -- 172.21.15.37:0/1413211870 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f7ec8018020 con 0x7f7ee0101e30 2024-03-26T06:21:09.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.426+0000 7f7ed77fe700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7ecc0a2b40 0x7f7ecc0a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:09.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.426+0000 7f7eddd9b700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7ecc0a2b40 0x7f7ecc0a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:09.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.426+0000 7f7ed77fe700 1 -- 172.21.15.37:0/1413211870 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7ec80ce100 con 0x7f7ee0101e30 2024-03-26T06:21:09.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.427+0000 7f7eddd9b700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7ecc0a2b40 0x7f7ecc0a5000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0006fd0 tx=0x7f7ed0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:09.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.431+0000 7f7ed77fe700 1 -- 172.21.15.37:0/1413211870 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7ec801f020 con 0x7f7ee0101e30 2024-03-26T06:21:09.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.636+0000 7f7ee4f16700 1 -- 172.21.15.37:0/1413211870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ee002d9f0 con 0x7f7ee0101e30 2024-03-26T06:21:09.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.637+0000 7f7ed77fe700 1 -- 172.21.15.37:0/1413211870 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22145 (secure 0 0 0) 0x7f7ec809b300 con 0x7f7ee0101e30 2024-03-26T06:21:09.638 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:21:09.638 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":85,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:20:55.279252+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:21:09.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 -- 172.21.15.37:0/1413211870 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7ecc0a2b40 msgr2=0x7f7ecc0a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:09.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7ecc0a2b40 0x7f7ecc0a5000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0006fd0 tx=0x7f7ed0008040 comp rx=0 tx=0).stop 2024-03-26T06:21:09.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 -- 172.21.15.37:0/1413211870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ee0101e30 msgr2=0x7f7ee00740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:09.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ee0101e30 0x7f7ee00740e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800b530 tx=0x7f7ec800b8f0 comp rx=0 tx=0).stop 2024-03-26T06:21:09.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 -- 172.21.15.37:0/1413211870 shutdown_connections 2024-03-26T06:21:09.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ee0101e30 0x7f7ee00740e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:09.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7ecc0a2b40 0x7f7ecc0a5000 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:09.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ee0107ee0 0x7f7ee0074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:09.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 --2- 172.21.15.37:0/1413211870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ee01027f0 0x7f7ee0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:09.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 -- 172.21.15.37:0/1413211870 >> 172.21.15.37:0/1413211870 conn(0x7f7ee0078f10 msgr2=0x7f7ee0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:09.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 -- 172.21.15.37:0/1413211870 shutdown_connections 2024-03-26T06:21:09.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:09.639+0000 7f7ee4f16700 1 -- 172.21.15.37:0/1413211870 wait complete. 2024-03-26T06:21:09.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:09 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1413211870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:21:09.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:09 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1413211870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:21:10.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:09 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1413211870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:21:10.277 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-26T06:21:10.277 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T06:21:10.526 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:10.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:10 smithi149 ceph-mon[36681]: pgmap v296: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.8 KiB/s wr, 26 op/s 2024-03-26T06:21:10.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:10 smithi037 ceph-mon[31974]: pgmap v296: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.8 KiB/s wr, 26 op/s 2024-03-26T06:21:11.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:10 smithi066 ceph-mon[36504]: pgmap v296: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.8 KiB/s wr, 26 op/s 2024-03-26T06:21:12.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.186+0000 7f110329f700 1 -- 172.21.15.37:0/3288932234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc1071a0 msgr2=0x7f10fc10ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:12.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.186+0000 7f110329f700 1 --2- 172.21.15.37:0/3288932234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc1071a0 0x7f10fc10ed20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0099c0 tx=0x7f10ec009cd0 comp rx=0 tx=0).stop 2024-03-26T06:21:12.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.186+0000 7f110329f700 1 -- 172.21.15.37:0/3288932234 shutdown_connections 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.186+0000 7f110329f700 1 --2- 172.21.15.37:0/3288932234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc1071a0 0x7f10fc10ed20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.186+0000 7f110329f700 1 --2- 172.21.15.37:0/3288932234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10fc100980 0x7f10fc100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.186+0000 7f110329f700 1 --2- 172.21.15.37:0/3288932234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10fc101330 0x7f10fc106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.186+0000 7f110329f700 1 -- 172.21.15.37:0/3288932234 >> 172.21.15.37:0/3288932234 conn(0x7f10fc076020 msgr2=0x7f10fc076430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.186+0000 7f110329f700 1 -- 172.21.15.37:0/3288932234 shutdown_connections 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.186+0000 7f110329f700 1 -- 172.21.15.37:0/3288932234 wait complete. 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.186+0000 7f110329f700 1 Processor -- start 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.186+0000 7f110329f700 1 -- start start 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110329f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10fc100980 0x7f10fc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110329f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10fc101330 0x7f10fc198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110329f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc1071a0 0x7f10fc19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110329f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc112210 con 0x7f10fc1071a0 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110329f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc112090 con 0x7f10fc101330 2024-03-26T06:21:12.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110329f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc112390 con 0x7f10fc100980 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110183c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc1071a0 0x7f10fc19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110183c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc1071a0 0x7f10fc19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39062/0 (socket says 172.21.15.37:39062) 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110183c700 1 -- 172.21.15.37:0/557304792 learned_addr learned my addr 172.21.15.37:0/557304792 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110103b700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10fc100980 0x7f10fc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110083a700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10fc101330 0x7f10fc198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110183c700 1 -- 172.21.15.37:0/557304792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10fc100980 msgr2=0x7f10fc1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110183c700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10fc100980 0x7f10fc1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110183c700 1 -- 172.21.15.37:0/557304792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10fc101330 msgr2=0x7f10fc198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110183c700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10fc101330 0x7f10fc198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110183c700 1 -- 172.21.15.37:0/557304792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc19d5d0 con 0x7f10fc1071a0 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110183c700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc1071a0 0x7f10fc19ce90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0057c0 tx=0x7f10ec00bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f10fa7fc700 1 -- 172.21.15.37:0/557304792 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec027070 con 0x7f10fc1071a0 2024-03-26T06:21:12.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f10fa7fc700 1 -- 172.21.15.37:0/557304792 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10ec0040b0 con 0x7f10fc1071a0 2024-03-26T06:21:12.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110329f700 1 -- 172.21.15.37:0/557304792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10fc19d860 con 0x7f10fc1071a0 2024-03-26T06:21:12.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f10fa7fc700 1 -- 172.21.15.37:0/557304792 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec022d00 con 0x7f10fc1071a0 2024-03-26T06:21:12.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.187+0000 7f110329f700 1 -- 172.21.15.37:0/557304792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10fc1a5140 con 0x7f10fc1071a0 2024-03-26T06:21:12.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.188+0000 7f110329f700 1 -- 172.21.15.37:0/557304792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10fc069fd0 con 0x7f10fc1071a0 2024-03-26T06:21:12.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.189+0000 7f10fa7fc700 1 -- 172.21.15.37:0/557304792 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f10ec02a070 con 0x7f10fc1071a0 2024-03-26T06:21:12.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.190+0000 7f10fa7fc700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f10e40a2920 0x7f10e40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:12.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.190+0000 7f110103b700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f10e40a2920 0x7f10e40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:12.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.190+0000 7f10fa7fc700 1 -- 172.21.15.37:0/557304792 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f10ec0d01c0 con 0x7f10fc1071a0 2024-03-26T06:21:12.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.191+0000 7f110103b700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f10e40a2920 0x7f10e40a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f10f400a400 tx=0x7f10f400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:12.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.194+0000 7f10fa7fc700 1 -- 172.21.15.37:0/557304792 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f10ec031020 con 0x7f10fc1071a0 2024-03-26T06:21:12.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.396+0000 7f110329f700 1 -- 172.21.15.37:0/557304792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10fc1a5310 con 0x7f10fc1071a0 2024-03-26T06:21:12.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.398+0000 7f10fa7fc700 1 -- 172.21.15.37:0/557304792 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22145 (secure 0 0 0) 0x7f10ec013070 con 0x7f10fc1071a0 2024-03-26T06:21:12.399 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:21:12.399 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":85,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:20:55.279252+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:21:12.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 -- 172.21.15.37:0/557304792 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f10e40a2920 msgr2=0x7f10e40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:12.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f10e40a2920 0x7f10e40a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f10f400a400 tx=0x7f10f400d040 comp rx=0 tx=0).stop 2024-03-26T06:21:12.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 -- 172.21.15.37:0/557304792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc1071a0 msgr2=0x7f10fc19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:12.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc1071a0 0x7f10fc19ce90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0057c0 tx=0x7f10ec00bfb0 comp rx=0 tx=0).stop 2024-03-26T06:21:12.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 -- 172.21.15.37:0/557304792 shutdown_connections 2024-03-26T06:21:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10fc1071a0 0x7f10fc19ce90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f10e40a2920 0x7f10e40a4de0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10fc100980 0x7f10fc1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 --2- 172.21.15.37:0/557304792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10fc101330 0x7f10fc198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 -- 172.21.15.37:0/557304792 >> 172.21.15.37:0/557304792 conn(0x7f10fc076020 msgr2=0x7f10fc0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 -- 172.21.15.37:0/557304792 shutdown_connections 2024-03-26T06:21:12.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:12.400+0000 7f110329f700 1 -- 172.21.15.37:0/557304792 wait complete. 2024-03-26T06:21:12.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:12 smithi149 ceph-mon[36681]: pgmap v297: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-03-26T06:21:12.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:12 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/557304792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:21:12.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:12 smithi037 ceph-mon[31974]: pgmap v297: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-03-26T06:21:12.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:12 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/557304792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:21:13.029 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-26T06:21:13.030 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-26T06:21:13.030 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-26T06:21:13.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-26T06:21:13.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-26T06:21:13.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-26T06:21:13.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-26T06:21:13.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-26T06:21:13.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-26T06:21:13.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-26T06:21:13.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-26T06:21:13.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-26T06:21:13.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:12 smithi066 ceph-mon[36504]: pgmap v297: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-03-26T06:21:13.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:12 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/557304792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:21:13.708 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:13.716 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:14.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:14 smithi037 ceph-mon[31974]: pgmap v298: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.1 KiB/s wr, 21 op/s 2024-03-26T06:21:15.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:14 smithi066 ceph-mon[36504]: pgmap v298: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.1 KiB/s wr, 21 op/s 2024-03-26T06:21:15.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:14 smithi149 ceph-mon[36681]: pgmap v298: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.1 KiB/s wr, 21 op/s 2024-03-26T06:21:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:15.974 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:15.975 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:15.975 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:15.975 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:15.977 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:15.977 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:15.979 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:15.983 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:16.888 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:16 smithi037 ceph-mon[31974]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-03-26T06:21:17.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:16 smithi066 ceph-mon[36504]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-03-26T06:21:17.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:16 smithi149 ceph-mon[36681]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-03-26T06:21:18.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.301+0000 7fd8ad963700 1 -- 172.21.15.37:0/2415677691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a8100ce0 msgr2=0x7fd8a8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:18.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.301+0000 7fd8ad963700 1 --2- 172.21.15.37:0/2415677691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a8100ce0 0x7fd8a8101140 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009990 tx=0x7fd89c009ca0 comp rx=0 tx=0).stop 2024-03-26T06:21:18.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.301+0000 7fd8ad963700 1 -- 172.21.15.37:0/2415677691 shutdown_connections 2024-03-26T06:21:18.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.301+0000 7fd8ad963700 1 --2- 172.21.15.37:0/2415677691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a8101680 0x7fd8a810db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:18.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.301+0000 7fd8ad963700 1 --2- 172.21.15.37:0/2415677691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8a8106d20 0x7fd8a8107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:18.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.301+0000 7fd8ad963700 1 --2- 172.21.15.37:0/2415677691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a8100ce0 0x7fd8a8101140 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:18.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.301+0000 7fd8ad963700 1 -- 172.21.15.37:0/2415677691 >> 172.21.15.37:0/2415677691 conn(0x7fd8a8077c10 msgr2=0x7fd8a8078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:18.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.302+0000 7fd8ad963700 1 -- 172.21.15.37:0/2415677691 shutdown_connections 2024-03-26T06:21:18.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.302+0000 7fd8ad963700 1 -- 172.21.15.37:0/2415677691 wait complete. 2024-03-26T06:21:18.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8ad963700 1 Processor -- start 2024-03-26T06:21:18.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8ad963700 1 -- start start 2024-03-26T06:21:18.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8ad963700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a8100ce0 0x7fd8a81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:18.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8ad963700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8a8101680 0x7fd8a8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:18.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8ad963700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a8106d20 0x7fd8a819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:18.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8ad963700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8110f80 con 0x7fd8a8106d20 2024-03-26T06:21:18.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8ad963700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8110e00 con 0x7fd8a8100ce0 2024-03-26T06:21:18.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8ad963700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8111100 con 0x7fd8a8101680 2024-03-26T06:21:18.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a8106d20 0x7fd8a819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:18.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a8106d20 0x7fd8a819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57406/0 (socket says 172.21.15.37:57406) 2024-03-26T06:21:18.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8a6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a8100ce0 0x7fd8a81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:18.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8a77fe700 1 -- 172.21.15.37:0/3256011005 learned_addr learned my addr 172.21.15.37:0/3256011005 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:18.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.303+0000 7fd8a67fc700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8a8101680 0x7fd8a8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:18.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8a77fe700 1 -- 172.21.15.37:0/3256011005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8a8101680 msgr2=0x7fd8a8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:18.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8a77fe700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8a8101680 0x7fd8a8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:18.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8a77fe700 1 -- 172.21.15.37:0/3256011005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a8100ce0 msgr2=0x7fd8a81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:18.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8a77fe700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8a8100ce0 0x7fd8a81985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:18.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8a77fe700 1 -- 172.21.15.37:0/3256011005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a819d580 con 0x7fd8a8106d20 2024-03-26T06:21:18.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8a67fc700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8a8101680 0x7fd8a8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:21:18.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8a77fe700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a8106d20 0x7fd8a819ce40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0007ee0 tx=0x7fd8a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:18.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8ac961700 1 -- 172.21.15.37:0/3256011005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000dd20 con 0x7fd8a8106d20 2024-03-26T06:21:18.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8ad963700 1 -- 172.21.15.37:0/3256011005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a819d870 con 0x7fd8a8106d20 2024-03-26T06:21:18.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8ad963700 1 -- 172.21.15.37:0/3256011005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a8107b30 con 0x7fd8a8106d20 2024-03-26T06:21:18.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8ac961700 1 -- 172.21.15.37:0/3256011005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8a0004020 con 0x7fd8a8106d20 2024-03-26T06:21:18.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.304+0000 7fd8ac961700 1 -- 172.21.15.37:0/3256011005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0015660 con 0x7fd8a8106d20 2024-03-26T06:21:18.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.305+0000 7fd8ad963700 1 -- 172.21.15.37:0/3256011005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd894001030 con 0x7fd8a8106d20 2024-03-26T06:21:18.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.307+0000 7fd8ac961700 1 -- 172.21.15.37:0/3256011005 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fd8a0003a00 con 0x7fd8a8106d20 2024-03-26T06:21:18.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.308+0000 7fd8ac961700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd8900a29a0 0x7fd8900a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:18.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.308+0000 7fd8a6ffd700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd8900a29a0 0x7fd8900a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:18.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.308+0000 7fd8ac961700 1 -- 172.21.15.37:0/3256011005 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd8a00cefb0 con 0x7fd8a8106d20 2024-03-26T06:21:18.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.308+0000 7fd8ac961700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917] conn(0x7fd8900ac1e0 0x7fd8900ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:18.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.308+0000 7fd8ac961700 1 -- 172.21.15.37:0/3256011005 --> [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8a0012ec0 con 0x7fd8900ac1e0 2024-03-26T06:21:18.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.308+0000 7fd8ac961700 1 -- 172.21.15.37:0/3256011005 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8a00cf360 con 0x7fd8a8106d20 2024-03-26T06:21:18.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.308+0000 7fd8a67fc700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917] conn(0x7fd8900ac1e0 0x7fd8900ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:18.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.308+0000 7fd8a6ffd700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd8900a29a0 0x7fd8900a4e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd898009be0 tx=0x7fd898009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:18.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.309+0000 7fd8a67fc700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917] conn(0x7fd8900ac1e0 0x7fd8900ae600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:18.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.309+0000 7fd8ac961700 1 -- 172.21.15.37:0/3256011005 <== osd.6 v2:172.21.15.37:6818/1971653917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd8a0012ec0 con 0x7fd8900ac1e0 2024-03-26T06:21:18.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.326+0000 7fd8ad963700 1 -- 172.21.15.37:0/3256011005 --> [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd894002e60 con 0x7fd8900ac1e0 2024-03-26T06:21:18.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.326+0000 7fd8ac961700 1 -- 172.21.15.37:0/3256011005 <== osd.6 v2:172.21.15.37:6818/1971653917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd894002e60 con 0x7fd8900ac1e0 2024-03-26T06:21:18.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.327+0000 7fd8ad963700 1 -- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917] conn(0x7fd8900ac1e0 msgr2=0x7fd8900ae600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:18.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.327+0000 7fd8ad963700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917] conn(0x7fd8900ac1e0 0x7fd8900ae600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:18.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.327+0000 7fd8ad963700 1 -- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd8900a29a0 msgr2=0x7fd8900a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:18.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.327+0000 7fd8ad963700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd8900a29a0 0x7fd8900a4e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd898009be0 tx=0x7fd898009260 comp rx=0 tx=0).stop 2024-03-26T06:21:18.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.327+0000 7fd8ad963700 1 -- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a8106d20 msgr2=0x7fd8a819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:18.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.327+0000 7fd8ad963700 1 --2- 172.21.15.37:0/3256011005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8a8106d20 0x7fd8a819ce40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0007ee0 tx=0x7fd8a000d3b0 comp rx=0 tx=0).stop 2024-03-26T06:21:18.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.327+0000 7fd8a77fe700 1 -- 172.21.15.37:0/3256011005 reap_dead start 2024-03-26T06:21:18.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.327+0000 7fd8ad963700 1 -- 172.21.15.37:0/3256011005 shutdown_connections 2024-03-26T06:21:18.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.327+0000 7fd8ad963700 1 -- 172.21.15.37:0/3256011005 >> 172.21.15.37:0/3256011005 conn(0x7fd8a8077c10 msgr2=0x7fd8a80fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:18.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.327+0000 7fd8ad963700 1 -- 172.21.15.37:0/3256011005 shutdown_connections 2024-03-26T06:21:18.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:18.327+0000 7fd8ad963700 1 -- 172.21.15.37:0/3256011005 wait complete. 2024-03-26T06:21:18.874 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:18 smithi037 ceph-mon[31974]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-03-26T06:21:19.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:18 smithi066 ceph-mon[36504]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-03-26T06:21:19.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:18 smithi149 ceph-mon[36681]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-03-26T06:21:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.528+0000 7ff3097ab700 1 -- 172.21.15.37:0/3669551293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff304101e30 msgr2=0x7ff3041022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.528+0000 7ff3097ab700 1 --2- 172.21.15.37:0/3669551293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff304101e30 0x7ff3041022b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009990 tx=0x7ff2f8009ca0 comp rx=0 tx=0).stop 2024-03-26T06:21:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.528+0000 7ff3097ab700 1 -- 172.21.15.37:0/3669551293 shutdown_connections 2024-03-26T06:21:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.528+0000 7ff3097ab700 1 --2- 172.21.15.37:0/3669551293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3041027f0 0x7ff30410ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.528+0000 7ff3097ab700 1 --2- 172.21.15.37:0/3669551293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff304107ee0 0x7ff3041082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.528+0000 7ff3097ab700 1 --2- 172.21.15.37:0/3669551293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff304101e30 0x7ff3041022b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.528+0000 7ff3097ab700 1 -- 172.21.15.37:0/3669551293 >> 172.21.15.37:0/3669551293 conn(0x7ff304078f10 msgr2=0x7ff304079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.528+0000 7ff3097ab700 1 -- 172.21.15.37:0/3669551293 shutdown_connections 2024-03-26T06:21:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.528+0000 7ff3097ab700 1 -- 172.21.15.37:0/3669551293 wait complete. 2024-03-26T06:21:20.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.529+0000 7ff3097ab700 1 Processor -- start 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.529+0000 7ff3097ab700 1 -- start start 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.529+0000 7ff3097ab700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff304101e30 0x7ff304198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.529+0000 7ff3097ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3041027f0 0x7ff304198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.529+0000 7ff3097ab700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff304107ee0 0x7ff30419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.529+0000 7ff3097ab700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff304112120 con 0x7ff3041027f0 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.529+0000 7ff3097ab700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff304111fa0 con 0x7ff304101e30 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.529+0000 7ff3097ab700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041122a0 con 0x7ff304107ee0 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3041027f0 0x7ff304198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3041027f0 0x7ff304198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57436/0 (socket says 172.21.15.37:57436) 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3027fc700 1 -- 172.21.15.37:0/3847538680 learned_addr learned my addr 172.21.15.37:0/3847538680 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff302ffd700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff304101e30 0x7ff304198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3037fe700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff304107ee0 0x7ff30419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3027fc700 1 -- 172.21.15.37:0/3847538680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff304107ee0 msgr2=0x7ff30419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3027fc700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff304107ee0 0x7ff30419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3027fc700 1 -- 172.21.15.37:0/3847538680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff304101e30 msgr2=0x7ff304198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3027fc700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff304101e30 0x7ff304198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3027fc700 1 -- 172.21.15.37:0/3847538680 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30419d510 con 0x7ff3041027f0 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff302ffd700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff304101e30 0x7ff304198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3027fc700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3041027f0 0x7ff304198a90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8000c00 tx=0x7ff2f8003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff2ebfff700 1 -- 172.21.15.37:0/3847538680 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8027070 con 0x7ff3041027f0 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3097ab700 1 -- 172.21.15.37:0/3847538680 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30419d7a0 con 0x7ff3041027f0 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff2ebfff700 1 -- 172.21.15.37:0/3847538680 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2f8004190 con 0x7ff3041027f0 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff2ebfff700 1 -- 172.21.15.37:0/3847538680 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f800f460 con 0x7ff3041027f0 2024-03-26T06:21:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.530+0000 7ff3097ab700 1 -- 172.21.15.37:0/3847538680 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff304074010 con 0x7ff3041027f0 2024-03-26T06:21:20.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.531+0000 7ff3097ab700 1 -- 172.21.15.37:0/3847538680 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3040624b0 con 0x7ff3041027f0 2024-03-26T06:21:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.532+0000 7ff2ebfff700 1 -- 172.21.15.37:0/3847538680 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff2f802a070 con 0x7ff3041027f0 2024-03-26T06:21:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.533+0000 7ff2ebfff700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2ec0a29a0 0x7ff2ec0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.533+0000 7ff302ffd700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2ec0a29a0 0x7ff2ec0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.533+0000 7ff2ebfff700 1 -- 172.21.15.37:0/3847538680 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff2f80d0070 con 0x7ff3041027f0 2024-03-26T06:21:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.533+0000 7ff2ebfff700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667] conn(0x7ff2ec0ac180 0x7ff2ec0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.533+0000 7ff2ebfff700 1 -- 172.21.15.37:0/3847538680 --> [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2f800b0b0 con 0x7ff2ec0ac180 2024-03-26T06:21:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.533+0000 7ff2ebfff700 1 -- 172.21.15.37:0/3847538680 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2f80d0550 con 0x7ff3041027f0 2024-03-26T06:21:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.533+0000 7ff302ffd700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2ec0a29a0 0x7ff2ec0a4e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4005fd0 tx=0x7ff2f4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.534+0000 7ff3037fe700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667] conn(0x7ff2ec0ac180 0x7ff2ec0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.534+0000 7ff3037fe700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667] conn(0x7ff2ec0ac180 0x7ff2ec0ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:20.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.535+0000 7ff2ebfff700 1 -- 172.21.15.37:0/3847538680 <== osd.1 v2:172.21.15.66:6800/331563667 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff2f800b0b0 con 0x7ff2ec0ac180 2024-03-26T06:21:20.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.552+0000 7ff3097ab700 1 -- 172.21.15.37:0/3847538680 --> [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff3040628b0 con 0x7ff2ec0ac180 2024-03-26T06:21:20.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.552+0000 7ff2ebfff700 1 -- 172.21.15.37:0/3847538680 <== osd.1 v2:172.21.15.66:6800/331563667 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff3040628b0 con 0x7ff2ec0ac180 2024-03-26T06:21:20.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.552+0000 7ff3097ab700 1 -- 172.21.15.37:0/3847538680 >> [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667] conn(0x7ff2ec0ac180 msgr2=0x7ff2ec0ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:20.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.552+0000 7ff3097ab700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667] conn(0x7ff2ec0ac180 0x7ff2ec0ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:20.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.552+0000 7ff3097ab700 1 -- 172.21.15.37:0/3847538680 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2ec0a29a0 msgr2=0x7ff2ec0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:20.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.552+0000 7ff3097ab700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff2ec0a29a0 0x7ff2ec0a4e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4005fd0 tx=0x7ff2f4005e30 comp rx=0 tx=0).stop 2024-03-26T06:21:20.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.552+0000 7ff3097ab700 1 -- 172.21.15.37:0/3847538680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3041027f0 msgr2=0x7ff304198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:20.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.552+0000 7ff3097ab700 1 --2- 172.21.15.37:0/3847538680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3041027f0 0x7ff304198a90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8000c00 tx=0x7ff2f8003730 comp rx=0 tx=0).stop 2024-03-26T06:21:20.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.552+0000 7ff3037fe700 1 -- 172.21.15.37:0/3847538680 reap_dead start 2024-03-26T06:21:20.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.552+0000 7ff3097ab700 1 -- 172.21.15.37:0/3847538680 shutdown_connections 2024-03-26T06:21:20.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.552+0000 7ff3097ab700 1 -- 172.21.15.37:0/3847538680 >> 172.21.15.37:0/3847538680 conn(0x7ff304078f10 msgr2=0x7ff304100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:20.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.553+0000 7ff3097ab700 1 -- 172.21.15.37:0/3847538680 shutdown_connections 2024-03-26T06:21:20.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:20.553+0000 7ff3097ab700 1 -- 172.21.15.37:0/3847538680 wait complete. 2024-03-26T06:21:20.814 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:20 smithi037 ceph-mon[31974]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-03-26T06:21:21.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:20 smithi066 ceph-mon[36504]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-03-26T06:21:21.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:20 smithi149 ceph-mon[36681]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-03-26T06:21:22.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:22 smithi037 ceph-mon[31974]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-03-26T06:21:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:22 smithi066 ceph-mon[36504]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-03-26T06:21:23.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.136+0000 7f97644f0700 1 -- 172.21.15.37:0/692115085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f975c069a90 msgr2=0x7f975c069e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.136+0000 7f97644f0700 1 --2- 172.21.15.37:0/692115085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f975c069a90 0x7f975c069e70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9758009990 tx=0x7f9758009ca0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.136+0000 7f97644f0700 1 -- 172.21.15.37:0/692115085 shutdown_connections 2024-03-26T06:21:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.136+0000 7f97644f0700 1 --2- 172.21.15.37:0/692115085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f975c069a90 0x7f975c069e70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.136+0000 7f97644f0700 1 --2- 172.21.15.37:0/692115085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975c10e890 0x7f975c110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.136+0000 7f97644f0700 1 --2- 172.21.15.37:0/692115085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f975c06a3b0 0x7f975c10e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.136+0000 7f97644f0700 1 -- 172.21.15.37:0/692115085 >> 172.21.15.37:0/692115085 conn(0x7f975c077c10 msgr2=0x7f975c078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.136+0000 7f97644f0700 1 -- 172.21.15.37:0/692115085 shutdown_connections 2024-03-26T06:21:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.136+0000 7f97644f0700 1 -- 172.21.15.37:0/692115085 wait complete. 2024-03-26T06:21:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.137+0000 7f97644f0700 1 Processor -- start 2024-03-26T06:21:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.137+0000 7f97644f0700 1 -- start start 2024-03-26T06:21:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.137+0000 7f97644f0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f975c069a90 0x7f975c1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.137+0000 7f97644f0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f975c06a3b0 0x7f975c1a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.137+0000 7f97644f0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975c10e890 0x7f975c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.137+0000 7f97644f0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c114340 con 0x7f975c069a90 2024-03-26T06:21:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.137+0000 7f97644f0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c1141c0 con 0x7f975c10e890 2024-03-26T06:21:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.137+0000 7f97644f0700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c1144c0 con 0x7f975c06a3b0 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.137+0000 7f976228c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f975c069a90 0x7f975c1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.137+0000 7f976228c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f975c069a90 0x7f975c1a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57476/0 (socket says 172.21.15.37:57476) 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.137+0000 7f976228c700 1 -- 172.21.15.37:0/1310255499 learned_addr learned my addr 172.21.15.37:0/1310255499 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f9761a8b700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f975c06a3b0 0x7f975c1a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f976228c700 1 -- 172.21.15.37:0/1310255499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f975c06a3b0 msgr2=0x7f975c1a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f976228c700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f975c06a3b0 0x7f975c1a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f976228c700 1 -- 172.21.15.37:0/1310255499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975c10e890 msgr2=0x7f975c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f976228c700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f975c10e890 0x7f975c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f976228c700 1 -- 172.21.15.37:0/1310255499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c19ad40 con 0x7f975c069a90 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f9761a8b700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f975c06a3b0 0x7f975c1a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f976228c700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f975c069a90 0x7f975c1a0420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9758000c00 tx=0x7f97580047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f974f7fe700 1 -- 172.21.15.37:0/1310255499 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758027070 con 0x7f975c069a90 2024-03-26T06:21:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f974f7fe700 1 -- 172.21.15.37:0/1310255499 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9758004050 con 0x7f975c069a90 2024-03-26T06:21:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f974f7fe700 1 -- 172.21.15.37:0/1310255499 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975800f460 con 0x7f975c069a90 2024-03-26T06:21:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f97644f0700 1 -- 172.21.15.37:0/1310255499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975c19afd0 con 0x7f975c069a90 2024-03-26T06:21:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.138+0000 7f97644f0700 1 -- 172.21.15.37:0/1310255499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c1a72f0 con 0x7f975c069a90 2024-03-26T06:21:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.139+0000 7f97644f0700 1 -- 172.21.15.37:0/1310255499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f975c062570 con 0x7f975c069a90 2024-03-26T06:21:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.140+0000 7f974f7fe700 1 -- 172.21.15.37:0/1310255499 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f975802a070 con 0x7f975c069a90 2024-03-26T06:21:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.141+0000 7f974f7fe700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f97480a2920 0x7f97480a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.141+0000 7f9761a8b700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f97480a2920 0x7f97480a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.141+0000 7f974f7fe700 1 -- 172.21.15.37:0/1310255499 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f97580d0180 con 0x7f975c069a90 2024-03-26T06:21:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.141+0000 7f974f7fe700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875] conn(0x7f97480ac100 0x7f97480ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.141+0000 7f974f7fe700 1 -- 172.21.15.37:0/1310255499 --> [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97580043d0 con 0x7f97480ac100 2024-03-26T06:21:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.141+0000 7f974f7fe700 1 -- 172.21.15.37:0/1310255499 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97580d0530 con 0x7f975c069a90 2024-03-26T06:21:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.141+0000 7f9761a8b700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f97480a2920 0x7f97480a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9750005fd0 tx=0x7f9750005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.141+0000 7f9762a8d700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875] conn(0x7f97480ac100 0x7f97480ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.142+0000 7f9762a8d700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875] conn(0x7f97480ac100 0x7f97480ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.143+0000 7f974f7fe700 1 -- 172.21.15.37:0/1310255499 <== osd.7 v2:172.21.15.66:6816/1024919875 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f97580043d0 con 0x7f97480ac100 2024-03-26T06:21:23.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:22 smithi149 ceph-mon[36681]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-03-26T06:21:23.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.159+0000 7f97644f0700 1 -- 172.21.15.37:0/1310255499 --> [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f975c0629f0 con 0x7f97480ac100 2024-03-26T06:21:23.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f974f7fe700 1 -- 172.21.15.37:0/1310255499 <== osd.7 v2:172.21.15.66:6816/1024919875 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f975c0629f0 con 0x7f97480ac100 2024-03-26T06:21:23.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f97644f0700 1 -- 172.21.15.37:0/1310255499 >> [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875] conn(0x7f97480ac100 msgr2=0x7f97480ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f97644f0700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875] conn(0x7f97480ac100 0x7f97480ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f97644f0700 1 -- 172.21.15.37:0/1310255499 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f97480a2920 msgr2=0x7f97480a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f97644f0700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f97480a2920 0x7f97480a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9750005fd0 tx=0x7f9750005ee0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f97644f0700 1 -- 172.21.15.37:0/1310255499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f975c069a90 msgr2=0x7f975c1a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f97644f0700 1 --2- 172.21.15.37:0/1310255499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f975c069a90 0x7f975c1a0420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9758000c00 tx=0x7f97580047b0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f9762a8d700 1 -- 172.21.15.37:0/1310255499 reap_dead start 2024-03-26T06:21:23.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f97644f0700 1 -- 172.21.15.37:0/1310255499 shutdown_connections 2024-03-26T06:21:23.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f97644f0700 1 -- 172.21.15.37:0/1310255499 >> 172.21.15.37:0/1310255499 conn(0x7f975c077c10 msgr2=0x7f975c10fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:23.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f97644f0700 1 -- 172.21.15.37:0/1310255499 shutdown_connections 2024-03-26T06:21:23.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.160+0000 7f97644f0700 1 -- 172.21.15.37:0/1310255499 wait complete. 2024-03-26T06:21:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.051+0000 7fcc894bd700 1 -- 172.21.15.37:0/3242724459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc84107120 msgr2=0x7fcc8410ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.051+0000 7fcc894bd700 1 --2- 172.21.15.37:0/3242724459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc84107120 0x7fcc8410ecf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00b0d0 tx=0x7fcc7c00b3e0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.051+0000 7fcc894bd700 1 -- 172.21.15.37:0/3242724459 shutdown_connections 2024-03-26T06:21:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.051+0000 7fcc894bd700 1 --2- 172.21.15.37:0/3242724459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc84107120 0x7fcc8410ecf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.051+0000 7fcc894bd700 1 --2- 172.21.15.37:0/3242724459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc84102040 0x7fcc84102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.051+0000 7fcc894bd700 1 --2- 172.21.15.37:0/3242724459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc841029f0 0x7fcc84106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.051+0000 7fcc894bd700 1 -- 172.21.15.37:0/3242724459 >> 172.21.15.37:0/3242724459 conn(0x7fcc840fd8b0 msgr2=0x7fcc840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.051+0000 7fcc894bd700 1 -- 172.21.15.37:0/3242724459 shutdown_connections 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.051+0000 7fcc894bd700 1 -- 172.21.15.37:0/3242724459 wait complete. 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.052+0000 7fcc894bd700 1 Processor -- start 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.052+0000 7fcc894bd700 1 -- start start 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc894bd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc84102040 0x7fcc840740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc894bd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc841029f0 0x7fcc84074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc894bd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc84107120 0x7fcc84074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc894bd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc84112110 con 0x7fcc841029f0 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc894bd700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc84111f90 con 0x7fcc84102040 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc894bd700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc84112290 con 0x7fcc84107120 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc827fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc841029f0 0x7fcc84074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc827fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc841029f0 0x7fcc84074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57460/0 (socket says 172.21.15.37:57460) 2024-03-26T06:21:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc827fc700 1 -- 172.21.15.37:0/736957464 learned_addr learned my addr 172.21.15.37:0/736957464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc82ffd700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc84102040 0x7fcc840740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc837fe700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc84107120 0x7fcc84074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc827fc700 1 -- 172.21.15.37:0/736957464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc84107120 msgr2=0x7fcc84074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc827fc700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc84107120 0x7fcc84074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc827fc700 1 -- 172.21.15.37:0/736957464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc84102040 msgr2=0x7fcc840740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc827fc700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc84102040 0x7fcc840740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc827fc700 1 -- 172.21.15.37:0/736957464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc841a6a70 con 0x7fcc841029f0 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.053+0000 7fcc827fc700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc841029f0 0x7fcc84074620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00d980 tx=0x7fcc6c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.054+0000 7fcc7bfff700 1 -- 172.21.15.37:0/736957464 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c017070 con 0x7fcc841029f0 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.054+0000 7fcc894bd700 1 -- 172.21.15.37:0/736957464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc841a6d60 con 0x7fcc841029f0 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.054+0000 7fcc7bfff700 1 -- 172.21.15.37:0/736957464 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc6c00fd70 con 0x7fcc841029f0 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.054+0000 7fcc7bfff700 1 -- 172.21.15.37:0/736957464 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00e3b0 con 0x7fcc841029f0 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.054+0000 7fcc894bd700 1 -- 172.21.15.37:0/736957464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc841a7260 con 0x7fcc841029f0 2024-03-26T06:21:23.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.055+0000 7fcc894bd700 1 -- 172.21.15.37:0/736957464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc840624f0 con 0x7fcc841029f0 2024-03-26T06:21:23.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.056+0000 7fcc7bfff700 1 -- 172.21.15.37:0/736957464 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fcc6c00f890 con 0x7fcc841029f0 2024-03-26T06:21:23.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.056+0000 7fcc7bfff700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcc700a2920 0x7fcc700a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:23.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.056+0000 7fcc82ffd700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcc700a2920 0x7fcc700a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:23.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.057+0000 7fcc7bfff700 1 -- 172.21.15.37:0/736957464 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcc6c0ce7f0 con 0x7fcc841029f0 2024-03-26T06:21:23.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.057+0000 7fcc7bfff700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283] conn(0x7fcc700ac100 0x7fcc700ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:23.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.057+0000 7fcc7bfff700 1 -- 172.21.15.37:0/736957464 --> [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc6c004500 con 0x7fcc700ac100 2024-03-26T06:21:23.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.057+0000 7fcc7bfff700 1 -- 172.21.15.37:0/736957464 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc6c0cecd0 con 0x7fcc841029f0 2024-03-26T06:21:23.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.057+0000 7fcc82ffd700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcc700a2920 0x7fcc700a4de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcc74007be0 tx=0x7fcc740073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:23.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.057+0000 7fcc837fe700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283] conn(0x7fcc700ac100 0x7fcc700ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:23.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.057+0000 7fcc837fe700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283] conn(0x7fcc700ac100 0x7fcc700ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.058+0000 7fcc7bfff700 1 -- 172.21.15.37:0/736957464 <== osd.8 v2:172.21.15.149:6816/192122283 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcc6c004500 con 0x7fcc700ac100 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.075+0000 7fcc894bd700 1 -- 172.21.15.37:0/736957464 --> [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc84062970 con 0x7fcc700ac100 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.075+0000 7fcc7bfff700 1 -- 172.21.15.37:0/736957464 <== osd.8 v2:172.21.15.149:6816/192122283 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcc84062970 con 0x7fcc700ac100 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.075+0000 7fcc894bd700 1 -- 172.21.15.37:0/736957464 >> [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283] conn(0x7fcc700ac100 msgr2=0x7fcc700ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.075+0000 7fcc894bd700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283] conn(0x7fcc700ac100 0x7fcc700ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.075+0000 7fcc894bd700 1 -- 172.21.15.37:0/736957464 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcc700a2920 msgr2=0x7fcc700a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.075+0000 7fcc894bd700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcc700a2920 0x7fcc700a4de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcc74007be0 tx=0x7fcc740073d0 comp rx=0 tx=0).stop 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.075+0000 7fcc894bd700 1 -- 172.21.15.37:0/736957464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc841029f0 msgr2=0x7fcc84074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.075+0000 7fcc894bd700 1 --2- 172.21.15.37:0/736957464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc841029f0 0x7fcc84074620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00d980 tx=0x7fcc6c00dc90 comp rx=0 tx=0).stop 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.076+0000 7fcc837fe700 1 -- 172.21.15.37:0/736957464 reap_dead start 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.076+0000 7fcc894bd700 1 -- 172.21.15.37:0/736957464 shutdown_connections 2024-03-26T06:21:23.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.076+0000 7fcc894bd700 1 -- 172.21.15.37:0/736957464 >> 172.21.15.37:0/736957464 conn(0x7fcc840fd8b0 msgr2=0x7fcc840fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:23.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.076+0000 7fcc894bd700 1 -- 172.21.15.37:0/736957464 shutdown_connections 2024-03-26T06:21:23.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:23.076+0000 7fcc894bd700 1 -- 172.21.15.37:0/736957464 wait complete. 2024-03-26T06:21:24.003 INFO:teuthology.orchestra.run.smithi037.stdout:197568495660 2024-03-26T06:21:24.003 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-26T06:21:24.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:24 smithi037 ceph-mon[31974]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-26T06:21:25.005 INFO:teuthology.orchestra.run.smithi037.stdout:55834574916 2024-03-26T06:21:25.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-26T06:21:25.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:24 smithi066 ceph-mon[36504]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-26T06:21:25.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:24 smithi149 ceph-mon[36681]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-26T06:21:26.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.764+0000 7f701452d700 1 -- 172.21.15.37:0/3412036062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700c06ae00 msgr2=0x7f700c10e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:26.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.764+0000 7f701452d700 1 --2- 172.21.15.37:0/3412036062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700c06ae00 0x7f700c10e380 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009990 tx=0x7f6ffc009ca0 comp rx=0 tx=0).stop 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.764+0000 7f701452d700 1 -- 172.21.15.37:0/3412036062 shutdown_connections 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.764+0000 7f701452d700 1 --2- 172.21.15.37:0/3412036062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c10e8c0 0x7f700c110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.764+0000 7f701452d700 1 --2- 172.21.15.37:0/3412036062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c06a4e0 0x7f700c06a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.764+0000 7f701452d700 1 --2- 172.21.15.37:0/3412036062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700c06ae00 0x7f700c10e380 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.764+0000 7f701452d700 1 -- 172.21.15.37:0/3412036062 >> 172.21.15.37:0/3412036062 conn(0x7f700c076d00 msgr2=0x7f700c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.764+0000 7f701452d700 1 -- 172.21.15.37:0/3412036062 shutdown_connections 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.764+0000 7f701452d700 1 -- 172.21.15.37:0/3412036062 wait complete. 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.764+0000 7f701452d700 1 Processor -- start 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.764+0000 7f701452d700 1 -- start start 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f701452d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700c06a4e0 0x7f700c1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f701452d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c06ae00 0x7f700c1a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f701452d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c10e8c0 0x7f700c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f701452d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c1142e0 con 0x7f700c06a4e0 2024-03-26T06:21:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f701452d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c114160 con 0x7f700c06ae00 2024-03-26T06:21:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f701452d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c114460 con 0x7f700c10e8c0 2024-03-26T06:21:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70122c9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700c06a4e0 0x7f700c1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70122c9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700c06a4e0 0x7f700c1a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57494/0 (socket says 172.21.15.37:57494) 2024-03-26T06:21:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70122c9700 1 -- 172.21.15.37:0/3606366172 learned_addr learned my addr 172.21.15.37:0/3606366172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f7011ac8700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c06ae00 0x7f700c1a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f7012aca700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c10e8c0 0x7f700c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70122c9700 1 -- 172.21.15.37:0/3606366172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c10e8c0 msgr2=0x7f700c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70122c9700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c10e8c0 0x7f700c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70122c9700 1 -- 172.21.15.37:0/3606366172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c06ae00 msgr2=0x7f700c1a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70122c9700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c06ae00 0x7f700c1a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70122c9700 1 -- 172.21.15.37:0/3606366172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c19ad40 con 0x7f700c06a4e0 2024-03-26T06:21:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f7012aca700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c10e8c0 0x7f700c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:21:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70122c9700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700c06a4e0 0x7f700c1a0420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f700800c9b0 tx=0x7f700800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70037fe700 1 -- 172.21.15.37:0/3606366172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008007b00 con 0x7f700c06a4e0 2024-03-26T06:21:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70037fe700 1 -- 172.21.15.37:0/3606366172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7008007ca0 con 0x7f700c06a4e0 2024-03-26T06:21:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f70037fe700 1 -- 172.21.15.37:0/3606366172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008018730 con 0x7f700c06a4e0 2024-03-26T06:21:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.765+0000 7f701452d700 1 -- 172.21.15.37:0/3606366172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c19b030 con 0x7f700c06a4e0 2024-03-26T06:21:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.766+0000 7f701452d700 1 -- 172.21.15.37:0/3606366172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c1a72f0 con 0x7f700c06a4e0 2024-03-26T06:21:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.766+0000 7f701452d700 1 -- 172.21.15.37:0/3606366172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f700c0624f0 con 0x7f700c06a4e0 2024-03-26T06:21:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.767+0000 7f70037fe700 1 -- 172.21.15.37:0/3606366172 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f70080188d0 con 0x7f700c06a4e0 2024-03-26T06:21:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.768+0000 7f70037fe700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6ff80a29a0 0x7f6ff80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.768+0000 7f7011ac8700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6ff80a29a0 0x7f6ff80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.768+0000 7f70037fe700 1 -- 172.21.15.37:0/3606366172 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f70080cedf0 con 0x7f700c06a4e0 2024-03-26T06:21:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.768+0000 7f70037fe700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230] conn(0x7f6ff80ac180 0x7f6ff80ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.768+0000 7f70037fe700 1 -- 172.21.15.37:0/3606366172 --> [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ff80aec50 con 0x7f6ff80ac180 2024-03-26T06:21:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.768+0000 7f70037fe700 1 -- 172.21.15.37:0/3606366172 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70080cf2d0 con 0x7f700c06a4e0 2024-03-26T06:21:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.768+0000 7f7012aca700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230] conn(0x7f6ff80ac180 0x7f6ff80ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.769+0000 7f7011ac8700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6ff80a29a0 0x7f6ff80a4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00b5c0 tx=0x7f6ffc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.769+0000 7f7012aca700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230] conn(0x7f6ff80ac180 0x7f6ff80ae5a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.769+0000 7f70037fe700 1 -- 172.21.15.37:0/3606366172 <== osd.0 v2:172.21.15.37:6802/362526230 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6ff80aec50 con 0x7f6ff80ac180 2024-03-26T06:21:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.786+0000 7f701452d700 1 -- 172.21.15.37:0/3606366172 --> [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f700c062970 con 0x7f6ff80ac180 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.786+0000 7f70037fe700 1 -- 172.21.15.37:0/3606366172 <== osd.0 v2:172.21.15.37:6802/362526230 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f700c062970 con 0x7f6ff80ac180 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.787+0000 7f701452d700 1 -- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230] conn(0x7f6ff80ac180 msgr2=0x7f6ff80ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.787+0000 7f701452d700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230] conn(0x7f6ff80ac180 0x7f6ff80ae5a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.787+0000 7f701452d700 1 -- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6ff80a29a0 msgr2=0x7f6ff80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.787+0000 7f701452d700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6ff80a29a0 0x7f6ff80a4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00b5c0 tx=0x7f6ffc005960 comp rx=0 tx=0).stop 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.787+0000 7f701452d700 1 -- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700c06a4e0 msgr2=0x7f700c1a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.787+0000 7f701452d700 1 --2- 172.21.15.37:0/3606366172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700c06a4e0 0x7f700c1a0420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f700800c9b0 tx=0x7f700800ccc0 comp rx=0 tx=0).stop 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.787+0000 7f7012aca700 1 -- 172.21.15.37:0/3606366172 reap_dead start 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.787+0000 7f701452d700 1 -- 172.21.15.37:0/3606366172 shutdown_connections 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.787+0000 7f701452d700 1 -- 172.21.15.37:0/3606366172 >> 172.21.15.37:0/3606366172 conn(0x7f700c076d00 msgr2=0x7f700c10fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.787+0000 7f701452d700 1 -- 172.21.15.37:0/3606366172 shutdown_connections 2024-03-26T06:21:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:26.787+0000 7f701452d700 1 -- 172.21.15.37:0/3606366172 wait complete. 2024-03-26T06:21:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:26 smithi037 ceph-mon[31974]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-26T06:21:26.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:26 smithi149 ceph-mon[36681]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-26T06:21:27.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:26 smithi066 ceph-mon[36504]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-26T06:21:28.379 INFO:teuthology.orchestra.run.smithi037.stdout:253403070498 2024-03-26T06:21:28.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-26T06:21:28.820 INFO:teuthology.orchestra.run.smithi037.stdout:219043332136 2024-03-26T06:21:28.821 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-26T06:21:28.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:28 smithi037 ceph-mon[31974]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:21:29.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:28 smithi066 ceph-mon[36504]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:21:29.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:28 smithi149 ceph-mon[36681]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:21:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.753+0000 7f537600d700 1 -- 172.21.15.37:0/2933187564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5370107100 msgr2=0x7f537010ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.753+0000 7f537600d700 1 --2- 172.21.15.37:0/2933187564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5370107100 0x7f537010ecd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f536800b0d0 tx=0x7f536800b3e0 comp rx=0 tx=0).stop 2024-03-26T06:21:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.753+0000 7f537600d700 1 -- 172.21.15.37:0/2933187564 shutdown_connections 2024-03-26T06:21:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.753+0000 7f537600d700 1 --2- 172.21.15.37:0/2933187564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5370107100 0x7f537010ecd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.753+0000 7f537600d700 1 --2- 172.21.15.37:0/2933187564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5370102020 0x7f5370102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.753+0000 7f537600d700 1 --2- 172.21.15.37:0/2933187564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53701029d0 0x7f5370106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.753+0000 7f537600d700 1 -- 172.21.15.37:0/2933187564 >> 172.21.15.37:0/2933187564 conn(0x7f53700fd890 msgr2=0x7f53700ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.754+0000 7f537600d700 1 -- 172.21.15.37:0/2933187564 shutdown_connections 2024-03-26T06:21:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.754+0000 7f537600d700 1 -- 172.21.15.37:0/2933187564 wait complete. 2024-03-26T06:21:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.754+0000 7f537600d700 1 Processor -- start 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.754+0000 7f537600d700 1 -- start start 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.754+0000 7f537600d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5370102020 0x7f5370198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f537600d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53701029d0 0x7f5370198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f537600d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5370107100 0x7f537019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f537600d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53701121b0 con 0x7f5370107100 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f537600d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5370112030 con 0x7f53701029d0 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f537600d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5370112330 con 0x7f5370102020 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5370107100 0x7f537019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5370107100 0x7f537019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50250/0 (socket says 172.21.15.37:50250) 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536ffff700 1 -- 172.21.15.37:0/1811346624 learned_addr learned my addr 172.21.15.37:0/1811346624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536f7fe700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5370102020 0x7f5370198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536effd700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53701029d0 0x7f5370198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:30.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536ffff700 1 -- 172.21.15.37:0/1811346624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5370102020 msgr2=0x7f5370198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:30.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536ffff700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5370102020 0x7f5370198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:30.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536ffff700 1 -- 172.21.15.37:0/1811346624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53701029d0 msgr2=0x7f5370198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:30.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536ffff700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53701029d0 0x7f5370198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:30.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536ffff700 1 -- 172.21.15.37:0/1811346624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537019d520 con 0x7f5370107100 2024-03-26T06:21:30.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536ffff700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5370107100 0x7f537019cde0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5368004dc0 tx=0x7f5368004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:30.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536cff9700 1 -- 172.21.15.37:0/1811346624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368021020 con 0x7f5370107100 2024-03-26T06:21:30.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536cff9700 1 -- 172.21.15.37:0/1811346624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5368007c00 con 0x7f5370107100 2024-03-26T06:21:30.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f537600d700 1 -- 172.21.15.37:0/1811346624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537019d7b0 con 0x7f5370107100 2024-03-26T06:21:30.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f536cff9700 1 -- 172.21.15.37:0/1811346624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368005370 con 0x7f5370107100 2024-03-26T06:21:30.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.755+0000 7f537600d700 1 -- 172.21.15.37:0/1811346624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53701a5090 con 0x7f5370107100 2024-03-26T06:21:30.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.756+0000 7f537600d700 1 -- 172.21.15.37:0/1811346624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f53700feca0 con 0x7f5370107100 2024-03-26T06:21:30.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.758+0000 7f536cff9700 1 -- 172.21.15.37:0/1811346624 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f5368024020 con 0x7f5370107100 2024-03-26T06:21:30.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.758+0000 7f536cff9700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f53580a2920 0x7f53580a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:30.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.758+0000 7f536f7fe700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f53580a2920 0x7f53580a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:30.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.758+0000 7f536cff9700 1 -- 172.21.15.37:0/1811346624 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f536800f020 con 0x7f5370107100 2024-03-26T06:21:30.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.758+0000 7f536cff9700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638] conn(0x7f53580ac100 0x7f53580ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:30.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.758+0000 7f536cff9700 1 -- 172.21.15.37:0/1811346624 --> [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53580aebd0 con 0x7f53580ac100 2024-03-26T06:21:30.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.758+0000 7f536cff9700 1 -- 172.21.15.37:0/1811346624 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53680d0370 con 0x7f5370107100 2024-03-26T06:21:30.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.759+0000 7f536f7fe700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f53580a2920 0x7f53580a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5360009a10 tx=0x7f5360009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:30.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.759+0000 7f536effd700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638] conn(0x7f53580ac100 0x7f53580ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:30.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.759+0000 7f536effd700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638] conn(0x7f53580ac100 0x7f53580ae520 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:30.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.760+0000 7f536cff9700 1 -- 172.21.15.37:0/1811346624 <== osd.5 v2:172.21.15.149:6808/3463585638 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f53580aebd0 con 0x7f53580ac100 2024-03-26T06:21:30.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.776+0000 7f537600d700 1 -- 172.21.15.37:0/1811346624 --> [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5370103500 con 0x7f53580ac100 2024-03-26T06:21:30.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.777+0000 7f536cff9700 1 -- 172.21.15.37:0/1811346624 <== osd.5 v2:172.21.15.149:6808/3463585638 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5370103500 con 0x7f53580ac100 2024-03-26T06:21:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.777+0000 7f537600d700 1 -- 172.21.15.37:0/1811346624 >> [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638] conn(0x7f53580ac100 msgr2=0x7f53580ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.777+0000 7f537600d700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638] conn(0x7f53580ac100 0x7f53580ae520 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.777+0000 7f537600d700 1 -- 172.21.15.37:0/1811346624 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f53580a2920 msgr2=0x7f53580a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.777+0000 7f537600d700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f53580a2920 0x7f53580a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5360009a10 tx=0x7f5360009190 comp rx=0 tx=0).stop 2024-03-26T06:21:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.777+0000 7f537600d700 1 -- 172.21.15.37:0/1811346624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5370107100 msgr2=0x7f537019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.777+0000 7f537600d700 1 --2- 172.21.15.37:0/1811346624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5370107100 0x7f537019cde0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5368004dc0 tx=0x7f5368004df0 comp rx=0 tx=0).stop 2024-03-26T06:21:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.777+0000 7f536ffff700 1 -- 172.21.15.37:0/1811346624 reap_dead start 2024-03-26T06:21:30.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.777+0000 7f537600d700 1 -- 172.21.15.37:0/1811346624 shutdown_connections 2024-03-26T06:21:30.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.777+0000 7f537600d700 1 -- 172.21.15.37:0/1811346624 >> 172.21.15.37:0/1811346624 conn(0x7f53700fd890 msgr2=0x7f53700fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:30.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.778+0000 7f537600d700 1 -- 172.21.15.37:0/1811346624 shutdown_connections 2024-03-26T06:21:30.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:30.778+0000 7f537600d700 1 -- 172.21.15.37:0/1811346624 wait complete. 2024-03-26T06:21:30.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:30 smithi037 ceph-mon[31974]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:21:31.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:30 smithi066 ceph-mon[36504]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:21:31.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:30 smithi149 ceph-mon[36681]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:21:31.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.362+0000 7f6d290e3700 1 -- 172.21.15.37:0/1463260907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d24100cc0 msgr2=0x7f6d24101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.362+0000 7f6d290e3700 1 --2- 172.21.15.37:0/1463260907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d24100cc0 0x7f6d24101140 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009990 tx=0x7f6d18009ca0 comp rx=0 tx=0).stop 2024-03-26T06:21:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.362+0000 7f6d290e3700 1 -- 172.21.15.37:0/1463260907 shutdown_connections 2024-03-26T06:21:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.362+0000 7f6d290e3700 1 --2- 172.21.15.37:0/1463260907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d24101680 0x7f6d2410db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.362+0000 7f6d290e3700 1 --2- 172.21.15.37:0/1463260907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d24106d70 0x7f6d24107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.362+0000 7f6d290e3700 1 --2- 172.21.15.37:0/1463260907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d24100cc0 0x7f6d24101140 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.362+0000 7f6d290e3700 1 -- 172.21.15.37:0/1463260907 >> 172.21.15.37:0/1463260907 conn(0x7f6d24077cf0 msgr2=0x7f6d24078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.362+0000 7f6d290e3700 1 -- 172.21.15.37:0/1463260907 shutdown_connections 2024-03-26T06:21:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.362+0000 7f6d290e3700 1 -- 172.21.15.37:0/1463260907 wait complete. 2024-03-26T06:21:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.363+0000 7f6d290e3700 1 Processor -- start 2024-03-26T06:21:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.363+0000 7f6d290e3700 1 -- start start 2024-03-26T06:21:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d290e3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d24100cc0 0x7f6d24199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d290e3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d24101680 0x7f6d24199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d290e3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d24106d70 0x7f6d241a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d290e3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d24110f90 con 0x7f6d24100cc0 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d290e3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d24110e10 con 0x7f6d24101680 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d290e3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d24111110 con 0x7f6d24106d70 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d22d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d24100cc0 0x7f6d24199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d22d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d24100cc0 0x7f6d24199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50264/0 (socket says 172.21.15.37:50264) 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d22d9d700 1 -- 172.21.15.37:0/3865243577 learned_addr learned my addr 172.21.15.37:0/3865243577 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d2259c700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d24101680 0x7f6d24199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d2359e700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d24106d70 0x7f6d241a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d22d9d700 1 -- 172.21.15.37:0/3865243577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d24106d70 msgr2=0x7f6d241a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d22d9d700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d24106d70 0x7f6d241a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d22d9d700 1 -- 172.21.15.37:0/3865243577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d24101680 msgr2=0x7f6d24199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d22d9d700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d24101680 0x7f6d24199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d22d9d700 1 -- 172.21.15.37:0/3865243577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d241a47f0 con 0x7f6d24100cc0 2024-03-26T06:21:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d22d9d700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d24100cc0 0x7f6d24199230 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400e9a0 tx=0x7f6d1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d0bfff700 1 -- 172.21.15.37:0/3865243577 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400cbd0 con 0x7f6d24100cc0 2024-03-26T06:21:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d290e3700 1 -- 172.21.15.37:0/3865243577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d241a4a80 con 0x7f6d24100cc0 2024-03-26T06:21:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d0bfff700 1 -- 172.21.15.37:0/3865243577 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d1400cd70 con 0x7f6d24100cc0 2024-03-26T06:21:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d0bfff700 1 -- 172.21.15.37:0/3865243577 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14018750 con 0x7f6d24100cc0 2024-03-26T06:21:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.364+0000 7f6d290e3700 1 -- 172.21.15.37:0/3865243577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d241a4f90 con 0x7f6d24100cc0 2024-03-26T06:21:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.365+0000 7f6d290e3700 1 -- 172.21.15.37:0/3865243577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d240624f0 con 0x7f6d24100cc0 2024-03-26T06:21:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.367+0000 7f6d0bfff700 1 -- 172.21.15.37:0/3865243577 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f6d14010430 con 0x7f6d24100cc0 2024-03-26T06:21:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.367+0000 7f6d0bfff700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6d0c0a2920 0x7f6d0c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.368+0000 7f6d2259c700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6d0c0a2920 0x7f6d0c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.368+0000 7f6d0bfff700 1 -- 172.21.15.37:0/3865243577 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d140cf590 con 0x7f6d24100cc0 2024-03-26T06:21:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.368+0000 7f6d0bfff700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060] conn(0x7f6d0c0ac100 0x7f6d0c0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.368+0000 7f6d0bfff700 1 -- 172.21.15.37:0/3865243577 --> [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d1401bec0 con 0x7f6d0c0ac100 2024-03-26T06:21:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.368+0000 7f6d0bfff700 1 -- 172.21.15.37:0/3865243577 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d140cf940 con 0x7f6d24100cc0 2024-03-26T06:21:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.368+0000 7f6d2359e700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060] conn(0x7f6d0c0ac100 0x7f6d0c0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.368+0000 7f6d2259c700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6d0c0a2920 0x7f6d0c0a4de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d180098e0 tx=0x7f6d18005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.368+0000 7f6d2359e700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060] conn(0x7f6d0c0ac100 0x7f6d0c0ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.369+0000 7f6d0bfff700 1 -- 172.21.15.37:0/3865243577 <== osd.3 v2:172.21.15.37:6810/2227972060 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6d1401bec0 con 0x7f6d0c0ac100 2024-03-26T06:21:31.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.387+0000 7f6d290e3700 1 -- 172.21.15.37:0/3865243577 --> [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6d24062970 con 0x7f6d0c0ac100 2024-03-26T06:21:31.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.387+0000 7f6d0bfff700 1 -- 172.21.15.37:0/3865243577 <== osd.3 v2:172.21.15.37:6810/2227972060 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6d24062970 con 0x7f6d0c0ac100 2024-03-26T06:21:31.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.388+0000 7f6d290e3700 1 -- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060] conn(0x7f6d0c0ac100 msgr2=0x7f6d0c0ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:31.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.388+0000 7f6d290e3700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060] conn(0x7f6d0c0ac100 0x7f6d0c0ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:31.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.388+0000 7f6d290e3700 1 -- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6d0c0a2920 msgr2=0x7f6d0c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:31.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.388+0000 7f6d290e3700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f6d0c0a2920 0x7f6d0c0a4de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d180098e0 tx=0x7f6d18005960 comp rx=0 tx=0).stop 2024-03-26T06:21:31.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.388+0000 7f6d290e3700 1 -- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d24100cc0 msgr2=0x7f6d24199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:31.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.388+0000 7f6d290e3700 1 --2- 172.21.15.37:0/3865243577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d24100cc0 0x7f6d24199230 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400e9a0 tx=0x7f6d1400ecb0 comp rx=0 tx=0).stop 2024-03-26T06:21:31.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.388+0000 7f6d2359e700 1 -- 172.21.15.37:0/3865243577 reap_dead start 2024-03-26T06:21:31.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.388+0000 7f6d290e3700 1 -- 172.21.15.37:0/3865243577 shutdown_connections 2024-03-26T06:21:31.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.388+0000 7f6d290e3700 1 -- 172.21.15.37:0/3865243577 >> 172.21.15.37:0/3865243577 conn(0x7f6d24077cf0 msgr2=0x7f6d240ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:31.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.388+0000 7f6d290e3700 1 -- 172.21.15.37:0/3865243577 shutdown_connections 2024-03-26T06:21:31.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:31.388+0000 7f6d290e3700 1 -- 172.21.15.37:0/3865243577 wait complete. 2024-03-26T06:21:32.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:32 smithi037 ceph-mon[31974]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-26T06:21:32.908 INFO:teuthology.orchestra.run.smithi037.stdout:163208757299 2024-03-26T06:21:32.909 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-26T06:21:33.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:32 smithi066 ceph-mon[36504]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-26T06:21:33.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:32 smithi149 ceph-mon[36681]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-26T06:21:33.600 INFO:teuthology.orchestra.run.smithi037.stdout:107374182461 2024-03-26T06:21:33.601 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-26T06:21:34.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.560+0000 7fc538c45700 1 -- 172.21.15.37:0/228874260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5341020f0 msgr2=0x7fc534105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:34.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.560+0000 7fc538c45700 1 --2- 172.21.15.37:0/228874260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5341020f0 0x7fc534105980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5280099c0 tx=0x7fc528009cd0 comp rx=0 tx=0).stop 2024-03-26T06:21:34.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.560+0000 7fc538c45700 1 -- 172.21.15.37:0/228874260 shutdown_connections 2024-03-26T06:21:34.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.560+0000 7fc538c45700 1 --2- 172.21.15.37:0/228874260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc534106060 0x7fc53410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:34.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.560+0000 7fc538c45700 1 --2- 172.21.15.37:0/228874260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc534101740 0x7fc534101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:34.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.560+0000 7fc538c45700 1 --2- 172.21.15.37:0/228874260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5341020f0 0x7fc534105980 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:34.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.560+0000 7fc538c45700 1 -- 172.21.15.37:0/228874260 >> 172.21.15.37:0/228874260 conn(0x7fc5340772e0 msgr2=0x7fc5340fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:34.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.560+0000 7fc538c45700 1 -- 172.21.15.37:0/228874260 shutdown_connections 2024-03-26T06:21:34.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.560+0000 7fc538c45700 1 -- 172.21.15.37:0/228874260 wait complete. 2024-03-26T06:21:34.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.561+0000 7fc538c45700 1 Processor -- start 2024-03-26T06:21:34.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.561+0000 7fc538c45700 1 -- start start 2024-03-26T06:21:34.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc538c45700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc534101740 0x7fc5341994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc538c45700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5341020f0 0x7fc5341999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc538c45700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc534106060 0x7fc5341a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc538c45700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5341110a0 con 0x7fc534106060 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc538c45700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc534110f20 con 0x7fc5341020f0 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc538c45700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc534111220 con 0x7fc534101740 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc532d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc534106060 0x7fc5341a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc532d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc534106060 0x7fc5341a4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50288/0 (socket says 172.21.15.37:50288) 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc532d9d700 1 -- 172.21.15.37:0/2558561842 learned_addr learned my addr 172.21.15.37:0/2558561842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc531d9b700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5341020f0 0x7fc5341999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc53259c700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc534101740 0x7fc5341994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc532d9d700 1 -- 172.21.15.37:0/2558561842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc534101740 msgr2=0x7fc5341994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc532d9d700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc534101740 0x7fc5341994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:34.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc532d9d700 1 -- 172.21.15.37:0/2558561842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5341020f0 msgr2=0x7fc5341999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:34.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc532d9d700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5341020f0 0x7fc5341999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:34.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc532d9d700 1 -- 172.21.15.37:0/2558561842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5341a4a60 con 0x7fc534106060 2024-03-26T06:21:34.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc532d9d700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc534106060 0x7fc5341a4520 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00b970 tx=0x7fc52c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:34.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.562+0000 7fc5237fe700 1 -- 172.21.15.37:0/2558561842 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c00d860 con 0x7fc534106060 2024-03-26T06:21:34.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.563+0000 7fc5237fe700 1 -- 172.21.15.37:0/2558561842 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc52c00e430 con 0x7fc534106060 2024-03-26T06:21:34.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.563+0000 7fc538c45700 1 -- 172.21.15.37:0/2558561842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5341a4c90 con 0x7fc534106060 2024-03-26T06:21:34.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.563+0000 7fc5237fe700 1 -- 172.21.15.37:0/2558561842 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c017660 con 0x7fc534106060 2024-03-26T06:21:34.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.563+0000 7fc538c45700 1 -- 172.21.15.37:0/2558561842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5341a51a0 con 0x7fc534106060 2024-03-26T06:21:34.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.563+0000 7fc538c45700 1 -- 172.21.15.37:0/2558561842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc534062470 con 0x7fc534106060 2024-03-26T06:21:34.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.565+0000 7fc5237fe700 1 -- 172.21.15.37:0/2558561842 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fc52c00e5e0 con 0x7fc534106060 2024-03-26T06:21:34.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.565+0000 7fc5237fe700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc51c0a29a0 0x7fc51c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:34.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.565+0000 7fc53259c700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc51c0a29a0 0x7fc51c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:34.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.565+0000 7fc5237fe700 1 -- 172.21.15.37:0/2558561842 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc52c0ce920 con 0x7fc534106060 2024-03-26T06:21:34.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.566+0000 7fc5237fe700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514] conn(0x7fc51c0ac180 0x7fc51c0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:34.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.566+0000 7fc5237fe700 1 -- 172.21.15.37:0/2558561842 --> [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc51c0aec50 con 0x7fc51c0ac180 2024-03-26T06:21:34.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.566+0000 7fc5237fe700 1 -- 172.21.15.37:0/2558561842 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc52c0cee00 con 0x7fc534106060 2024-03-26T06:21:34.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.566+0000 7fc53259c700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc51c0a29a0 0x7fc51c0a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc524005950 tx=0x7fc52400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:34.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.566+0000 7fc531d9b700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514] conn(0x7fc51c0ac180 0x7fc51c0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:34.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.566+0000 7fc531d9b700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514] conn(0x7fc51c0ac180 0x7fc51c0ae5a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:34.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.567+0000 7fc5237fe700 1 -- 172.21.15.37:0/2558561842 <== osd.4 v2:172.21.15.66:6808/1994195514 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc51c0aec50 con 0x7fc51c0ac180 2024-03-26T06:21:34.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.584+0000 7fc538c45700 1 -- 172.21.15.37:0/2558561842 --> [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc534062870 con 0x7fc51c0ac180 2024-03-26T06:21:34.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc5237fe700 1 -- 172.21.15.37:0/2558561842 <== osd.4 v2:172.21.15.66:6808/1994195514 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc534062870 con 0x7fc51c0ac180 2024-03-26T06:21:34.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc538c45700 1 -- 172.21.15.37:0/2558561842 >> [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514] conn(0x7fc51c0ac180 msgr2=0x7fc51c0ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:34.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc538c45700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514] conn(0x7fc51c0ac180 0x7fc51c0ae5a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:34.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc538c45700 1 -- 172.21.15.37:0/2558561842 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc51c0a29a0 msgr2=0x7fc51c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:34.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc538c45700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fc51c0a29a0 0x7fc51c0a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc524005950 tx=0x7fc52400b410 comp rx=0 tx=0).stop 2024-03-26T06:21:34.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc538c45700 1 -- 172.21.15.37:0/2558561842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc534106060 msgr2=0x7fc5341a4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:34.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc538c45700 1 --2- 172.21.15.37:0/2558561842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc534106060 0x7fc5341a4520 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00b970 tx=0x7fc52c00bc80 comp rx=0 tx=0).stop 2024-03-26T06:21:34.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc532d9d700 1 -- 172.21.15.37:0/2558561842 reap_dead start 2024-03-26T06:21:34.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc538c45700 1 -- 172.21.15.37:0/2558561842 shutdown_connections 2024-03-26T06:21:34.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc538c45700 1 -- 172.21.15.37:0/2558561842 >> 172.21.15.37:0/2558561842 conn(0x7fc5340772e0 msgr2=0x7fc534104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:34.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc538c45700 1 -- 172.21.15.37:0/2558561842 shutdown_connections 2024-03-26T06:21:34.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:34.585+0000 7fc538c45700 1 -- 172.21.15.37:0/2558561842 wait complete. 2024-03-26T06:21:34.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:34 smithi037 ceph-mon[31974]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-03-26T06:21:35.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:34 smithi066 ceph-mon[36504]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-03-26T06:21:35.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:34 smithi149 ceph-mon[36681]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-03-26T06:21:35.719 INFO:teuthology.orchestra.run.smithi037.stdout:141733920825 2024-03-26T06:21:35.720 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.907+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1347842157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d34101920 msgr2=0x7f5d341058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.907+0000 7f5d3c6e2700 1 --2- 172.21.15.37:0/1347842157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d34101920 0x7f5d341058e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5d240099c0 tx=0x7f5d24009cd0 comp rx=0 tx=0).stop 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.907+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1347842157 shutdown_connections 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.907+0000 7f5d3c6e2700 1 --2- 172.21.15.37:0/1347842157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d34105fc0 0x7f5d3410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.907+0000 7f5d3c6e2700 1 --2- 172.21.15.37:0/1347842157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d34100f70 0x7f5d34101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.907+0000 7f5d3c6e2700 1 --2- 172.21.15.37:0/1347842157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d34101920 0x7f5d341058e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.907+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1347842157 >> 172.21.15.37:0/1347842157 conn(0x7f5d34076d30 msgr2=0x7f5d34077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.907+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1347842157 shutdown_connections 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.907+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1347842157 wait complete. 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.908+0000 7f5d3c6e2700 1 Processor -- start 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.908+0000 7f5d3c6e2700 1 -- start start 2024-03-26T06:21:35.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.908+0000 7f5d3c6e2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d34100f70 0x7f5d34198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.908+0000 7f5d3c6e2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d34101920 0x7f5d34198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.908+0000 7f5d3c6e2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d34105fc0 0x7f5d3419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.908+0000 7f5d3c6e2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d34110f80 con 0x7f5d34100f70 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.908+0000 7f5d3c6e2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d34110e00 con 0x7f5d34105fc0 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.908+0000 7f5d3c6e2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d34111100 con 0x7f5d34101920 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.908+0000 7f5d3a47e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d34100f70 0x7f5d34198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.908+0000 7f5d3a47e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d34100f70 0x7f5d34198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50310/0 (socket says 172.21.15.37:50310) 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.908+0000 7f5d3a47e700 1 -- 172.21.15.37:0/1022802182 learned_addr learned my addr 172.21.15.37:0/1022802182 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d39c7d700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d34101920 0x7f5d34198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d3ac7f700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d34105fc0 0x7f5d3419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d3a47e700 1 -- 172.21.15.37:0/1022802182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d34101920 msgr2=0x7f5d34198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d3a47e700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d34101920 0x7f5d34198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d3a47e700 1 -- 172.21.15.37:0/1022802182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d34105fc0 msgr2=0x7f5d3419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d3a47e700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d34105fc0 0x7f5d3419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d3a47e700 1 -- 172.21.15.37:0/1022802182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3419d570 con 0x7f5d34100f70 2024-03-26T06:21:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d3a47e700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d34100f70 0x7f5d34198560 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c920 tx=0x7f5d3000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d2b7fe700 1 -- 172.21.15.37:0/1022802182 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30007b00 con 0x7f5d34100f70 2024-03-26T06:21:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d2b7fe700 1 -- 172.21.15.37:0/1022802182 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d30004d10 con 0x7f5d34100f70 2024-03-26T06:21:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1022802182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3419d860 con 0x7f5d34100f70 2024-03-26T06:21:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d2b7fe700 1 -- 172.21.15.37:0/1022802182 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30005680 con 0x7f5d34100f70 2024-03-26T06:21:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.909+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1022802182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d341a50e0 con 0x7f5d34100f70 2024-03-26T06:21:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.910+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1022802182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d34062530 con 0x7f5d34100f70 2024-03-26T06:21:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.911+0000 7f5d2b7fe700 1 -- 172.21.15.37:0/1022802182 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f5d30004750 con 0x7f5d34100f70 2024-03-26T06:21:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.912+0000 7f5d2b7fe700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5d200a2920 0x7f5d200a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.912+0000 7f5d39c7d700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5d200a2920 0x7f5d200a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.912+0000 7f5d2b7fe700 1 -- 172.21.15.37:0/1022802182 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5d300ce9c0 con 0x7f5d34100f70 2024-03-26T06:21:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.912+0000 7f5d2b7fe700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449] conn(0x7f5d200ac100 0x7f5d200ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.912+0000 7f5d2b7fe700 1 -- 172.21.15.37:0/1022802182 --> [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d30012ec0 con 0x7f5d200ac100 2024-03-26T06:21:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.912+0000 7f5d2b7fe700 1 -- 172.21.15.37:0/1022802182 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d300ceea0 con 0x7f5d34100f70 2024-03-26T06:21:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.912+0000 7f5d39c7d700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5d200a2920 0x7f5d200a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5d24000c00 tx=0x7f5d24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.913+0000 7f5d3ac7f700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449] conn(0x7f5d200ac100 0x7f5d200ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.913+0000 7f5d3ac7f700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449] conn(0x7f5d200ac100 0x7f5d200ae520 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.914+0000 7f5d2b7fe700 1 -- 172.21.15.37:0/1022802182 <== osd.11 v2:172.21.15.149:6824/2983393449 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5d30012ec0 con 0x7f5d200ac100 2024-03-26T06:21:35.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.930+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1022802182 --> [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d340629b0 con 0x7f5d200ac100 2024-03-26T06:21:35.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.931+0000 7f5d2b7fe700 1 -- 172.21.15.37:0/1022802182 <== osd.11 v2:172.21.15.149:6824/2983393449 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5d340629b0 con 0x7f5d200ac100 2024-03-26T06:21:35.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.931+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1022802182 >> [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449] conn(0x7f5d200ac100 msgr2=0x7f5d200ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:35.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.931+0000 7f5d3c6e2700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449] conn(0x7f5d200ac100 0x7f5d200ae520 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:35.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.931+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1022802182 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5d200a2920 msgr2=0x7f5d200a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:35.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.931+0000 7f5d3c6e2700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5d200a2920 0x7f5d200a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5d24000c00 tx=0x7f5d24005fb0 comp rx=0 tx=0).stop 2024-03-26T06:21:35.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.931+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1022802182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d34100f70 msgr2=0x7f5d34198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:35.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.931+0000 7f5d3c6e2700 1 --2- 172.21.15.37:0/1022802182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d34100f70 0x7f5d34198560 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c920 tx=0x7f5d3000cc30 comp rx=0 tx=0).stop 2024-03-26T06:21:35.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.931+0000 7f5d3ac7f700 1 -- 172.21.15.37:0/1022802182 reap_dead start 2024-03-26T06:21:35.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.932+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1022802182 shutdown_connections 2024-03-26T06:21:35.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.932+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1022802182 >> 172.21.15.37:0/1022802182 conn(0x7f5d34076d30 msgr2=0x7f5d3406a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:35.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.932+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1022802182 shutdown_connections 2024-03-26T06:21:35.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:35.932+0000 7f5d3c6e2700 1 -- 172.21.15.37:0/1022802182 wait complete. 2024-03-26T06:21:36.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.354+0000 7fcac766a700 1 -- 172.21.15.37:0/1553145082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac006a3b0 msgr2=0x7fcac010e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:36.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.354+0000 7fcac766a700 1 --2- 172.21.15.37:0/1553145082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac006a3b0 0x7fcac010e350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009990 tx=0x7fcab0009ca0 comp rx=0 tx=0).stop 2024-03-26T06:21:36.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.354+0000 7fcac766a700 1 -- 172.21.15.37:0/1553145082 shutdown_connections 2024-03-26T06:21:36.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.354+0000 7fcac766a700 1 --2- 172.21.15.37:0/1553145082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac010e890 0x7fcac0110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:36.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.354+0000 7fcac766a700 1 --2- 172.21.15.37:0/1553145082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcac0069a90 0x7fcac0069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:36.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.354+0000 7fcac766a700 1 --2- 172.21.15.37:0/1553145082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac006a3b0 0x7fcac010e350 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:36.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.355+0000 7fcac766a700 1 -- 172.21.15.37:0/1553145082 >> 172.21.15.37:0/1553145082 conn(0x7fcac0077c10 msgr2=0x7fcac0078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:36.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.355+0000 7fcac766a700 1 -- 172.21.15.37:0/1553145082 shutdown_connections 2024-03-26T06:21:36.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.355+0000 7fcac766a700 1 -- 172.21.15.37:0/1553145082 wait complete. 2024-03-26T06:21:36.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.355+0000 7fcac766a700 1 Processor -- start 2024-03-26T06:21:36.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.355+0000 7fcac766a700 1 -- start start 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac766a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcac0069a90 0x7fcac019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac766a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac006a3b0 0x7fcac019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac766a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac010e890 0x7fcac019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac766a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01142b0 con 0x7fcac006a3b0 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac766a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac0114130 con 0x7fcac0069a90 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac766a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac0114430 con 0x7fcac010e890 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac5406700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcac0069a90 0x7fcac019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac5c07700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac010e890 0x7fcac019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac5406700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcac0069a90 0x7fcac019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:53934/0 (socket says 172.21.15.37:53934) 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac5c07700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac010e890 0x7fcac019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:56192/0 (socket says 172.21.15.37:56192) 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac5406700 1 -- 172.21.15.37:0/1216784355 learned_addr learned my addr 172.21.15.37:0/1216784355 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac4c05700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac006a3b0 0x7fcac019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac5406700 1 -- 172.21.15.37:0/1216784355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac010e890 msgr2=0x7fcac019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:36.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac5406700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac010e890 0x7fcac019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:36.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac5406700 1 -- 172.21.15.37:0/1216784355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac006a3b0 msgr2=0x7fcac019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:36.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac5406700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcac006a3b0 0x7fcac019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:36.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac5406700 1 -- 172.21.15.37:0/1216784355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac019f700 con 0x7fcac0069a90 2024-03-26T06:21:36.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.356+0000 7fcac5c07700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac010e890 0x7fcac019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:21:36.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.357+0000 7fcac5406700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcac0069a90 0x7fcac019a740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00c9b0 tx=0x7fcabc00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:36.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.357+0000 7fcab67fc700 1 -- 172.21.15.37:0/1216784355 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc007b00 con 0x7fcac0069a90 2024-03-26T06:21:36.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.357+0000 7fcac766a700 1 -- 172.21.15.37:0/1216784355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac019f9f0 con 0x7fcac0069a90 2024-03-26T06:21:36.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.357+0000 7fcab67fc700 1 -- 172.21.15.37:0/1216784355 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcabc007ca0 con 0x7fcac0069a90 2024-03-26T06:21:36.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.357+0000 7fcab67fc700 1 -- 172.21.15.37:0/1216784355 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc018730 con 0x7fcac0069a90 2024-03-26T06:21:36.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.357+0000 7fcac766a700 1 -- 172.21.15.37:0/1216784355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac01a7270 con 0x7fcac0069a90 2024-03-26T06:21:36.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.358+0000 7fcac766a700 1 -- 172.21.15.37:0/1216784355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcac00624f0 con 0x7fcac0069a90 2024-03-26T06:21:36.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.359+0000 7fcab67fc700 1 -- 172.21.15.37:0/1216784355 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fcabc0188d0 con 0x7fcac0069a90 2024-03-26T06:21:36.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.360+0000 7fcab67fc700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcaac0a2920 0x7fcaac0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:36.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.360+0000 7fcac4c05700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcaac0a2920 0x7fcaac0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:36.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.360+0000 7fcab67fc700 1 -- 172.21.15.37:0/1216784355 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcabc0cef60 con 0x7fcac0069a90 2024-03-26T06:21:36.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.360+0000 7fcab67fc700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377] conn(0x7fcaac0ac100 0x7fcaac0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:36.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.360+0000 7fcab67fc700 1 -- 172.21.15.37:0/1216784355 --> [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcabc024e50 con 0x7fcaac0ac100 2024-03-26T06:21:36.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.360+0000 7fcab67fc700 1 -- 172.21.15.37:0/1216784355 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcabc0cf310 con 0x7fcac0069a90 2024-03-26T06:21:36.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.360+0000 7fcac4c05700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcaac0a2920 0x7fcaac0a4de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009750 tx=0x7fcab000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:36.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.360+0000 7fcac5c07700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377] conn(0x7fcaac0ac100 0x7fcaac0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:36.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.361+0000 7fcac5c07700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377] conn(0x7fcaac0ac100 0x7fcaac0ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:36.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.362+0000 7fcab67fc700 1 -- 172.21.15.37:0/1216784355 <== osd.2 v2:172.21.15.149:6800/611823377 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcabc024e50 con 0x7fcaac0ac100 2024-03-26T06:21:36.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.378+0000 7fcac766a700 1 -- 172.21.15.37:0/1216784355 --> [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcac0062920 con 0x7fcaac0ac100 2024-03-26T06:21:36.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.378+0000 7fcab67fc700 1 -- 172.21.15.37:0/1216784355 <== osd.2 v2:172.21.15.149:6800/611823377 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcac0062920 con 0x7fcaac0ac100 2024-03-26T06:21:36.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.379+0000 7fcac766a700 1 -- 172.21.15.37:0/1216784355 >> [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377] conn(0x7fcaac0ac100 msgr2=0x7fcaac0ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:36.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.379+0000 7fcac766a700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377] conn(0x7fcaac0ac100 0x7fcaac0ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:36.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.379+0000 7fcac766a700 1 -- 172.21.15.37:0/1216784355 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcaac0a2920 msgr2=0x7fcaac0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:36.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.379+0000 7fcac766a700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcaac0a2920 0x7fcaac0a4de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009750 tx=0x7fcab000b540 comp rx=0 tx=0).stop 2024-03-26T06:21:36.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.379+0000 7fcac766a700 1 -- 172.21.15.37:0/1216784355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcac0069a90 msgr2=0x7fcac019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.379+0000 7fcac766a700 1 --2- 172.21.15.37:0/1216784355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcac0069a90 0x7fcac019a740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00c9b0 tx=0x7fcabc00ccc0 comp rx=0 tx=0).stop 2024-03-26T06:21:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.379+0000 7fcac5c07700 1 -- 172.21.15.37:0/1216784355 reap_dead start 2024-03-26T06:21:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.379+0000 7fcac766a700 1 -- 172.21.15.37:0/1216784355 shutdown_connections 2024-03-26T06:21:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.379+0000 7fcac766a700 1 -- 172.21.15.37:0/1216784355 >> 172.21.15.37:0/1216784355 conn(0x7fcac0077c10 msgr2=0x7fcac010fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.379+0000 7fcac766a700 1 -- 172.21.15.37:0/1216784355 shutdown_connections 2024-03-26T06:21:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:36.379+0000 7fcac766a700 1 -- 172.21.15.37:0/1216784355 wait complete. 2024-03-26T06:21:36.839 INFO:teuthology.orchestra.run.smithi037.stdout:34359738443 2024-03-26T06:21:36.840 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-26T06:21:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:36 smithi037 ceph-mon[31974]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:21:37.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:36 smithi066 ceph-mon[36504]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:21:37.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:36 smithi149 ceph-mon[36681]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:21:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.610+0000 7f1a175e5700 1 -- 172.21.15.37:0/358187446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10101680 msgr2=0x7f1a1010db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.610+0000 7f1a175e5700 1 --2- 172.21.15.37:0/358187446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10101680 0x7f1a1010db60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800b0a0 tx=0x7f1a0800b3b0 comp rx=0 tx=0).stop 2024-03-26T06:21:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.610+0000 7f1a175e5700 1 -- 172.21.15.37:0/358187446 shutdown_connections 2024-03-26T06:21:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.610+0000 7f1a175e5700 1 --2- 172.21.15.37:0/358187446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10101680 0x7f1a1010db60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:37.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.610+0000 7f1a175e5700 1 --2- 172.21.15.37:0/358187446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a10106d20 0x7f1a10107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:37.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.610+0000 7f1a175e5700 1 --2- 172.21.15.37:0/358187446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a10100ce0 0x7f1a10101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:37.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.610+0000 7f1a175e5700 1 -- 172.21.15.37:0/358187446 >> 172.21.15.37:0/358187446 conn(0x7f1a10077c10 msgr2=0x7f1a10078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:37.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.610+0000 7f1a175e5700 1 -- 172.21.15.37:0/358187446 shutdown_connections 2024-03-26T06:21:37.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.610+0000 7f1a175e5700 1 -- 172.21.15.37:0/358187446 wait complete. 2024-03-26T06:21:37.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a175e5700 1 Processor -- start 2024-03-26T06:21:37.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a175e5700 1 -- start start 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a175e5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a10100ce0 0x7f1a10102990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a175e5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10101680 0x7f1a10102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a175e5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a10106d20 0x7f1a10103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a175e5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a10111020 con 0x7f1a10101680 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a175e5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a10110ea0 con 0x7f1a10106d20 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a175e5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a101111a0 con 0x7f1a10100ce0 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a14b80700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10101680 0x7f1a10102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a14b80700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10101680 0x7f1a10102ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46852/0 (socket says 172.21.15.37:46852) 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a14b80700 1 -- 172.21.15.37:0/2163405381 learned_addr learned my addr 172.21.15.37:0/2163405381 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a15b82700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a10106d20 0x7f1a10103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a14b80700 1 -- 172.21.15.37:0/2163405381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a10100ce0 msgr2=0x7f1a10102990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a14b80700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a10100ce0 0x7f1a10102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:37.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a14b80700 1 -- 172.21.15.37:0/2163405381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a10106d20 msgr2=0x7f1a10103410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:37.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a14b80700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a10106d20 0x7f1a10103410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:37.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.611+0000 7f1a14b80700 1 -- 172.21.15.37:0/2163405381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a101a6a20 con 0x7f1a10101680 2024-03-26T06:21:37.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.612+0000 7f1a14b80700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10101680 0x7f1a10102ed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00b530 tx=0x7f1a0c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:37.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.612+0000 7f1a067fc700 1 -- 172.21.15.37:0/2163405381 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c015020 con 0x7f1a10101680 2024-03-26T06:21:37.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.612+0000 7f1a067fc700 1 -- 172.21.15.37:0/2163405381 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a0c011d10 con 0x7f1a10101680 2024-03-26T06:21:37.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.612+0000 7f1a175e5700 1 -- 172.21.15.37:0/2163405381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a101a6d10 con 0x7f1a10101680 2024-03-26T06:21:37.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.612+0000 7f1a067fc700 1 -- 172.21.15.37:0/2163405381 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c010e50 con 0x7f1a10101680 2024-03-26T06:21:37.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.612+0000 7f1a175e5700 1 -- 172.21.15.37:0/2163405381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a101a71f0 con 0x7f1a10101680 2024-03-26T06:21:37.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.613+0000 7f1a175e5700 1 -- 172.21.15.37:0/2163405381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a10104da0 con 0x7f1a10101680 2024-03-26T06:21:37.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.614+0000 7f1a067fc700 1 -- 172.21.15.37:0/2163405381 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f1a0c018020 con 0x7f1a10101680 2024-03-26T06:21:37.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.614+0000 7f1a067fc700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f19fc0a2ac0 0x7f19fc0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:37.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.614+0000 7f1a15381700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f19fc0a2ac0 0x7f19fc0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:37.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.614+0000 7f1a067fc700 1 -- 172.21.15.37:0/2163405381 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1a0c0ce900 con 0x7f1a10101680 2024-03-26T06:21:37.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.614+0000 7f1a067fc700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155] conn(0x7f19fc0ac2a0 0x7f19fc0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:37.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.615+0000 7f1a067fc700 1 -- 172.21.15.37:0/2163405381 --> [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a0c010e50 con 0x7f19fc0ac2a0 2024-03-26T06:21:37.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.615+0000 7f1a067fc700 1 -- 172.21.15.37:0/2163405381 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a0c0cecb0 con 0x7f1a10101680 2024-03-26T06:21:37.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.615+0000 7f1a15b82700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155] conn(0x7f19fc0ac2a0 0x7f19fc0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:37.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.615+0000 7f1a15381700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f19fc0a2ac0 0x7f19fc0a4f80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1a00005fd0 tx=0x7f1a00005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:37.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.615+0000 7f1a15b82700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155] conn(0x7f19fc0ac2a0 0x7f19fc0ae6c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:37.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.616+0000 7f1a067fc700 1 -- 172.21.15.37:0/2163405381 <== osd.10 v2:172.21.15.66:6824/364342155 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1a0c010e50 con 0x7f19fc0ac2a0 2024-03-26T06:21:37.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.632+0000 7f1a175e5700 1 -- 172.21.15.37:0/2163405381 --> [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1a101023d0 con 0x7f19fc0ac2a0 2024-03-26T06:21:37.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.633+0000 7f1a067fc700 1 -- 172.21.15.37:0/2163405381 <== osd.10 v2:172.21.15.66:6824/364342155 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1a101023d0 con 0x7f19fc0ac2a0 2024-03-26T06:21:37.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.633+0000 7f1a175e5700 1 -- 172.21.15.37:0/2163405381 >> [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155] conn(0x7f19fc0ac2a0 msgr2=0x7f19fc0ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:37.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.633+0000 7f1a175e5700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155] conn(0x7f19fc0ac2a0 0x7f19fc0ae6c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:37.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.633+0000 7f1a175e5700 1 -- 172.21.15.37:0/2163405381 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f19fc0a2ac0 msgr2=0x7f19fc0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:37.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.633+0000 7f1a175e5700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f19fc0a2ac0 0x7f19fc0a4f80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1a00005fd0 tx=0x7f1a00005d00 comp rx=0 tx=0).stop 2024-03-26T06:21:37.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.633+0000 7f1a175e5700 1 -- 172.21.15.37:0/2163405381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10101680 msgr2=0x7f1a10102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:37.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.633+0000 7f1a175e5700 1 --2- 172.21.15.37:0/2163405381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10101680 0x7f1a10102ed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00b530 tx=0x7f1a0c00b8f0 comp rx=0 tx=0).stop 2024-03-26T06:21:37.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.633+0000 7f1a15b82700 1 -- 172.21.15.37:0/2163405381 reap_dead start 2024-03-26T06:21:37.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.634+0000 7f1a175e5700 1 -- 172.21.15.37:0/2163405381 shutdown_connections 2024-03-26T06:21:37.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.634+0000 7f1a175e5700 1 -- 172.21.15.37:0/2163405381 >> 172.21.15.37:0/2163405381 conn(0x7f1a10077c10 msgr2=0x7f1a100ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:37.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.634+0000 7f1a175e5700 1 -- 172.21.15.37:0/2163405381 shutdown_connections 2024-03-26T06:21:37.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:37.634+0000 7f1a175e5700 1 -- 172.21.15.37:0/2163405381 wait complete. 2024-03-26T06:21:38.098 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:38.099 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:38.101 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:38 smithi037 ceph-mon[31974]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:21:39.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:38 smithi066 ceph-mon[36504]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:21:39.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:38 smithi149 ceph-mon[36681]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:21:39.371 INFO:teuthology.orchestra.run.smithi037.stdout:309237645338 2024-03-26T06:21:39.371 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-26T06:21:40.050 INFO:teuthology.orchestra.run.smithi037.stdout:77309411395 2024-03-26T06:21:40.050 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-26T06:21:40.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:40 smithi037 ceph-mon[31974]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:21:41.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:40 smithi066 ceph-mon[36504]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:21:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.134+0000 7f2a77c4a700 1 -- 172.21.15.37:0/3168868493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a70106090 msgr2=0x7f2a7010dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.134+0000 7f2a77c4a700 1 --2- 172.21.15.37:0/3168868493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a70106090 0x7f2a7010dc60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800b0d0 tx=0x7f2a6800b3e0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.134+0000 7f2a77c4a700 1 -- 172.21.15.37:0/3168868493 shutdown_connections 2024-03-26T06:21:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.134+0000 7f2a77c4a700 1 --2- 172.21.15.37:0/3168868493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a70106090 0x7f2a7010dc60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.134+0000 7f2a77c4a700 1 --2- 172.21.15.37:0/3168868493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a70100fb0 0x7f2a70101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.134+0000 7f2a77c4a700 1 --2- 172.21.15.37:0/3168868493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a70101960 0x7f2a701059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.134+0000 7f2a77c4a700 1 -- 172.21.15.37:0/3168868493 >> 172.21.15.37:0/3168868493 conn(0x7f2a70077e10 msgr2=0x7f2a70078220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.134+0000 7f2a77c4a700 1 -- 172.21.15.37:0/3168868493 shutdown_connections 2024-03-26T06:21:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.134+0000 7f2a77c4a700 1 -- 172.21.15.37:0/3168868493 wait complete. 2024-03-26T06:21:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a77c4a700 1 Processor -- start 2024-03-26T06:21:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a77c4a700 1 -- start start 2024-03-26T06:21:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a77c4a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a70100fb0 0x7f2a70198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a77c4a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a70101960 0x7f2a70198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a77c4a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a70106090 0x7f2a7019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a77c4a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a701121f0 con 0x7f2a70106090 2024-03-26T06:21:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a77c4a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70112070 con 0x7f2a70100fb0 2024-03-26T06:21:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a77c4a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70112370 con 0x7f2a70101960 2024-03-26T06:21:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a761e7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a70106090 0x7f2a7019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:41.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a761e7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a70106090 0x7f2a7019ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46870/0 (socket says 172.21.15.37:46870) 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a761e7700 1 -- 172.21.15.37:0/2867526006 learned_addr learned my addr 172.21.15.37:0/2867526006 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a759e6700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a70100fb0 0x7f2a70198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.135+0000 7f2a751e5700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a70101960 0x7f2a70198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.136+0000 7f2a761e7700 1 -- 172.21.15.37:0/2867526006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a70101960 msgr2=0x7f2a70198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.136+0000 7f2a761e7700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a70101960 0x7f2a70198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.136+0000 7f2a761e7700 1 -- 172.21.15.37:0/2867526006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a70100fb0 msgr2=0x7f2a70198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.136+0000 7f2a761e7700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a70100fb0 0x7f2a70198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.136+0000 7f2a761e7700 1 -- 172.21.15.37:0/2867526006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7019d610 con 0x7f2a70106090 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.136+0000 7f2a761e7700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a70106090 0x7f2a7019ced0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a68006070 tx=0x7f2a68004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.136+0000 7f2a66ffd700 1 -- 172.21.15.37:0/2867526006 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68021070 con 0x7f2a70106090 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.136+0000 7f2a77c4a700 1 -- 172.21.15.37:0/2867526006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7019d8a0 con 0x7f2a70106090 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.136+0000 7f2a66ffd700 1 -- 172.21.15.37:0/2867526006 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a68007b80 con 0x7f2a70106090 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.136+0000 7f2a66ffd700 1 -- 172.21.15.37:0/2867526006 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a680052a0 con 0x7f2a70106090 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.136+0000 7f2a77c4a700 1 -- 172.21.15.37:0/2867526006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a701a5180 con 0x7f2a70106090 2024-03-26T06:21:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.137+0000 7f2a77c4a700 1 -- 172.21.15.37:0/2867526006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a700624f0 con 0x7f2a70106090 2024-03-26T06:21:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.138+0000 7f2a66ffd700 1 -- 172.21.15.37:0/2867526006 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2a68024070 con 0x7f2a70106090 2024-03-26T06:21:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.139+0000 7f2a66ffd700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2a5c0a29a0 0x7f2a5c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.139+0000 7f2a759e6700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2a5c0a29a0 0x7f2a5c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.139+0000 7f2a66ffd700 1 -- 172.21.15.37:0/2867526006 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2a6800f020 con 0x7f2a70106090 2024-03-26T06:21:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.139+0000 7f2a66ffd700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917] conn(0x7f2a5c0ac180 0x7f2a5c0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.139+0000 7f2a66ffd700 1 -- 172.21.15.37:0/2867526006 --> [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a5c0aec50 con 0x7f2a5c0ac180 2024-03-26T06:21:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.139+0000 7f2a66ffd700 1 -- 172.21.15.37:0/2867526006 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a680d0690 con 0x7f2a70106090 2024-03-26T06:21:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.139+0000 7f2a751e5700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917] conn(0x7f2a5c0ac180 0x7f2a5c0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.139+0000 7f2a759e6700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2a5c0a29a0 0x7f2a5c0a4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009a10 tx=0x7f2a6c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.140+0000 7f2a751e5700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917] conn(0x7f2a5c0ac180 0x7f2a5c0ae5a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.140+0000 7f2a66ffd700 1 -- 172.21.15.37:0/2867526006 <== osd.9 v2:172.21.15.37:6826/2324500917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2a5c0aec50 con 0x7f2a5c0ac180 2024-03-26T06:21:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.157+0000 7f2a77c4a700 1 -- 172.21.15.37:0/2867526006 --> [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2a70062920 con 0x7f2a5c0ac180 2024-03-26T06:21:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a66ffd700 1 -- 172.21.15.37:0/2867526006 <== osd.9 v2:172.21.15.37:6826/2324500917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2a70062920 con 0x7f2a5c0ac180 2024-03-26T06:21:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a77c4a700 1 -- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917] conn(0x7f2a5c0ac180 msgr2=0x7f2a5c0ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a77c4a700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917] conn(0x7f2a5c0ac180 0x7f2a5c0ae5a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a77c4a700 1 -- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2a5c0a29a0 msgr2=0x7f2a5c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a77c4a700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2a5c0a29a0 0x7f2a5c0a4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009a10 tx=0x7f2a6c009190 comp rx=0 tx=0).stop 2024-03-26T06:21:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a77c4a700 1 -- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a70106090 msgr2=0x7f2a7019ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a77c4a700 1 --2- 172.21.15.37:0/2867526006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a70106090 0x7f2a7019ced0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a68006070 tx=0x7f2a68004d00 comp rx=0 tx=0).stop 2024-03-26T06:21:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a761e7700 1 -- 172.21.15.37:0/2867526006 reap_dead start 2024-03-26T06:21:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a77c4a700 1 -- 172.21.15.37:0/2867526006 shutdown_connections 2024-03-26T06:21:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a77c4a700 1 -- 172.21.15.37:0/2867526006 >> 172.21.15.37:0/2867526006 conn(0x7f2a70077e10 msgr2=0x7f2a700fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a77c4a700 1 -- 172.21.15.37:0/2867526006 shutdown_connections 2024-03-26T06:21:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.158+0000 7f2a77c4a700 1 -- 172.21.15.37:0/2867526006 wait complete. 2024-03-26T06:21:41.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:40 smithi149 ceph-mon[36681]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:21:41.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.849+0000 7fdce3988700 1 -- 172.21.15.37:0/1211234805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcdc1030f0 msgr2=0x7fdcdc1034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.849+0000 7fdce3988700 1 --2- 172.21.15.37:0/1211234805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcdc1030f0 0x7fdcdc1034d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009990 tx=0x7fdcd8009ca0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.850+0000 7fdce3988700 1 -- 172.21.15.37:0/1211234805 shutdown_connections 2024-03-26T06:21:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.850+0000 7fdce3988700 1 --2- 172.21.15.37:0/1211234805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcdc1030f0 0x7fdcdc1034d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.850+0000 7fdce3988700 1 --2- 172.21.15.37:0/1211234805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcdc10c4d0 0x7fdcdc10e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.850+0000 7fdce3988700 1 --2- 172.21.15.37:0/1211234805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcdc103a10 0x7fdcdc10bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.850+0000 7fdce3988700 1 -- 172.21.15.37:0/1211234805 >> 172.21.15.37:0/1211234805 conn(0x7fdcdc077e10 msgr2=0x7fdcdc078220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.850+0000 7fdce3988700 1 -- 172.21.15.37:0/1211234805 shutdown_connections 2024-03-26T06:21:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.850+0000 7fdce3988700 1 -- 172.21.15.37:0/1211234805 wait complete. 2024-03-26T06:21:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.850+0000 7fdce3988700 1 Processor -- start 2024-03-26T06:21:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.850+0000 7fdce3988700 1 -- start start 2024-03-26T06:21:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce3988700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcdc1030f0 0x7fdcdc198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce3988700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcdc103a10 0x7fdcdc198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce3988700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcdc10c4d0 0x7fdcdc19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce3988700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc112220 con 0x7fdcdc1030f0 2024-03-26T06:21:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce3988700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc1120a0 con 0x7fdcdc10c4d0 2024-03-26T06:21:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce3988700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc1123a0 con 0x7fdcdc103a10 2024-03-26T06:21:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce1724700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcdc1030f0 0x7fdcdc198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce1724700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcdc1030f0 0x7fdcdc198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46898/0 (socket says 172.21.15.37:46898) 2024-03-26T06:21:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce1724700 1 -- 172.21.15.37:0/2856707138 learned_addr learned my addr 172.21.15.37:0/2856707138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce0f23700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcdc103a10 0x7fdcdc198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce1f25700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcdc10c4d0 0x7fdcdc19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce1724700 1 -- 172.21.15.37:0/2856707138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcdc103a10 msgr2=0x7fdcdc198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce1724700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcdc103a10 0x7fdcdc198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce1724700 1 -- 172.21.15.37:0/2856707138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcdc10c4d0 msgr2=0x7fdcdc19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce1724700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcdc10c4d0 0x7fdcdc19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce1724700 1 -- 172.21.15.37:0/2856707138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc19d640 con 0x7fdcdc1030f0 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce1f25700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcdc10c4d0 0x7fdcdc19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce0f23700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcdc103a10 0x7fdcdc198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce1724700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcdc1030f0 0x7fdcdc198630 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80038b0 tx=0x7fdcd8005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdcd27fc700 1 -- 172.21.15.37:0/2856707138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8027070 con 0x7fdcdc1030f0 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdcd27fc700 1 -- 172.21.15.37:0/2856707138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcd80040d0 con 0x7fdcdc1030f0 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdcd27fc700 1 -- 172.21.15.37:0/2856707138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8022e40 con 0x7fdcdc1030f0 2024-03-26T06:21:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.851+0000 7fdce3988700 1 -- 172.21.15.37:0/2856707138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc19d8d0 con 0x7fdcdc1030f0 2024-03-26T06:21:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.852+0000 7fdce3988700 1 -- 172.21.15.37:0/2856707138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc1a51b0 con 0x7fdcdc1030f0 2024-03-26T06:21:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.852+0000 7fdce3988700 1 -- 172.21.15.37:0/2856707138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc068aa0 con 0x7fdcdc1030f0 2024-03-26T06:21:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.854+0000 7fdcd27fc700 1 -- 172.21.15.37:0/2856707138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fdcd802a070 con 0x7fdcdc1030f0 2024-03-26T06:21:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.854+0000 7fdcd27fc700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdcc80a29a0 0x7fdcc80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.854+0000 7fdce0f23700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdcc80a29a0 0x7fdcc80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.854+0000 7fdcd27fc700 1 -- 172.21.15.37:0/2856707138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdcd80d0230 con 0x7fdcdc1030f0 2024-03-26T06:21:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.855+0000 7fdce0f23700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdcc80a29a0 0x7fdcc80a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdccc005950 tx=0x7fdccc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:41.858+0000 7fdcd27fc700 1 -- 172.21.15.37:0/2856707138 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fdcd8031070 con 0x7fdcdc1030f0 2024-03-26T06:21:42.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.067+0000 7fdce3988700 1 -- 172.21.15.37:0/2856707138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdcdc1a53e0 con 0x7fdcdc1030f0 2024-03-26T06:21:42.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.067+0000 7fdcd27fc700 1 -- 172.21.15.37:0/2856707138 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdcd8013070 con 0x7fdcdc1030f0 2024-03-26T06:21:42.068 INFO:teuthology.orchestra.run.smithi037.stdout:55834574920 2024-03-26T06:21:42.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.069+0000 7fdce3988700 1 -- 172.21.15.37:0/2856707138 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdcc80a29a0 msgr2=0x7fdcc80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:42.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.069+0000 7fdce3988700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdcc80a29a0 0x7fdcc80a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdccc005950 tx=0x7fdccc0058e0 comp rx=0 tx=0).stop 2024-03-26T06:21:42.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.069+0000 7fdce3988700 1 -- 172.21.15.37:0/2856707138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcdc1030f0 msgr2=0x7fdcdc198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:42.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.069+0000 7fdce3988700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcdc1030f0 0x7fdcdc198630 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80038b0 tx=0x7fdcd8005590 comp rx=0 tx=0).stop 2024-03-26T06:21:42.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.069+0000 7fdce3988700 1 -- 172.21.15.37:0/2856707138 shutdown_connections 2024-03-26T06:21:42.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.069+0000 7fdce3988700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcdc1030f0 0x7fdcdc198630 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:42.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.069+0000 7fdce3988700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fdcc80a29a0 0x7fdcc80a4e60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:42.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.069+0000 7fdce3988700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcdc10c4d0 0x7fdcdc19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:42.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.069+0000 7fdce3988700 1 --2- 172.21.15.37:0/2856707138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcdc103a10 0x7fdcdc198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:42.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.070+0000 7fdce3988700 1 -- 172.21.15.37:0/2856707138 >> 172.21.15.37:0/2856707138 conn(0x7fdcdc077e10 msgr2=0x7fdcdc0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:42.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.070+0000 7fdce3988700 1 -- 172.21.15.37:0/2856707138 shutdown_connections 2024-03-26T06:21:42.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:42.070+0000 7fdce3988700 1 -- 172.21.15.37:0/2856707138 wait complete. 2024-03-26T06:21:42.727 INFO:teuthology.orchestra.run.smithi037.stdout:343597383700 2024-03-26T06:21:42.728 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-26T06:21:42.945 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:43.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:42 smithi037 ceph-mon[31974]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:21:43.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:42 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2856707138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T06:21:43.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:42 smithi066 ceph-mon[36504]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:21:43.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:42 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2856707138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T06:21:43.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:42 smithi149 ceph-mon[36681]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:21:43.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:42 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2856707138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T06:21:43.793 INFO:teuthology.orchestra.run.smithi037.stdout:287762808865 2024-03-26T06:21:43.793 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-26T06:21:43.832 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574916 got 55834574920 for osd.1 2024-03-26T06:21:43.832 DEBUG:teuthology.parallel:result is None 2024-03-26T06:21:44.750 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:44 smithi037 ceph-mon[31974]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-03-26T06:21:45.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:44 smithi066 ceph-mon[36504]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-03-26T06:21:45.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:44 smithi149 ceph-mon[36681]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-03-26T06:21:45.370 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:45.371 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:45.372 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.759+0000 7f8bb66e5700 1 -- 172.21.15.37:0/867026842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bb0105870 msgr2=0x7f8bb010d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.759+0000 7f8bb66e5700 1 --2- 172.21.15.37:0/867026842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bb0105870 0x7f8bb010d3f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800b0a0 tx=0x7f8ba800b3b0 comp rx=0 tx=0).stop 2024-03-26T06:21:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.760+0000 7f8bb66e5700 1 -- 172.21.15.37:0/867026842 shutdown_connections 2024-03-26T06:21:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.760+0000 7f8bb66e5700 1 --2- 172.21.15.37:0/867026842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bb0105870 0x7f8bb010d3f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.760+0000 7f8bb66e5700 1 --2- 172.21.15.37:0/867026842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb01009e0 0x7f8bb0100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.760+0000 7f8bb66e5700 1 --2- 172.21.15.37:0/867026842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bb0101390 0x7f8bb0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.760+0000 7f8bb66e5700 1 -- 172.21.15.37:0/867026842 >> 172.21.15.37:0/867026842 conn(0x7f8bb0076060 msgr2=0x7f8bb0076470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.760+0000 7f8bb66e5700 1 -- 172.21.15.37:0/867026842 shutdown_connections 2024-03-26T06:21:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.760+0000 7f8bb66e5700 1 -- 172.21.15.37:0/867026842 wait complete. 2024-03-26T06:21:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.760+0000 7f8bb66e5700 1 Processor -- start 2024-03-26T06:21:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.760+0000 7f8bb66e5700 1 -- start start 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8bb66e5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bb01009e0 0x7f8bb0198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8bb66e5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bb0101390 0x7f8bb0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8bb66e5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb0105870 0x7f8bb019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8bb66e5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb0111100 con 0x7f8bb01009e0 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8bb66e5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb0110f80 con 0x7f8bb0101390 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8bb66e5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb0111280 con 0x7f8bb0105870 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8baffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bb01009e0 0x7f8bb0198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8baffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bb01009e0 0x7f8bb0198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46930/0 (socket says 172.21.15.37:46930) 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8baffff700 1 -- 172.21.15.37:0/3747571459 learned_addr learned my addr 172.21.15.37:0/3747571459 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8baf7fe700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bb0101390 0x7f8bb0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8bb4c82700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb0105870 0x7f8bb019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8baffff700 1 -- 172.21.15.37:0/3747571459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb0105870 msgr2=0x7f8bb019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:45.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8baffff700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb0105870 0x7f8bb019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8baffff700 1 -- 172.21.15.37:0/3747571459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bb0101390 msgr2=0x7f8bb0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8baffff700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bb0101390 0x7f8bb0198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8baffff700 1 -- 172.21.15.37:0/3747571459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb019d590 con 0x7f8bb01009e0 2024-03-26T06:21:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.761+0000 7f8baffff700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bb01009e0 0x7f8bb0198580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000b500 tx=0x7f8ba000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.762+0000 7f8bad7fa700 1 -- 172.21.15.37:0/3747571459 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0015020 con 0x7f8bb01009e0 2024-03-26T06:21:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.762+0000 7f8bb66e5700 1 -- 172.21.15.37:0/3747571459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb019d880 con 0x7f8bb01009e0 2024-03-26T06:21:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.762+0000 7f8bb66e5700 1 -- 172.21.15.37:0/3747571459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb01a5100 con 0x7f8bb01009e0 2024-03-26T06:21:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.762+0000 7f8bad7fa700 1 -- 172.21.15.37:0/3747571459 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ba0011d10 con 0x7f8bb01009e0 2024-03-26T06:21:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.762+0000 7f8bad7fa700 1 -- 172.21.15.37:0/3747571459 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0010e50 con 0x7f8bb01009e0 2024-03-26T06:21:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.763+0000 7f8bb66e5700 1 -- 172.21.15.37:0/3747571459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b90005330 con 0x7f8bb01009e0 2024-03-26T06:21:45.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.764+0000 7f8bad7fa700 1 -- 172.21.15.37:0/3747571459 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f8ba0018020 con 0x7f8bb01009e0 2024-03-26T06:21:45.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.764+0000 7f8bad7fa700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8b9c0a2b40 0x7f8b9c0a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:45.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.764+0000 7f8baf7fe700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8b9c0a2b40 0x7f8b9c0a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:45.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.764+0000 7f8bad7fa700 1 -- 172.21.15.37:0/3747571459 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8ba00cdef0 con 0x7f8bb01009e0 2024-03-26T06:21:45.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.765+0000 7f8baf7fe700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8b9c0a2b40 0x7f8b9c0a5000 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0106d10 tx=0x7f8b9800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:45.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.769+0000 7f8bad7fa700 1 -- 172.21.15.37:0/3747571459 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8ba001f020 con 0x7f8bb01009e0 2024-03-26T06:21:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.976+0000 7f8bb66e5700 1 -- 172.21.15.37:0/3747571459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8b900058f0 con 0x7f8bb01009e0 2024-03-26T06:21:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.976+0000 7f8bad7fa700 1 -- 172.21.15.37:0/3747571459 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ba009b0f0 con 0x7f8bb01009e0 2024-03-26T06:21:45.977 INFO:teuthology.orchestra.run.smithi037.stdout:219043332140 2024-03-26T06:21:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 -- 172.21.15.37:0/3747571459 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8b9c0a2b40 msgr2=0x7f8b9c0a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8b9c0a2b40 0x7f8b9c0a5000 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0106d10 tx=0x7f8b9800b410 comp rx=0 tx=0).stop 2024-03-26T06:21:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 -- 172.21.15.37:0/3747571459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bb01009e0 msgr2=0x7f8bb0198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bb01009e0 0x7f8bb0198580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000b500 tx=0x7f8ba000b810 comp rx=0 tx=0).stop 2024-03-26T06:21:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 -- 172.21.15.37:0/3747571459 shutdown_connections 2024-03-26T06:21:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bb01009e0 0x7f8bb0198580 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8b9c0a2b40 0x7f8b9c0a5000 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb0105870 0x7f8bb019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 --2- 172.21.15.37:0/3747571459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8bb0101390 0x7f8bb0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 -- 172.21.15.37:0/3747571459 >> 172.21.15.37:0/3747571459 conn(0x7f8bb0076060 msgr2=0x7f8bb010df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 -- 172.21.15.37:0/3747571459 shutdown_connections 2024-03-26T06:21:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:45.978+0000 7f8bb66e5700 1 -- 172.21.15.37:0/3747571459 wait complete. 2024-03-26T06:21:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:46 smithi037 ceph-mon[31974]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:21:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:46 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3747571459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T06:21:47.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:46 smithi066 ceph-mon[36504]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:21:47.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:46 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3747571459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T06:21:47.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:46 smithi149 ceph-mon[36681]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:21:47.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:46 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3747571459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T06:21:47.624 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332136 got 219043332140 for osd.7 2024-03-26T06:21:47.624 DEBUG:teuthology.parallel:result is None 2024-03-26T06:21:47.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-26T06:21:47.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-26T06:21:47.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-26T06:21:47.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-26T06:21:47.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-03-26T06:21:47.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-03-26T06:21:47.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-03-26T06:21:47.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:47 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-26T06:21:48.059 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:48.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-26T06:21:48.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-26T06:21:48.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-26T06:21:48.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-26T06:21:48.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-03-26T06:21:48.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-03-26T06:21:48.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-03-26T06:21:48.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:47 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-26T06:21:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-26T06:21:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-26T06:21:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-26T06:21:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-26T06:21:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-03-26T06:21:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-03-26T06:21:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-03-26T06:21:48.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:47 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-26T06:21:49.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:48 smithi037 ceph-mon[31974]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:21:49.016 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:48 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-26T06:21:49.016 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:48 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-26T06:21:49.016 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:48 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-26T06:21:49.016 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:48 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-26T06:21:49.016 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:48 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-03-26T06:21:49.016 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:48 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-03-26T06:21:49.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:48 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-03-26T06:21:49.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:48 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-26T06:21:49.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:48 smithi037 ceph-mon[31974]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T06:21:49.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:48 smithi066 ceph-mon[36504]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:21:49.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:48 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-26T06:21:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:48 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-26T06:21:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:48 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-26T06:21:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:48 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-26T06:21:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:48 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-03-26T06:21:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:48 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-03-26T06:21:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:48 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-03-26T06:21:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:48 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-26T06:21:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:48 smithi066 ceph-mon[36504]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T06:21:49.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:48 smithi149 ceph-mon[36681]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:21:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:48 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-26T06:21:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:48 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-26T06:21:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:48 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-26T06:21:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:48 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-26T06:21:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:48 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-03-26T06:21:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:48 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-03-26T06:21:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:48 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-03-26T06:21:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:48 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-26T06:21:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:48 smithi149 ceph-mon[36681]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T06:21:49.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:49 smithi037 ceph-mon[31974]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T06:21:50.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:49 smithi066 ceph-mon[36504]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T06:21:50.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:49 smithi149 ceph-mon[36681]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T06:21:50.423 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:50.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.423+0000 7f63f4545700 1 -- 172.21.15.37:0/125138191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63ec101680 msgr2=0x7f63ec10db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:50.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.423+0000 7f63f4545700 1 --2- 172.21.15.37:0/125138191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63ec101680 0x7f63ec10db60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f63e400b0a0 tx=0x7f63e400b3b0 comp rx=0 tx=0).stop 2024-03-26T06:21:50.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.423+0000 7f63f4545700 1 -- 172.21.15.37:0/125138191 shutdown_connections 2024-03-26T06:21:50.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.423+0000 7f63f4545700 1 --2- 172.21.15.37:0/125138191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63ec101680 0x7f63ec10db60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:50.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.423+0000 7f63f4545700 1 --2- 172.21.15.37:0/125138191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec106d20 0x7f63ec107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:50.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.423+0000 7f63f4545700 1 --2- 172.21.15.37:0/125138191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63ec100ce0 0x7f63ec101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:50.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.423+0000 7f63f4545700 1 -- 172.21.15.37:0/125138191 >> 172.21.15.37:0/125138191 conn(0x7f63ec077c10 msgr2=0x7f63ec078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:50.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.423+0000 7f63f4545700 1 -- 172.21.15.37:0/125138191 shutdown_connections 2024-03-26T06:21:50.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.423+0000 7f63f4545700 1 -- 172.21.15.37:0/125138191 wait complete. 2024-03-26T06:21:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.424+0000 7f63f4545700 1 Processor -- start 2024-03-26T06:21:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.424+0000 7f63f4545700 1 -- start start 2024-03-26T06:21:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.424+0000 7f63f4545700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63ec100ce0 0x7f63ec19b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.424+0000 7f63f4545700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec101680 0x7f63ec19b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.424+0000 7f63f4545700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63ec106d20 0x7f63ec1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.424+0000 7f63f4545700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec110fe0 con 0x7f63ec106d20 2024-03-26T06:21:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.424+0000 7f63f4545700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec110e60 con 0x7f63ec100ce0 2024-03-26T06:21:50.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.424+0000 7f63f4545700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec111160 con 0x7f63ec101680 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.424+0000 7f63f1ae0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec101680 0x7f63ec19b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.424+0000 7f63f22e1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63ec100ce0 0x7f63ec19b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.424+0000 7f63f22e1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63ec100ce0 0x7f63ec19b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:55878/0 (socket says 172.21.15.37:55878) 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.425+0000 7f63f22e1700 1 -- 172.21.15.37:0/3427011689 learned_addr learned my addr 172.21.15.37:0/3427011689 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.425+0000 7f63f2ae2700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63ec106d20 0x7f63ec1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.425+0000 7f63f22e1700 1 -- 172.21.15.37:0/3427011689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec101680 msgr2=0x7f63ec19b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.425+0000 7f63f22e1700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec101680 0x7f63ec19b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.425+0000 7f63f22e1700 1 -- 172.21.15.37:0/3427011689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63ec106d20 msgr2=0x7f63ec1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.425+0000 7f63f22e1700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63ec106d20 0x7f63ec1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.425+0000 7f63f22e1700 1 -- 172.21.15.37:0/3427011689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec196af0 con 0x7f63ec100ce0 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.425+0000 7f63f22e1700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63ec100ce0 0x7f63ec19b0e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b530 tx=0x7f63e800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.426+0000 7f63e37fe700 1 -- 172.21.15.37:0/3427011689 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8015020 con 0x7f63ec100ce0 2024-03-26T06:21:50.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.426+0000 7f63e37fe700 1 -- 172.21.15.37:0/3427011689 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63e8011d10 con 0x7f63ec100ce0 2024-03-26T06:21:50.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.426+0000 7f63f4545700 1 -- 172.21.15.37:0/3427011689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ec196d80 con 0x7f63ec100ce0 2024-03-26T06:21:50.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.426+0000 7f63e37fe700 1 -- 172.21.15.37:0/3427011689 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e800f2e0 con 0x7f63ec100ce0 2024-03-26T06:21:50.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.426+0000 7f63f4545700 1 -- 172.21.15.37:0/3427011689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ec1972b0 con 0x7f63ec100ce0 2024-03-26T06:21:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.429+0000 7f63f4545700 1 -- 172.21.15.37:0/3427011689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ec1023d0 con 0x7f63ec100ce0 2024-03-26T06:21:50.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.429+0000 7f63e37fe700 1 -- 172.21.15.37:0/3427011689 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f63e8018020 con 0x7f63ec100ce0 2024-03-26T06:21:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.431+0000 7f63e37fe700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63d80a29a0 0x7f63d80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.431+0000 7f63e37fe700 1 -- 172.21.15.37:0/3427011689 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f63e80ce300 con 0x7f63ec100ce0 2024-03-26T06:21:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.431+0000 7f63f1ae0700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63d80a29a0 0x7f63d80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:50.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.433+0000 7f63f1ae0700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63d80a29a0 0x7f63d80a4e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005fd0 tx=0x7f63dc00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:50.435 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:21:50.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.435+0000 7f63e37fe700 1 -- 172.21.15.37:0/3427011689 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f63e801f020 con 0x7f63ec100ce0 2024-03-26T06:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.649+0000 7f63f4545700 1 -- 172.21.15.37:0/3427011689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f63ec06b8c0 con 0x7f63ec100ce0 2024-03-26T06:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.650+0000 7f63e37fe700 1 -- 172.21.15.37:0/3427011689 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f63e809b360 con 0x7f63ec100ce0 2024-03-26T06:21:50.651 INFO:teuthology.orchestra.run.smithi037.stdout:197568495666 2024-03-26T06:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 -- 172.21.15.37:0/3427011689 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63d80a29a0 msgr2=0x7f63d80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63d80a29a0 0x7f63d80a4e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005fd0 tx=0x7f63dc00a560 comp rx=0 tx=0).stop 2024-03-26T06:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 -- 172.21.15.37:0/3427011689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63ec100ce0 msgr2=0x7f63ec19b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63ec100ce0 0x7f63ec19b0e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b530 tx=0x7f63e800b840 comp rx=0 tx=0).stop 2024-03-26T06:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 -- 172.21.15.37:0/3427011689 shutdown_connections 2024-03-26T06:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63ec106d20 0x7f63ec1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f63d80a29a0 0x7f63d80a4e60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f63ec100ce0 0x7f63ec19b0e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 --2- 172.21.15.37:0/3427011689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec101680 0x7f63ec19b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 -- 172.21.15.37:0/3427011689 >> 172.21.15.37:0/3427011689 conn(0x7f63ec077c10 msgr2=0x7f63ec0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 -- 172.21.15.37:0/3427011689 shutdown_connections 2024-03-26T06:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:50.652+0000 7f63f4545700 1 -- 172.21.15.37:0/3427011689 wait complete. 2024-03-26T06:21:50.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:50 smithi037 ceph-mon[31974]: pgmap v318: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-03-26T06:21:50.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:50 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3427011689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T06:21:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:50 smithi066 ceph-mon[36504]: pgmap v318: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-03-26T06:21:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:50 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3427011689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T06:21:51.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:50 smithi149 ceph-mon[36681]: pgmap v318: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-03-26T06:21:51.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:50 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3427011689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T06:21:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.916+0000 7faf400ef700 1 -- 172.21.15.37:0/737143572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf3806a6e0 msgr2=0x7faf3806aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.916+0000 7faf400ef700 1 --2- 172.21.15.37:0/737143572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf3806a6e0 0x7faf3806aac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faf340099c0 tx=0x7faf34009cd0 comp rx=0 tx=0).stop 2024-03-26T06:21:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.916+0000 7faf400ef700 1 -- 172.21.15.37:0/737143572 shutdown_connections 2024-03-26T06:21:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.916+0000 7faf400ef700 1 --2- 172.21.15.37:0/737143572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf3806a6e0 0x7faf3806aac0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.916+0000 7faf400ef700 1 --2- 172.21.15.37:0/737143572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf38107150 0x7faf3810ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.916+0000 7faf400ef700 1 --2- 172.21.15.37:0/737143572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf3806b090 0x7faf38106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.916+0000 7faf400ef700 1 -- 172.21.15.37:0/737143572 >> 172.21.15.37:0/737143572 conn(0x7faf38077c50 msgr2=0x7faf38078060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.916+0000 7faf400ef700 1 -- 172.21.15.37:0/737143572 shutdown_connections 2024-03-26T06:21:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.916+0000 7faf400ef700 1 -- 172.21.15.37:0/737143572 wait complete. 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf400ef700 1 Processor -- start 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf400ef700 1 -- start start 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf400ef700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf3806a6e0 0x7faf38198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf400ef700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf3806b090 0x7faf38198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf400ef700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf38107150 0x7faf3819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf400ef700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf381120e0 con 0x7faf3806b090 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf400ef700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38111f60 con 0x7faf3806a6e0 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf400ef700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38112260 con 0x7faf38107150 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf3d68a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf3806b090 0x7faf38198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf3d68a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf3806b090 0x7faf38198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50954/0 (socket says 172.21.15.37:50954) 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf3d68a700 1 -- 172.21.15.37:0/1928034911 learned_addr learned my addr 172.21.15.37:0/1928034911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf3de8b700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf3806a6e0 0x7faf38198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf3e68c700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf38107150 0x7faf3819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf3d68a700 1 -- 172.21.15.37:0/1928034911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf38107150 msgr2=0x7faf3819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf3d68a700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf38107150 0x7faf3819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf3d68a700 1 -- 172.21.15.37:0/1928034911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf3806a6e0 msgr2=0x7faf38198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf3d68a700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf3806a6e0 0x7faf38198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.917+0000 7faf3d68a700 1 -- 172.21.15.37:0/1928034911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3819d510 con 0x7faf3806b090 2024-03-26T06:21:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.918+0000 7faf3d68a700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf3806b090 0x7faf38198a90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf2800d950 tx=0x7faf2800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.918+0000 7faf2effd700 1 -- 172.21.15.37:0/1928034911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28017070 con 0x7faf3806b090 2024-03-26T06:21:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.918+0000 7faf400ef700 1 -- 172.21.15.37:0/1928034911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3819d800 con 0x7faf3806b090 2024-03-26T06:21:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.918+0000 7faf2effd700 1 -- 172.21.15.37:0/1928034911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf28009e10 con 0x7faf3806b090 2024-03-26T06:21:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.918+0000 7faf2effd700 1 -- 172.21.15.37:0/1928034911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800f3c0 con 0x7faf3806b090 2024-03-26T06:21:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.918+0000 7faf400ef700 1 -- 172.21.15.37:0/1928034911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf381a5080 con 0x7faf3806b090 2024-03-26T06:21:51.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.919+0000 7faf400ef700 1 -- 172.21.15.37:0/1928034911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3810a320 con 0x7faf3806b090 2024-03-26T06:21:51.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.920+0000 7faf2effd700 1 -- 172.21.15.37:0/1928034911 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7faf2801a070 con 0x7faf3806b090 2024-03-26T06:21:51.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.921+0000 7faf2effd700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf240a2920 0x7faf240a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:51.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.921+0000 7faf2effd700 1 -- 172.21.15.37:0/1928034911 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faf280ce470 con 0x7faf3806b090 2024-03-26T06:21:51.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.921+0000 7faf3de8b700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf240a2920 0x7faf240a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.921+0000 7faf3de8b700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf240a2920 0x7faf240a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faf34000c00 tx=0x7faf34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:51.927+0000 7faf2effd700 1 -- 172.21.15.37:0/1928034911 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7faf2809b3c0 con 0x7faf3806b090 2024-03-26T06:21:52.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.138+0000 7faf400ef700 1 -- 172.21.15.37:0/1928034911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7faf38199860 con 0x7faf3806b090 2024-03-26T06:21:52.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.139+0000 7faf2effd700 1 -- 172.21.15.37:0/1928034911 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf2809acc0 con 0x7faf3806b090 2024-03-26T06:21:52.140 INFO:teuthology.orchestra.run.smithi037.stdout:253403070504 2024-03-26T06:21:52.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.140+0000 7faf400ef700 1 -- 172.21.15.37:0/1928034911 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf240a2920 msgr2=0x7faf240a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:52.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.140+0000 7faf400ef700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf240a2920 0x7faf240a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faf34000c00 tx=0x7faf34005fb0 comp rx=0 tx=0).stop 2024-03-26T06:21:52.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.141+0000 7faf400ef700 1 -- 172.21.15.37:0/1928034911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf3806b090 msgr2=0x7faf38198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:52.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.141+0000 7faf400ef700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf3806b090 0x7faf38198a90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf2800d950 tx=0x7faf2800dc60 comp rx=0 tx=0).stop 2024-03-26T06:21:52.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.141+0000 7faf400ef700 1 -- 172.21.15.37:0/1928034911 shutdown_connections 2024-03-26T06:21:52.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.141+0000 7faf400ef700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7faf240a2920 0x7faf240a4de0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:52.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.141+0000 7faf400ef700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf38107150 0x7faf3819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:52.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.141+0000 7faf400ef700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf3806a6e0 0x7faf38198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:52.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.141+0000 7faf400ef700 1 --2- 172.21.15.37:0/1928034911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf3806b090 0x7faf38198a90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:52.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.141+0000 7faf400ef700 1 -- 172.21.15.37:0/1928034911 >> 172.21.15.37:0/1928034911 conn(0x7faf38077c50 msgr2=0x7faf380fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:52.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.141+0000 7faf400ef700 1 -- 172.21.15.37:0/1928034911 shutdown_connections 2024-03-26T06:21:52.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:52.141+0000 7faf400ef700 1 -- 172.21.15.37:0/1928034911 wait complete. 2024-03-26T06:21:52.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:52 smithi037 ceph-mon[31974]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-03-26T06:21:52.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:52 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1928034911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T06:21:53.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:52 smithi066 ceph-mon[36504]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-03-26T06:21:53.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:52 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1928034911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T06:21:53.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:52 smithi149 ceph-mon[36681]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-03-26T06:21:53.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:52 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1928034911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T06:21:53.932 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:53 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:21:54.086 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:53 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:21:54.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:53 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:21:54.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:54 smithi037 ceph-mon[31974]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-26T06:21:55.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:54 smithi066 ceph-mon[36504]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-26T06:21:55.159 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:54 smithi149 ceph-mon[36681]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-26T06:21:56.161 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070498 got 253403070504 for osd.8 2024-03-26T06:21:56.161 DEBUG:teuthology.parallel:result is None 2024-03-26T06:21:56.260 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:55 smithi037 ceph-mon[31974]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 72 B/s, 0 objects/s recovering 2024-03-26T06:21:56.260 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:55 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:55 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:55 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:55 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.327 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:55 smithi149 ceph-mon[36681]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 72 B/s, 0 objects/s recovering 2024-03-26T06:21:56.327 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:55 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.327 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:55 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.327 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:55 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.327 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:55 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:55 smithi066 ceph-mon[36504]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 72 B/s, 0 objects/s recovering 2024-03-26T06:21:56.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:55 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:55 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:55 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:55 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:21:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.365+0000 7f696e3ac700 1 -- 172.21.15.37:0/1080450088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968100720 msgr2=0x7f6968100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.365+0000 7f696e3ac700 1 --2- 172.21.15.37:0/1080450088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968100720 0x7f6968100ba0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f695c009990 tx=0x7f695c009ca0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.365+0000 7f696e3ac700 1 -- 172.21.15.37:0/1080450088 shutdown_connections 2024-03-26T06:21:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.365+0000 7f696e3ac700 1 --2- 172.21.15.37:0/1080450088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69681010e0 0x7f696810d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.365+0000 7f696e3ac700 1 --2- 172.21.15.37:0/1080450088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6968106590 0x7f6968106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.365+0000 7f696e3ac700 1 --2- 172.21.15.37:0/1080450088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968100720 0x7f6968100ba0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.365+0000 7f696e3ac700 1 -- 172.21.15.37:0/1080450088 >> 172.21.15.37:0/1080450088 conn(0x7f6968076020 msgr2=0x7f6968076430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.365+0000 7f696e3ac700 1 -- 172.21.15.37:0/1080450088 shutdown_connections 2024-03-26T06:21:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.365+0000 7f696e3ac700 1 -- 172.21.15.37:0/1080450088 wait complete. 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.366+0000 7f696e3ac700 1 Processor -- start 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.366+0000 7f696e3ac700 1 -- start start 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.366+0000 7f696e3ac700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6968100720 0x7f69681020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.366+0000 7f696e3ac700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69681010e0 0x7f69681025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.366+0000 7f696e3ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968106590 0x7f6968102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.366+0000 7f696e3ac700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6968111030 con 0x7f6968106590 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.366+0000 7f696e3ac700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6968110eb0 con 0x7f69681010e0 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.366+0000 7f696e3ac700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69681111b0 con 0x7f6968100720 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.366+0000 7f696c949700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968106590 0x7f6968102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f6967fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6968100720 0x7f69681020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f696c949700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968106590 0x7f6968102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50972/0 (socket says 172.21.15.37:50972) 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f696c949700 1 -- 172.21.15.37:0/2845974331 learned_addr learned my addr 172.21.15.37:0/2845974331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f69677fe700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69681010e0 0x7f69681025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f696c949700 1 -- 172.21.15.37:0/2845974331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6968100720 msgr2=0x7f69681020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f696c949700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6968100720 0x7f69681020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f696c949700 1 -- 172.21.15.37:0/2845974331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69681010e0 msgr2=0x7f69681025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f696c949700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69681010e0 0x7f69681025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f696c949700 1 -- 172.21.15.37:0/2845974331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69681a6aa0 con 0x7f6968106590 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f69677fe700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69681010e0 0x7f69681025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f696c949700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968106590 0x7f6968102b30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6960007ee0 tx=0x7f696000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f69657fa700 1 -- 172.21.15.37:0/2845974331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696000dbd0 con 0x7f6968106590 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f696e3ac700 1 -- 172.21.15.37:0/2845974331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69681a6d90 con 0x7f6968106590 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f69657fa700 1 -- 172.21.15.37:0/2845974331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f696000dd70 con 0x7f6968106590 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f69657fa700 1 -- 172.21.15.37:0/2845974331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960005a60 con 0x7f6968106590 2024-03-26T06:21:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.367+0000 7f696e3ac700 1 -- 172.21.15.37:0/2845974331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69681a72a0 con 0x7f6968106590 2024-03-26T06:21:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.368+0000 7f696e3ac700 1 -- 172.21.15.37:0/2845974331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6968068aa0 con 0x7f6968106590 2024-03-26T06:21:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.369+0000 7f69657fa700 1 -- 172.21.15.37:0/2845974331 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f696001a020 con 0x7f6968106590 2024-03-26T06:21:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.370+0000 7f69657fa700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69500a2920 0x7f69500a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:56.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.370+0000 7f6967fff700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69500a2920 0x7f69500a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:56.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.370+0000 7f69657fa700 1 -- 172.21.15.37:0/2845974331 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f69600ce330 con 0x7f6968106590 2024-03-26T06:21:56.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.370+0000 7f6967fff700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69500a2920 0x7f69500a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6958007b60 tx=0x7f69580073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:56.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.374+0000 7f69657fa700 1 -- 172.21.15.37:0/2845974331 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6960010070 con 0x7f6968106590 2024-03-26T06:21:56.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.582+0000 7f696e3ac700 1 -- 172.21.15.37:0/2845974331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f696802da90 con 0x7f6968106590 2024-03-26T06:21:56.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.582+0000 7f69657fa700 1 -- 172.21.15.37:0/2845974331 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f696009b220 con 0x7f6968106590 2024-03-26T06:21:56.584 INFO:teuthology.orchestra.run.smithi037.stdout:107374182465 2024-03-26T06:21:56.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.584+0000 7f696e3ac700 1 -- 172.21.15.37:0/2845974331 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69500a2920 msgr2=0x7f69500a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.584+0000 7f696e3ac700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69500a2920 0x7f69500a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6958007b60 tx=0x7f69580073d0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.584+0000 7f696e3ac700 1 -- 172.21.15.37:0/2845974331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968106590 msgr2=0x7f6968102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.584+0000 7f696e3ac700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968106590 0x7f6968102b30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6960007ee0 tx=0x7f696000d3b0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.584+0000 7f696e3ac700 1 -- 172.21.15.37:0/2845974331 shutdown_connections 2024-03-26T06:21:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.585+0000 7f696e3ac700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968106590 0x7f6968102b30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.585+0000 7f696e3ac700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f69500a2920 0x7f69500a4de0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.585+0000 7f696e3ac700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6968100720 0x7f69681020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.585+0000 7f696e3ac700 1 --2- 172.21.15.37:0/2845974331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69681010e0 0x7f69681025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.585+0000 7f696e3ac700 1 -- 172.21.15.37:0/2845974331 >> 172.21.15.37:0/2845974331 conn(0x7f6968076020 msgr2=0x7f696810df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.585+0000 7f696e3ac700 1 -- 172.21.15.37:0/2845974331 shutdown_connections 2024-03-26T06:21:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.585+0000 7f696e3ac700 1 -- 172.21.15.37:0/2845974331 wait complete. 2024-03-26T06:21:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.597+0000 7fd0e3644700 1 -- 172.21.15.37:0/3361655072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0dc1020f0 msgr2=0x7fd0dc105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.597+0000 7fd0e3644700 1 --2- 172.21.15.37:0/3361655072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0dc1020f0 0x7fd0dc105980 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0099c0 tx=0x7fd0cc009cd0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.597+0000 7fd0e3644700 1 -- 172.21.15.37:0/3361655072 shutdown_connections 2024-03-26T06:21:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.597+0000 7fd0e3644700 1 --2- 172.21.15.37:0/3361655072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0dc106060 0x7fd0dc10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.597+0000 7fd0e3644700 1 --2- 172.21.15.37:0/3361655072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0dc101740 0x7fd0dc101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.597+0000 7fd0e3644700 1 --2- 172.21.15.37:0/3361655072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0dc1020f0 0x7fd0dc105980 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.597+0000 7fd0e3644700 1 -- 172.21.15.37:0/3361655072 >> 172.21.15.37:0/3361655072 conn(0x7fd0dc0772e0 msgr2=0x7fd0dc0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.597+0000 7fd0e3644700 1 -- 172.21.15.37:0/3361655072 shutdown_connections 2024-03-26T06:21:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.597+0000 7fd0e3644700 1 -- 172.21.15.37:0/3361655072 wait complete. 2024-03-26T06:21:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e3644700 1 Processor -- start 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e3644700 1 -- start start 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e3644700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0dc101740 0x7fd0dc10fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e3644700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0dc1020f0 0x7fd0dc1101e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e3644700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0dc106060 0x7fd0dc110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e3644700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc112170 con 0x7fd0dc1020f0 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e3644700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc111ff0 con 0x7fd0dc106060 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e3644700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc1122f0 con 0x7fd0dc101740 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e0bdf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0dc1020f0 0x7fd0dc1101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e13e0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0dc101740 0x7fd0dc10fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e0bdf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0dc1020f0 0x7fd0dc1101e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50990/0 (socket says 172.21.15.37:50990) 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.598+0000 7fd0e0bdf700 1 -- 172.21.15.37:0/1021062074 learned_addr learned my addr 172.21.15.37:0/1021062074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0e1be1700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0dc106060 0x7fd0dc110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0e0bdf700 1 -- 172.21.15.37:0/1021062074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0dc101740 msgr2=0x7fd0dc10fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0e0bdf700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0dc101740 0x7fd0dc10fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0e0bdf700 1 -- 172.21.15.37:0/1021062074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0dc106060 msgr2=0x7fd0dc110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0e0bdf700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0dc106060 0x7fd0dc110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0e0bdf700 1 -- 172.21.15.37:0/1021062074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc1a6bb0 con 0x7fd0dc1020f0 2024-03-26T06:21:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0e13e0700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0dc101740 0x7fd0dc10fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:21:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0e0bdf700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0dc1020f0 0x7fd0dc1101e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c010 tx=0x7fd0cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0d27fc700 1 -- 172.21.15.37:0/1021062074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc027070 con 0x7fd0dc1020f0 2024-03-26T06:21:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0d27fc700 1 -- 172.21.15.37:0/1021062074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0cc003730 con 0x7fd0dc1020f0 2024-03-26T06:21:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0e3644700 1 -- 172.21.15.37:0/1021062074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc1a6e40 con 0x7fd0dc1020f0 2024-03-26T06:21:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0d27fc700 1 -- 172.21.15.37:0/1021062074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc022d40 con 0x7fd0dc1020f0 2024-03-26T06:21:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.599+0000 7fd0e3644700 1 -- 172.21.15.37:0/1021062074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc1a71e0 con 0x7fd0dc1020f0 2024-03-26T06:21:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.600+0000 7fd0e3644700 1 -- 172.21.15.37:0/1021062074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0dc068aa0 con 0x7fd0dc1020f0 2024-03-26T06:21:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.602+0000 7fd0d27fc700 1 -- 172.21.15.37:0/1021062074 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fd0cc02a070 con 0x7fd0dc1020f0 2024-03-26T06:21:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.602+0000 7fd0d27fc700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd0c80a2920 0x7fd0c80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.603+0000 7fd0e13e0700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd0c80a2920 0x7fd0c80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.603+0000 7fd0d27fc700 1 -- 172.21.15.37:0/1021062074 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd0cc0cffd0 con 0x7fd0dc1020f0 2024-03-26T06:21:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.603+0000 7fd0e13e0700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd0c80a2920 0x7fd0c80a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8005950 tx=0x7fd0d800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.606+0000 7fd0d27fc700 1 -- 172.21.15.37:0/1021062074 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd0cc031070 con 0x7fd0dc1020f0 2024-03-26T06:21:56.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.815+0000 7fd0e3644700 1 -- 172.21.15.37:0/1021062074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd0dc02d950 con 0x7fd0dc1020f0 2024-03-26T06:21:56.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.815+0000 7fd0d27fc700 1 -- 172.21.15.37:0/1021062074 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd0cc013070 con 0x7fd0dc1020f0 2024-03-26T06:21:56.817 INFO:teuthology.orchestra.run.smithi037.stdout:163208757304 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 -- 172.21.15.37:0/1021062074 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd0c80a2920 msgr2=0x7fd0c80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd0c80a2920 0x7fd0c80a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8005950 tx=0x7fd0d800a300 comp rx=0 tx=0).stop 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 -- 172.21.15.37:0/1021062074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0dc1020f0 msgr2=0x7fd0dc1101e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0dc1020f0 0x7fd0dc1101e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c010 tx=0x7fd0cc0047b0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 -- 172.21.15.37:0/1021062074 shutdown_connections 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd0c80a2920 0x7fd0c80a4de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0dc106060 0x7fd0dc110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0dc101740 0x7fd0dc10fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 --2- 172.21.15.37:0/1021062074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0dc1020f0 0x7fd0dc1101e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 -- 172.21.15.37:0/1021062074 >> 172.21.15.37:0/1021062074 conn(0x7fd0dc0772e0 msgr2=0x7fd0dc0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 -- 172.21.15.37:0/1021062074 shutdown_connections 2024-03-26T06:21:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:56.818+0000 7fd0e3644700 1 -- 172.21.15.37:0/1021062074 wait complete. 2024-03-26T06:21:57.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:56 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2845974331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T06:21:57.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:56 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1021062074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T06:21:57.258 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:56 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2845974331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T06:21:57.258 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:56 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1021062074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T06:21:57.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:56 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2845974331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T06:21:57.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:56 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1021062074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T06:21:57.645 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182465 for osd.3 2024-03-26T06:21:57.645 DEBUG:teuthology.parallel:result is None 2024-03-26T06:21:58.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:57 smithi037 ceph-mon[31974]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 64 B/s, 0 objects/s recovering 2024-03-26T06:21:58.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:58 smithi066 ceph-mon[36504]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 64 B/s, 0 objects/s recovering 2024-03-26T06:21:58.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:58 smithi149 ceph-mon[36681]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 64 B/s, 0 objects/s recovering 2024-03-26T06:21:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.510+0000 7f0706fa6700 1 -- 172.21.15.37:0/849581529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0700101680 msgr2=0x7f070010db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.510+0000 7f0706fa6700 1 --2- 172.21.15.37:0/849581529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0700101680 0x7f070010db60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f06f800b0a0 tx=0x7f06f800b3b0 comp rx=0 tx=0).stop 2024-03-26T06:21:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.510+0000 7f0706fa6700 1 -- 172.21.15.37:0/849581529 shutdown_connections 2024-03-26T06:21:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.510+0000 7f0706fa6700 1 --2- 172.21.15.37:0/849581529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0700101680 0x7f070010db60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.510+0000 7f0706fa6700 1 --2- 172.21.15.37:0/849581529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0700106d20 0x7f0700107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.510+0000 7f0706fa6700 1 --2- 172.21.15.37:0/849581529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0700100ce0 0x7f0700101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.510+0000 7f0706fa6700 1 -- 172.21.15.37:0/849581529 >> 172.21.15.37:0/849581529 conn(0x7f0700077c10 msgr2=0x7f0700078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.510+0000 7f0706fa6700 1 -- 172.21.15.37:0/849581529 shutdown_connections 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.510+0000 7f0706fa6700 1 -- 172.21.15.37:0/849581529 wait complete. 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.510+0000 7f0706fa6700 1 Processor -- start 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.510+0000 7f0706fa6700 1 -- start start 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0706fa6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0700100ce0 0x7f07001973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0706fa6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0700101680 0x7f07001978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0706fa6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0700106d20 0x7f070019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0706fa6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0700110fa0 con 0x7f0700100ce0 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0706fa6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0700110e20 con 0x7f0700106d20 2024-03-26T06:21:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0706fa6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0700111120 con 0x7f0700101680 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0704d42700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0700100ce0 0x7f07001973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0704d42700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0700100ce0 0x7f07001973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47804/0 (socket says 172.21.15.37:47804) 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0704d42700 1 -- 172.21.15.37:0/1350701701 learned_addr learned my addr 172.21.15.37:0/1350701701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0705543700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0700106d20 0x7f070019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f06fffff700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0700101680 0x7f07001978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0704d42700 1 -- 172.21.15.37:0/1350701701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0700101680 msgr2=0x7f07001978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0704d42700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0700101680 0x7f07001978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0704d42700 1 -- 172.21.15.37:0/1350701701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0700106d20 msgr2=0x7f070019bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0704d42700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0700106d20 0x7f070019bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0704d42700 1 -- 172.21.15.37:0/1350701701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070019c3c0 con 0x7f0700100ce0 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f06fffff700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0700101680 0x7f07001978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.511+0000 7f0704d42700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0700100ce0 0x7f07001973b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f06f4007910 tx=0x7f06f4007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.512+0000 7f06fdffb700 1 -- 172.21.15.37:0/1350701701 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4007e60 con 0x7f0700100ce0 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.512+0000 7f0706fa6700 1 -- 172.21.15.37:0/1350701701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070019c6b0 con 0x7f0700100ce0 2024-03-26T06:21:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.512+0000 7f06fdffb700 1 -- 172.21.15.37:0/1350701701 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06f4004500 con 0x7f0700100ce0 2024-03-26T06:21:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.512+0000 7f06fdffb700 1 -- 172.21.15.37:0/1350701701 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f40053c0 con 0x7f0700100ce0 2024-03-26T06:21:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.512+0000 7f0706fa6700 1 -- 172.21.15.37:0/1350701701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0700069a90 con 0x7f0700100ce0 2024-03-26T06:21:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.513+0000 7f0706fa6700 1 -- 172.21.15.37:0/1350701701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0700068aa0 con 0x7f0700100ce0 2024-03-26T06:21:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.514+0000 7f06fdffb700 1 -- 172.21.15.37:0/1350701701 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f06f401f020 con 0x7f0700100ce0 2024-03-26T06:21:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.515+0000 7f06fdffb700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f06f00a2920 0x7f06f00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.515+0000 7f06fffff700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f06f00a2920 0x7f06f00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.515+0000 7f06fdffb700 1 -- 172.21.15.37:0/1350701701 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f06f40d0640 con 0x7f0700100ce0 2024-03-26T06:21:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.516+0000 7f06fffff700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f06f00a2920 0x7f06f00a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f06ec005fd0 tx=0x7f06ec005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:58.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.522+0000 7f06fdffb700 1 -- 172.21.15.37:0/1350701701 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f06f4026030 con 0x7f0700100ce0 2024-03-26T06:21:58.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.737+0000 7f0706fa6700 1 -- 172.21.15.37:0/1350701701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f070006a4c0 con 0x7f0700100ce0 2024-03-26T06:21:58.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.737+0000 7f06fdffb700 1 -- 172.21.15.37:0/1350701701 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f06f409d530 con 0x7f0700100ce0 2024-03-26T06:21:58.739 INFO:teuthology.orchestra.run.smithi037.stdout:309237645342 2024-03-26T06:21:58.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.739+0000 7f0706fa6700 1 -- 172.21.15.37:0/1350701701 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f06f00a2920 msgr2=0x7f06f00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:58.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.739+0000 7f0706fa6700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f06f00a2920 0x7f06f00a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f06ec005fd0 tx=0x7f06ec005dc0 comp rx=0 tx=0).stop 2024-03-26T06:21:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.739+0000 7f0706fa6700 1 -- 172.21.15.37:0/1350701701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0700100ce0 msgr2=0x7f07001973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.739+0000 7f0706fa6700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0700100ce0 0x7f07001973b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f06f4007910 tx=0x7f06f4007c20 comp rx=0 tx=0).stop 2024-03-26T06:21:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.739+0000 7f0706fa6700 1 -- 172.21.15.37:0/1350701701 shutdown_connections 2024-03-26T06:21:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.739+0000 7f0706fa6700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0700100ce0 0x7f07001973b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.739+0000 7f0706fa6700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f06f00a2920 0x7f06f00a4de0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.739+0000 7f0706fa6700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0700106d20 0x7f070019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.739+0000 7f0706fa6700 1 --2- 172.21.15.37:0/1350701701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0700101680 0x7f07001978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.739+0000 7f0706fa6700 1 -- 172.21.15.37:0/1350701701 >> 172.21.15.37:0/1350701701 conn(0x7f0700077c10 msgr2=0x7f07000ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.739+0000 7f0706fa6700 1 -- 172.21.15.37:0/1350701701 shutdown_connections 2024-03-26T06:21:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.740+0000 7f0706fa6700 1 -- 172.21.15.37:0/1350701701 wait complete. 2024-03-26T06:21:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 -- 172.21.15.37:0/2722798506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf4101e30 msgr2=0x7efdf41022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 --2- 172.21.15.37:0/2722798506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf4101e30 0x7efdf41022b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efdf00099c0 tx=0x7efdf0009cd0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 -- 172.21.15.37:0/2722798506 shutdown_connections 2024-03-26T06:21:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 --2- 172.21.15.37:0/2722798506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdf41027f0 0x7efdf410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 --2- 172.21.15.37:0/2722798506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdf4107e10 0x7efdf41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 --2- 172.21.15.37:0/2722798506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf4101e30 0x7efdf41022b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 -- 172.21.15.37:0/2722798506 >> 172.21.15.37:0/2722798506 conn(0x7efdf4078f10 msgr2=0x7efdf4079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 -- 172.21.15.37:0/2722798506 shutdown_connections 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 -- 172.21.15.37:0/2722798506 wait complete. 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 Processor -- start 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 -- start start 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdf4101e30 0x7efdf4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdf41027f0 0x7efdf4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf4107e10 0x7efdf419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4112100 con 0x7efdf4107e10 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4111f80 con 0x7efdf41027f0 2024-03-26T06:21:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.395+0000 7efdfc13f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4112280 con 0x7efdf4101e30 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdfa6dc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf4107e10 0x7efdf419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdfa6dc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf4107e10 0x7efdf419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47784/0 (socket says 172.21.15.37:47784) 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdfa6dc700 1 -- 172.21.15.37:0/2285363435 learned_addr learned my addr 172.21.15.37:0/2285363435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdf9edb700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdf4101e30 0x7efdf4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdfa6dc700 1 -- 172.21.15.37:0/2285363435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdf4101e30 msgr2=0x7efdf4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdfa6dc700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdf4101e30 0x7efdf4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdfa6dc700 1 -- 172.21.15.37:0/2285363435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdf41027f0 msgr2=0x7efdf4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdf96da700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdf41027f0 0x7efdf4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdfa6dc700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdf41027f0 0x7efdf4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdfa6dc700 1 -- 172.21.15.37:0/2285363435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf419d550 con 0x7efdf4107e10 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdf96da700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdf41027f0 0x7efdf4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdfa6dc700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf4107e10 0x7efdf419ce10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efdec007ee0 tx=0x7efdec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdeaffd700 1 -- 172.21.15.37:0/2285363435 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec00dbd0 con 0x7efdf4107e10 2024-03-26T06:21:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdeaffd700 1 -- 172.21.15.37:0/2285363435 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdec00dd70 con 0x7efdf4107e10 2024-03-26T06:21:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdfc13f700 1 -- 172.21.15.37:0/2285363435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf419d840 con 0x7efdf4107e10 2024-03-26T06:21:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdeaffd700 1 -- 172.21.15.37:0/2285363435 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0133f0 con 0x7efdf4107e10 2024-03-26T06:21:59.015 INFO:teuthology.orchestra.run.smithi037.stdout:343597383704 2024-03-26T06:21:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.396+0000 7efdfc13f700 1 -- 172.21.15.37:0/2285363435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf41a50c0 con 0x7efdf4107e10 2024-03-26T06:21:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.397+0000 7efdfc13f700 1 -- 172.21.15.37:0/2285363435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf4068aa0 con 0x7efdf4107e10 2024-03-26T06:21:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.398+0000 7efdeaffd700 1 -- 172.21.15.37:0/2285363435 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7efdec01a020 con 0x7efdf4107e10 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.399+0000 7efdeaffd700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7efde00a29a0 0x7efde00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.399+0000 7efdf9edb700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7efde00a29a0 0x7efde00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.399+0000 7efdeaffd700 1 -- 172.21.15.37:0/2285363435 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efdec0cea40 con 0x7efdf4107e10 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.399+0000 7efdf9edb700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7efde00a29a0 0x7efde00a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efde4005950 tx=0x7efde40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.403+0000 7efdeaffd700 1 -- 172.21.15.37:0/2285363435 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7efdec010070 con 0x7efdf4107e10 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.617+0000 7efdfc13f700 1 -- 172.21.15.37:0/2285363435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7efdf402d830 con 0x7efdf4107e10 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.617+0000 7efdeaffd700 1 -- 172.21.15.37:0/2285363435 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7efdec09b990 con 0x7efdf4107e10 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.619+0000 7efdfc13f700 1 -- 172.21.15.37:0/2285363435 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7efde00a29a0 msgr2=0x7efde00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.619+0000 7efdfc13f700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7efde00a29a0 0x7efde00a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efde4005950 tx=0x7efde40058e0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.619+0000 7efdfc13f700 1 -- 172.21.15.37:0/2285363435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf4107e10 msgr2=0x7efdf419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.619+0000 7efdfc13f700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf4107e10 0x7efdf419ce10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efdec007ee0 tx=0x7efdec00d3b0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.619+0000 7efdfc13f700 1 -- 172.21.15.37:0/2285363435 shutdown_connections 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.619+0000 7efdfc13f700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf4107e10 0x7efdf419ce10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.619+0000 7efdfc13f700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7efde00a29a0 0x7efde00a4e60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.619+0000 7efdfc13f700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdf4101e30 0x7efdf4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.619+0000 7efdfc13f700 1 --2- 172.21.15.37:0/2285363435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdf41027f0 0x7efdf4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.619+0000 7efdfc13f700 1 -- 172.21.15.37:0/2285363435 >> 172.21.15.37:0/2285363435 conn(0x7efdf4078f10 msgr2=0x7efdf4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:59.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.620+0000 7efdfc13f700 1 -- 172.21.15.37:0/2285363435 shutdown_connections 2024-03-26T06:21:59.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:58.620+0000 7efdfc13f700 1 -- 172.21.15.37:0/2285363435 wait complete. 2024-03-26T06:21:59.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:59 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2285363435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T06:21:59.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:21:59 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1350701701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T06:21:59.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:59 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2285363435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T06:21:59.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:21:59 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1350701701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T06:21:59.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:59 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2285363435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T06:21:59.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:21:59 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1350701701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T06:21:59.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.684+0000 7f7074a76700 1 -- 172.21.15.37:0/991993229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7070069a90 msgr2=0x7f7070069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.684+0000 7f7074a76700 1 --2- 172.21.15.37:0/991993229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7070069a90 0x7f7070069ef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f70580099c0 tx=0x7f7058009cd0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.684+0000 7f7074a76700 1 -- 172.21.15.37:0/991993229 shutdown_connections 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.684+0000 7f7074a76700 1 --2- 172.21.15.37:0/991993229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707006a430 0x7f707010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.684+0000 7f7074a76700 1 --2- 172.21.15.37:0/991993229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7070107e30 0x7f7070108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.684+0000 7f7074a76700 1 --2- 172.21.15.37:0/991993229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7070069a90 0x7f7070069ef0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.684+0000 7f7074a76700 1 -- 172.21.15.37:0/991993229 >> 172.21.15.37:0/991993229 conn(0x7f7070077c10 msgr2=0x7f7070078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.684+0000 7f7074a76700 1 -- 172.21.15.37:0/991993229 shutdown_connections 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.684+0000 7f7074a76700 1 -- 172.21.15.37:0/991993229 wait complete. 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.685+0000 7f7074a76700 1 Processor -- start 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.685+0000 7f7074a76700 1 -- start start 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.685+0000 7f7074a76700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7070069a90 0x7f7070198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.685+0000 7f7074a76700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f707006a430 0x7f7070198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.685+0000 7f7074a76700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7070107e30 0x7f707019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.685+0000 7f7074a76700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7070112170 con 0x7f707006a430 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.685+0000 7f7074a76700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7070111ff0 con 0x7f7070069a90 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.685+0000 7f7074a76700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70701122f0 con 0x7f7070107e30 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.685+0000 7f706dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f707006a430 0x7f7070198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.685+0000 7f706dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f707006a430 0x7f7070198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47824/0 (socket says 172.21.15.37:47824) 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f706dd9b700 1 -- 172.21.15.37:0/1269092387 learned_addr learned my addr 172.21.15.37:0/1269092387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f706e59c700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7070069a90 0x7f7070198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f706ed9d700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7070107e30 0x7f707019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f706dd9b700 1 -- 172.21.15.37:0/1269092387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7070107e30 msgr2=0x7f707019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f706dd9b700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7070107e30 0x7f707019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f706dd9b700 1 -- 172.21.15.37:0/1269092387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7070069a90 msgr2=0x7f7070198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f706dd9b700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7070069a90 0x7f7070198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f706dd9b700 1 -- 172.21.15.37:0/1269092387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707019d690 con 0x7f707006a430 2024-03-26T06:21:59.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f706dd9b700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f707006a430 0x7f7070198bc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7058009fd0 tx=0x7f7058003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:59.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f70677fe700 1 -- 172.21.15.37:0/1269092387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058027070 con 0x7f707006a430 2024-03-26T06:21:59.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f70677fe700 1 -- 172.21.15.37:0/1269092387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7058004050 con 0x7f707006a430 2024-03-26T06:21:59.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f7074a76700 1 -- 172.21.15.37:0/1269092387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707019d920 con 0x7f707006a430 2024-03-26T06:21:59.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f70677fe700 1 -- 172.21.15.37:0/1269092387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058022db0 con 0x7f707006a430 2024-03-26T06:21:59.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.686+0000 7f7074a76700 1 -- 172.21.15.37:0/1269092387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70701a5200 con 0x7f707006a430 2024-03-26T06:21:59.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.687+0000 7f7074a76700 1 -- 172.21.15.37:0/1269092387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70701034e0 con 0x7f707006a430 2024-03-26T06:21:59.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.688+0000 7f70677fe700 1 -- 172.21.15.37:0/1269092387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f705802a070 con 0x7f707006a430 2024-03-26T06:21:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.689+0000 7f70677fe700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f705c0a2920 0x7f705c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:21:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.689+0000 7f706e59c700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f705c0a2920 0x7f705c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:21:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.689+0000 7f70677fe700 1 -- 172.21.15.37:0/1269092387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f70580d0250 con 0x7f707006a430 2024-03-26T06:21:59.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.690+0000 7f706e59c700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f705c0a2920 0x7f705c0a4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7060009a10 tx=0x7f7060009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:21:59.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.693+0000 7f70677fe700 1 -- 172.21.15.37:0/1269092387 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7058031070 con 0x7f707006a430 2024-03-26T06:21:59.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.898+0000 7f7074a76700 1 -- 172.21.15.37:0/1269092387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f70701999e0 con 0x7f707006a430 2024-03-26T06:21:59.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.899+0000 7f70677fe700 1 -- 172.21.15.37:0/1269092387 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7058013070 con 0x7f707006a430 2024-03-26T06:21:59.900 INFO:teuthology.orchestra.run.smithi037.stdout:77309411400 2024-03-26T06:21:59.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 -- 172.21.15.37:0/1269092387 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f705c0a2920 msgr2=0x7f705c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:59.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f705c0a2920 0x7f705c0a4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7060009a10 tx=0x7f7060009190 comp rx=0 tx=0).stop 2024-03-26T06:21:59.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 -- 172.21.15.37:0/1269092387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f707006a430 msgr2=0x7f7070198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:21:59.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f707006a430 0x7f7070198bc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7058009fd0 tx=0x7f7058003680 comp rx=0 tx=0).stop 2024-03-26T06:21:59.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 -- 172.21.15.37:0/1269092387 shutdown_connections 2024-03-26T06:21:59.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f705c0a2920 0x7f705c0a4de0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7070107e30 0x7f707019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7070069a90 0x7f7070198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 --2- 172.21.15.37:0/1269092387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f707006a430 0x7f7070198bc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:21:59.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 -- 172.21.15.37:0/1269092387 >> 172.21.15.37:0/1269092387 conn(0x7f7070077c10 msgr2=0x7f70700ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:21:59.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 -- 172.21.15.37:0/1269092387 shutdown_connections 2024-03-26T06:21:59.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:21:59.901+0000 7f7074a76700 1 -- 172.21.15.37:0/1269092387 wait complete. 2024-03-26T06:22:00.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:00 smithi037 ceph-mon[31974]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 57 B/s, 0 objects/s recovering 2024-03-26T06:22:00.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:00 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1269092387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T06:22:00.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:00 smithi066 ceph-mon[36504]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 57 B/s, 0 objects/s recovering 2024-03-26T06:22:00.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:00 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1269092387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T06:22:00.392 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645338 got 309237645342 for osd.10 2024-03-26T06:22:00.392 DEBUG:teuthology.parallel:result is None 2024-03-26T06:22:00.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:00 smithi149 ceph-mon[36681]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 57 B/s, 0 objects/s recovering 2024-03-26T06:22:00.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:00 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1269092387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T06:22:01.798 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757304 for osd.5 2024-03-26T06:22:01.798 DEBUG:teuthology.parallel:result is None 2024-03-26T06:22:02.036 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:02 smithi037 ceph-mon[31974]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 48 B/s, 0 objects/s recovering 2024-03-26T06:22:02.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:02 smithi066 ceph-mon[36504]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 48 B/s, 0 objects/s recovering 2024-03-26T06:22:02.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:02 smithi149 ceph-mon[36681]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 48 B/s, 0 objects/s recovering 2024-03-26T06:22:02.793 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411400 for osd.2 2024-03-26T06:22:02.793 DEBUG:teuthology.parallel:result is None 2024-03-26T06:22:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.231+0000 7f977e3dd700 1 -- 172.21.15.37:0/1713455968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9778074010 msgr2=0x7f97780743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.231+0000 7f977e3dd700 1 --2- 172.21.15.37:0/1713455968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9778074010 0x7f97780743f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9768009990 tx=0x7f9768009ca0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.232+0000 7f977e3dd700 1 -- 172.21.15.37:0/1713455968 shutdown_connections 2024-03-26T06:22:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.232+0000 7f977e3dd700 1 --2- 172.21.15.37:0/1713455968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9778074010 0x7f97780743f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.232+0000 7f977e3dd700 1 --2- 172.21.15.37:0/1713455968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f977810e810 0x7f9778110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.232+0000 7f977e3dd700 1 --2- 172.21.15.37:0/1713455968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9778074930 0x7f977810e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.232+0000 7f977e3dd700 1 -- 172.21.15.37:0/1713455968 >> 172.21.15.37:0/1713455968 conn(0x7f9778078e70 msgr2=0x7f9778079280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.232+0000 7f977e3dd700 1 -- 172.21.15.37:0/1713455968 shutdown_connections 2024-03-26T06:22:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.232+0000 7f977e3dd700 1 -- 172.21.15.37:0/1713455968 wait complete. 2024-03-26T06:22:03.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.232+0000 7f977e3dd700 1 Processor -- start 2024-03-26T06:22:03.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.232+0000 7f977e3dd700 1 -- start start 2024-03-26T06:22:03.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f977e3dd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9778074010 0x7f97781a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:03.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f977e3dd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9778074930 0x7f977819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:03.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f977e3dd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f977810e810 0x7f977819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:03.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f977e3dd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97781142a0 con 0x7f9778074930 2024-03-26T06:22:03.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f977e3dd700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9778114120 con 0x7f977810e810 2024-03-26T06:22:03.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f977e3dd700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9778114420 con 0x7f9778074010 2024-03-26T06:22:03.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f97777fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9778074930 0x7f977819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:03.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f97777fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9778074930 0x7f977819f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47862/0 (socket says 172.21.15.37:47862) 2024-03-26T06:22:03.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f97777fe700 1 -- 172.21.15.37:0/2074441273 learned_addr learned my addr 172.21.15.37:0/2074441273 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:03.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f9777fff700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9778074010 0x7f97781a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:03.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f977c97a700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f977810e810 0x7f977819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:03.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f97777fe700 1 -- 172.21.15.37:0/2074441273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9778074010 msgr2=0x7f97781a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:03.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f97777fe700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9778074010 0x7f97781a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f97777fe700 1 -- 172.21.15.37:0/2074441273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f977810e810 msgr2=0x7f977819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:03.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f97777fe700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f977810e810 0x7f977819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f97777fe700 1 -- 172.21.15.37:0/2074441273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97781a0350 con 0x7f9778074930 2024-03-26T06:22:03.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f977c97a700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f977810e810 0x7f977819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:03.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.233+0000 7f97777fe700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9778074930 0x7f977819f6e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f976c00d950 tx=0x7f976c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:03.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.234+0000 7f97757fa700 1 -- 172.21.15.37:0/2074441273 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c016070 con 0x7f9778074930 2024-03-26T06:22:03.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.234+0000 7f977e3dd700 1 -- 172.21.15.37:0/2074441273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977810d320 con 0x7f9778074930 2024-03-26T06:22:03.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.234+0000 7f97757fa700 1 -- 172.21.15.37:0/2074441273 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f976c009e20 con 0x7f9778074930 2024-03-26T06:22:03.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.234+0000 7f97757fa700 1 -- 172.21.15.37:0/2074441273 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c00f620 con 0x7f9778074930 2024-03-26T06:22:03.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.234+0000 7f977e3dd700 1 -- 172.21.15.37:0/2074441273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977810d860 con 0x7f9778074930 2024-03-26T06:22:03.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.235+0000 7f977e3dd700 1 -- 172.21.15.37:0/2074441273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9778068aa0 con 0x7f9778074930 2024-03-26T06:22:03.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.236+0000 7f97757fa700 1 -- 172.21.15.37:0/2074441273 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f976c019070 con 0x7f9778074930 2024-03-26T06:22:03.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.236+0000 7f97757fa700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f97600a2920 0x7f97600a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:03.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.237+0000 7f9777fff700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f97600a2920 0x7f97600a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:03.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.237+0000 7f97757fa700 1 -- 172.21.15.37:0/2074441273 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f976c0ce7f0 con 0x7f9778074930 2024-03-26T06:22:03.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.237+0000 7f9777fff700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f97600a2920 0x7f97600a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9768006010 tx=0x7f976800b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:03.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.245+0000 7f97757fa700 1 -- 172.21.15.37:0/2074441273 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f976c09b740 con 0x7f9778074930 2024-03-26T06:22:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.138+0000 7ffaaed00700 1 -- 172.21.15.37:0/2425955880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa8107e30 msgr2=0x7ffaa8108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.138+0000 7ffaaed00700 1 --2- 172.21.15.37:0/2425955880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa8107e30 0x7ffaa8108210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0099c0 tx=0x7ffa9c009cd0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.138+0000 7ffaaed00700 1 -- 172.21.15.37:0/2425955880 shutdown_connections 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.138+0000 7ffaaed00700 1 --2- 172.21.15.37:0/2425955880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa8107e30 0x7ffaa8108210 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.138+0000 7ffaaed00700 1 --2- 172.21.15.37:0/2425955880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaa806a430 0x7ffaa810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.138+0000 7ffaaed00700 1 --2- 172.21.15.37:0/2425955880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaa8069a90 0x7ffaa8069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.138+0000 7ffaaed00700 1 -- 172.21.15.37:0/2425955880 >> 172.21.15.37:0/2425955880 conn(0x7ffaa8077c10 msgr2=0x7ffaa8078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.138+0000 7ffaaed00700 1 -- 172.21.15.37:0/2425955880 shutdown_connections 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.138+0000 7ffaaed00700 1 -- 172.21.15.37:0/2425955880 wait complete. 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.138+0000 7ffaaed00700 1 Processor -- start 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaaed00700 1 -- start start 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaaed00700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaa8069a90 0x7ffaa81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaaed00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaa806a430 0x7ffaa8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaaed00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa8107e30 0x7ffaa819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaaed00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa8112090 con 0x7ffaa8107e30 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaaed00700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa8111f10 con 0x7ffaa8069a90 2024-03-26T06:22:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaaed00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa8112210 con 0x7ffaa806a430 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaad29d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa8107e30 0x7ffaa819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaad29d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa8107e30 0x7ffaa819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47844/0 (socket says 172.21.15.37:47844) 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaad29d700 1 -- 172.21.15.37:0/3875571121 learned_addr learned my addr 172.21.15.37:0/3875571121 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaaca9c700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaa8069a90 0x7ffaa81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaa7fff700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaa806a430 0x7ffaa8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaad29d700 1 -- 172.21.15.37:0/3875571121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaa806a430 msgr2=0x7ffaa8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaad29d700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaa806a430 0x7ffaa8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaad29d700 1 -- 172.21.15.37:0/3875571121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaa8069a90 msgr2=0x7ffaa81985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaad29d700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaa8069a90 0x7ffaa81985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaad29d700 1 -- 172.21.15.37:0/3875571121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa819d560 con 0x7ffaa8107e30 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaaca9c700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaa8069a90 0x7ffaa81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.139+0000 7ffaad29d700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa8107e30 0x7ffaa819ce20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0007ee0 tx=0x7ffaa000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.140+0000 7ffaa5ffb700 1 -- 172.21.15.37:0/3875571121 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa000dbd0 con 0x7ffaa8107e30 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.140+0000 7ffaa5ffb700 1 -- 172.21.15.37:0/3875571121 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffaa000dd70 con 0x7ffaa8107e30 2024-03-26T06:22:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.140+0000 7ffaa5ffb700 1 -- 172.21.15.37:0/3875571121 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00133f0 con 0x7ffaa8107e30 2024-03-26T06:22:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.140+0000 7ffaaed00700 1 -- 172.21.15.37:0/3875571121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa819d850 con 0x7ffaa8107e30 2024-03-26T06:22:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.140+0000 7ffaaed00700 1 -- 172.21.15.37:0/3875571121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa8100ce0 con 0x7ffaa8107e30 2024-03-26T06:22:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.141+0000 7ffaaed00700 1 -- 172.21.15.37:0/3875571121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa8068aa0 con 0x7ffaa8107e30 2024-03-26T06:22:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.142+0000 7ffaa5ffb700 1 -- 172.21.15.37:0/3875571121 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ffaa001a020 con 0x7ffaa8107e30 2024-03-26T06:22:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.143+0000 7ffaa5ffb700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa980a29a0 0x7ffa980a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.143+0000 7ffaaca9c700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa980a29a0 0x7ffa980a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.143+0000 7ffaa5ffb700 1 -- 172.21.15.37:0/3875571121 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffaa00ce310 con 0x7ffaa8107e30 2024-03-26T06:22:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.143+0000 7ffaaca9c700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa980a29a0 0x7ffa980a4e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c000c00 tx=0x7ffa9c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.147+0000 7ffaa5ffb700 1 -- 172.21.15.37:0/3875571121 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ffaa0010070 con 0x7ffaa8107e30 2024-03-26T06:22:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.361+0000 7ffaaed00700 1 -- 172.21.15.37:0/3875571121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffaa802d830 con 0x7ffaa8107e30 2024-03-26T06:22:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.361+0000 7ffaa5ffb700 1 -- 172.21.15.37:0/3875571121 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffaa009b200 con 0x7ffaa8107e30 2024-03-26T06:22:03.366 INFO:teuthology.orchestra.run.smithi037.stdout:34359738450 2024-03-26T06:22:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.363+0000 7ffaaed00700 1 -- 172.21.15.37:0/3875571121 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa980a29a0 msgr2=0x7ffa980a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.363+0000 7ffaaed00700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa980a29a0 0x7ffa980a4e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c000c00 tx=0x7ffa9c005fd0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.363+0000 7ffaaed00700 1 -- 172.21.15.37:0/3875571121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa8107e30 msgr2=0x7ffaa819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.364+0000 7ffaaed00700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa8107e30 0x7ffaa819ce20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0007ee0 tx=0x7ffaa000d3b0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.364+0000 7ffaaed00700 1 -- 172.21.15.37:0/3875571121 shutdown_connections 2024-03-26T06:22:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.364+0000 7ffaaed00700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa8107e30 0x7ffaa819ce20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.364+0000 7ffaaed00700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ffa980a29a0 0x7ffa980a4e60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.364+0000 7ffaaed00700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffaa8069a90 0x7ffaa81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.364+0000 7ffaaed00700 1 --2- 172.21.15.37:0/3875571121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffaa806a430 0x7ffaa8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.364+0000 7ffaaed00700 1 -- 172.21.15.37:0/3875571121 >> 172.21.15.37:0/3875571121 conn(0x7ffaa8077c10 msgr2=0x7ffaa80ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.364+0000 7ffaaed00700 1 -- 172.21.15.37:0/3875571121 shutdown_connections 2024-03-26T06:22:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.364+0000 7ffaaed00700 1 -- 172.21.15.37:0/3875571121 wait complete. 2024-03-26T06:22:03.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.458+0000 7f977e3dd700 1 -- 172.21.15.37:0/2074441273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f977810dac0 con 0x7f9778074930 2024-03-26T06:22:03.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.458+0000 7f97757fa700 1 -- 172.21.15.37:0/2074441273 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f976c09b040 con 0x7f9778074930 2024-03-26T06:22:03.460 INFO:teuthology.orchestra.run.smithi037.stdout:287762808869 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 -- 172.21.15.37:0/2074441273 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f97600a2920 msgr2=0x7f97600a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f97600a2920 0x7f97600a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9768006010 tx=0x7f976800b560 comp rx=0 tx=0).stop 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 -- 172.21.15.37:0/2074441273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9778074930 msgr2=0x7f977819f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9778074930 0x7f977819f6e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f976c00d950 tx=0x7f976c00dc60 comp rx=0 tx=0).stop 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 -- 172.21.15.37:0/2074441273 shutdown_connections 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f97600a2920 0x7f97600a4de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f977810e810 0x7f977819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9778074010 0x7f97781a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 --2- 172.21.15.37:0/2074441273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9778074930 0x7f977819f6e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 -- 172.21.15.37:0/2074441273 >> 172.21.15.37:0/2074441273 conn(0x7f9778078e70 msgr2=0x7f9778104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 -- 172.21.15.37:0/2074441273 shutdown_connections 2024-03-26T06:22:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:03.461+0000 7f977e3dd700 1 -- 172.21.15.37:0/2074441273 wait complete. 2024-03-26T06:22:04.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:04 smithi037 ceph-mon[31974]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 48 B/s, 0 objects/s recovering 2024-03-26T06:22:04.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:04 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3875571121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T06:22:04.174 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:04 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2074441273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T06:22:04.201 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808869 for osd.9 2024-03-26T06:22:04.202 DEBUG:teuthology.parallel:result is None 2024-03-26T06:22:04.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:04 smithi066 ceph-mon[36504]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 48 B/s, 0 objects/s recovering 2024-03-26T06:22:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:04 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3875571121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T06:22:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:04 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2074441273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T06:22:04.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.403+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/1234305406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e0106d10 msgr2=0x7ff6e01070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:04.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.403+0000 7ff6e4d4b700 1 --2- 172.21.15.37:0/1234305406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e0106d10 0x7ff6e01070f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009990 tx=0x7ff6c8009ca0 comp rx=0 tx=0).stop 2024-03-26T06:22:04.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.403+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/1234305406 shutdown_connections 2024-03-26T06:22:04.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.403+0000 7ff6e4d4b700 1 --2- 172.21.15.37:0/1234305406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e0106d10 0x7ff6e01070f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:04.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.403+0000 7ff6e4d4b700 1 --2- 172.21.15.37:0/1234305406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6e0101690 0x7ff6e010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:04.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.403+0000 7ff6e4d4b700 1 --2- 172.21.15.37:0/1234305406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e0100cd0 0x7ff6e0101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:04.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.403+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/1234305406 >> 172.21.15.37:0/1234305406 conn(0x7ff6e0077c80 msgr2=0x7ff6e0078090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:04.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.403+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/1234305406 shutdown_connections 2024-03-26T06:22:04.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.403+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/1234305406 wait complete. 2024-03-26T06:22:04.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.404+0000 7ff6e4d4b700 1 Processor -- start 2024-03-26T06:22:04.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.404+0000 7ff6e4d4b700 1 -- start start 2024-03-26T06:22:04.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.404+0000 7ff6e4d4b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6e0100cd0 0x7ff6e01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.404+0000 7ff6e4d4b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e0101690 0x7ff6e0198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.404+0000 7ff6e4d4b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e0106d10 0x7ff6e019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.404+0000 7ff6e4d4b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0111080 con 0x7ff6e0101690 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.404+0000 7ff6e4d4b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0110f00 con 0x7ff6e0100cd0 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.404+0000 7ff6e4d4b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0111200 con 0x7ff6e0106d10 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.404+0000 7ff6ddd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e0101690 0x7ff6e0198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6ddd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e0101690 0x7ff6e0198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47870/0 (socket says 172.21.15.37:47870) 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6ddd9b700 1 -- 172.21.15.37:0/2501457916 learned_addr learned my addr 172.21.15.37:0/2501457916 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6ded9d700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e0106d10 0x7ff6e019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6de59c700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6e0100cd0 0x7ff6e01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6ddd9b700 1 -- 172.21.15.37:0/2501457916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e0106d10 msgr2=0x7ff6e019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6ddd9b700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e0106d10 0x7ff6e019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:04.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6ddd9b700 1 -- 172.21.15.37:0/2501457916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6e0100cd0 msgr2=0x7ff6e01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:04.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6ddd9b700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6e0100cd0 0x7ff6e01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:04.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6ddd9b700 1 -- 172.21.15.37:0/2501457916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e019d6d0 con 0x7ff6e0101690 2024-03-26T06:22:04.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6de59c700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6e0100cd0 0x7ff6e01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:04.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6ddd9b700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e0101690 0x7ff6e0198c00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000b530 tx=0x7ff6d000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:04.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6d77fe700 1 -- 172.21.15.37:0/2501457916 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0015020 con 0x7ff6e0101690 2024-03-26T06:22:04.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/2501457916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e019d9c0 con 0x7ff6e0101690 2024-03-26T06:22:04.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6d77fe700 1 -- 172.21.15.37:0/2501457916 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6d0011d10 con 0x7ff6e0101690 2024-03-26T06:22:04.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6d77fe700 1 -- 172.21.15.37:0/2501457916 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000f2e0 con 0x7ff6e0101690 2024-03-26T06:22:04.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.405+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/2501457916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e01a5240 con 0x7ff6e0101690 2024-03-26T06:22:04.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.406+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/2501457916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e0068aa0 con 0x7ff6e0101690 2024-03-26T06:22:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:04 smithi149 ceph-mon[36681]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 48 B/s, 0 objects/s recovering 2024-03-26T06:22:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:04 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3875571121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T06:22:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:04 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2074441273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T06:22:04.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.407+0000 7ff6d77fe700 1 -- 172.21.15.37:0/2501457916 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff6d0018020 con 0x7ff6e0101690 2024-03-26T06:22:04.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.408+0000 7ff6d77fe700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff6cc0a2920 0x7ff6cc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:04.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.408+0000 7ff6de59c700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff6cc0a2920 0x7ff6cc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:04.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.408+0000 7ff6d77fe700 1 -- 172.21.15.37:0/2501457916 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff6d00ce740 con 0x7ff6e0101690 2024-03-26T06:22:04.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.409+0000 7ff6de59c700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff6cc0a2920 0x7ff6cc0a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800b5c0 tx=0x7ff6c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:04.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.412+0000 7ff6d77fe700 1 -- 172.21.15.37:0/2501457916 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff6d001f020 con 0x7ff6e0101690 2024-03-26T06:22:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.619+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/2501457916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff6e0199a20 con 0x7ff6e0101690 2024-03-26T06:22:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.619+0000 7ff6d77fe700 1 -- 172.21.15.37:0/2501457916 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff6e0199a20 con 0x7ff6e0101690 2024-03-26T06:22:04.621 INFO:teuthology.orchestra.run.smithi037.stdout:141733920831 2024-03-26T06:22:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.621+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/2501457916 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff6cc0a2920 msgr2=0x7ff6cc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.621+0000 7ff6e4d4b700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff6cc0a2920 0x7ff6cc0a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800b5c0 tx=0x7ff6c8005fb0 comp rx=0 tx=0).stop 2024-03-26T06:22:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.621+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/2501457916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e0101690 msgr2=0x7ff6e0198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.621+0000 7ff6e4d4b700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e0101690 0x7ff6e0198c00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000b530 tx=0x7ff6d000b840 comp rx=0 tx=0).stop 2024-03-26T06:22:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.621+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/2501457916 shutdown_connections 2024-03-26T06:22:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.621+0000 7ff6e4d4b700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff6cc0a2920 0x7ff6cc0a4de0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.621+0000 7ff6e4d4b700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e0106d10 0x7ff6e019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.621+0000 7ff6e4d4b700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6e0100cd0 0x7ff6e01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:04.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.621+0000 7ff6e4d4b700 1 --2- 172.21.15.37:0/2501457916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6e0101690 0x7ff6e0198c00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:04.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.621+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/2501457916 >> 172.21.15.37:0/2501457916 conn(0x7ff6e0077c80 msgr2=0x7ff6e006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:04.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.622+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/2501457916 shutdown_connections 2024-03-26T06:22:04.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:04.622+0000 7ff6e4d4b700 1 -- 172.21.15.37:0/2501457916 wait complete. 2024-03-26T06:22:04.820 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738450 for osd.0 2024-03-26T06:22:04.820 DEBUG:teuthology.parallel:result is None 2024-03-26T06:22:05.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:05 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2501457916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T06:22:05.316 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383700 got 343597383704 for osd.11 2024-03-26T06:22:05.317 DEBUG:teuthology.parallel:result is None 2024-03-26T06:22:05.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:05 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2501457916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T06:22:05.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:05 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2501457916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T06:22:06.018 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495660 got 197568495666 for osd.6 2024-03-26T06:22:06.018 DEBUG:teuthology.parallel:result is None 2024-03-26T06:22:06.350 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:06 smithi066 ceph-mon[36504]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 48 B/s, 0 objects/s recovering 2024-03-26T06:22:06.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:06 smithi149 ceph-mon[36681]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 48 B/s, 0 objects/s recovering 2024-03-26T06:22:06.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:06 smithi037 ceph-mon[31974]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 48 B/s, 0 objects/s recovering 2024-03-26T06:22:06.480 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920825 got 141733920831 for osd.4 2024-03-26T06:22:06.480 DEBUG:teuthology.parallel:result is None 2024-03-26T06:22:06.480 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-26T06:22:06.480 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T06:22:07.406 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:22:08.164 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:08 smithi037 ceph-mon[31974]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:08.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:08 smithi066 ceph-mon[36504]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:08.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:08 smithi149 ceph-mon[36681]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:09.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.992+0000 7f42fd7c0700 1 -- 172.21.15.37:0/361876151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f8101e30 msgr2=0x7f42f81022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:09.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.992+0000 7f42fd7c0700 1 --2- 172.21.15.37:0/361876151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f8101e30 0x7f42f81022b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0099c0 tx=0x7f42ec009cd0 comp rx=0 tx=0).stop 2024-03-26T06:22:09.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 -- 172.21.15.37:0/361876151 shutdown_connections 2024-03-26T06:22:09.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 --2- 172.21.15.37:0/361876151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42f81027f0 0x7f42f810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:09.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 --2- 172.21.15.37:0/361876151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42f8107e10 0x7f42f81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:09.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 --2- 172.21.15.37:0/361876151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f8101e30 0x7f42f81022b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:09.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 -- 172.21.15.37:0/361876151 >> 172.21.15.37:0/361876151 conn(0x7f42f8078f10 msgr2=0x7f42f8079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:09.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 -- 172.21.15.37:0/361876151 shutdown_connections 2024-03-26T06:22:09.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 -- 172.21.15.37:0/361876151 wait complete. 2024-03-26T06:22:09.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 Processor -- start 2024-03-26T06:22:09.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 -- start start 2024-03-26T06:22:09.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42f8101e30 0x7f42f81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:09.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42f81027f0 0x7f42f8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:09.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.993+0000 7f42fd7c0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f8107e10 0x7f42f819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:09.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42fd7c0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f81120b0 con 0x7f42f8107e10 2024-03-26T06:22:09.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42fd7c0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f8111f30 con 0x7f42f8101e30 2024-03-26T06:22:09.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42fd7c0700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f8112230 con 0x7f42f81027f0 2024-03-26T06:22:09.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f8107e10 0x7f42f819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:09.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f8107e10 0x7f42f819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52778/0 (socket says 172.21.15.37:52778) 2024-03-26T06:22:09.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f77fe700 1 -- 172.21.15.37:0/2346295506 learned_addr learned my addr 172.21.15.37:0/2346295506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:09.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f67fc700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42f81027f0 0x7f42f8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:09.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f6ffd700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42f8101e30 0x7f42f81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:09.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f77fe700 1 -- 172.21.15.37:0/2346295506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42f81027f0 msgr2=0x7f42f8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:09.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f77fe700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42f81027f0 0x7f42f8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:09.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f77fe700 1 -- 172.21.15.37:0/2346295506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42f8101e30 msgr2=0x7f42f81984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:09.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f77fe700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42f8101e30 0x7f42f81984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:09.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f77fe700 1 -- 172.21.15.37:0/2346295506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f819d4f0 con 0x7f42f8107e10 2024-03-26T06:22:09.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f6ffd700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42f8101e30 0x7f42f81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:09.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42f77fe700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f8107e10 0x7f42f819cdb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f42f000ef30 tx=0x7f42f000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:09.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42dffff700 1 -- 172.21.15.37:0/2346295506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f00092e0 con 0x7f42f8107e10 2024-03-26T06:22:09.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42fd7c0700 1 -- 172.21.15.37:0/2346295506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f819d780 con 0x7f42f8107e10 2024-03-26T06:22:09.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42dffff700 1 -- 172.21.15.37:0/2346295506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42f0004bb0 con 0x7f42f8107e10 2024-03-26T06:22:09.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42dffff700 1 -- 172.21.15.37:0/2346295506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0004280 con 0x7f42f8107e10 2024-03-26T06:22:09.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.994+0000 7f42fd7c0700 1 -- 172.21.15.37:0/2346295506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f81a5060 con 0x7f42f8107e10 2024-03-26T06:22:09.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.995+0000 7f42fd7c0700 1 -- 172.21.15.37:0/2346295506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f8068aa0 con 0x7f42f8107e10 2024-03-26T06:22:09.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.996+0000 7f42dffff700 1 -- 172.21.15.37:0/2346295506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f42f001b030 con 0x7f42f8107e10 2024-03-26T06:22:09.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.997+0000 7f42dffff700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42e00a2920 0x7f42e00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:09.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.997+0000 7f42f6ffd700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42e00a2920 0x7f42e00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:09.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.997+0000 7f42dffff700 1 -- 172.21.15.37:0/2346295506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f42f00cf220 con 0x7f42f8107e10 2024-03-26T06:22:09.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:08.997+0000 7f42f6ffd700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42e00a2920 0x7f42e00a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f42e800bcf0 tx=0x7f42e800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:09.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.001+0000 7f42dffff700 1 -- 172.21.15.37:0/2346295506 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f42f0022020 con 0x7f42f8107e10 2024-03-26T06:22:09.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.214+0000 7f42fd7c0700 1 -- 172.21.15.37:0/2346295506 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42f8062220 con 0x7f42e00a2920 2024-03-26T06:22:09.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.222+0000 7f42dffff700 1 -- 172.21.15.37:0/2346295506 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258763 (secure 0 0 0) 0x7f42f8062220 con 0x7f42e00a2920 2024-03-26T06:22:09.228 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:09.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 -- 172.21.15.37:0/2346295506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42e00a2920 msgr2=0x7f42e00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42e00a2920 0x7f42e00a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f42e800bcf0 tx=0x7f42e800b2c0 comp rx=0 tx=0).stop 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 -- 172.21.15.37:0/2346295506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f8107e10 msgr2=0x7f42f819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f8107e10 0x7f42f819cdb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f42f000ef30 tx=0x7f42f000c450 comp rx=0 tx=0).stop 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 -- 172.21.15.37:0/2346295506 shutdown_connections 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f8107e10 0x7f42f819cdb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f42e00a2920 0x7f42e00a4de0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42f8101e30 0x7f42f81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 --2- 172.21.15.37:0/2346295506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42f81027f0 0x7f42f8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 -- 172.21.15.37:0/2346295506 >> 172.21.15.37:0/2346295506 conn(0x7f42f8078f10 msgr2=0x7f42f81009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 -- 172.21.15.37:0/2346295506 shutdown_connections 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:09.225+0000 7f42fd7c0700 1 -- 172.21.15.37:0/2346295506 wait complete. 2024-03-26T06:22:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-26T06:22:09.916 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-03-26T06:22:08.763168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656327,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":851,"num_read_kb":551,"num_write":416,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366684,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575140,"statfs":{"total":1151940427776,"available":1151564943360,"internally_reserved":0,"allocated":13627392,"data_stored":7436457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002947"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.721646+0000","last_change":"2024-03-26T06:20:54.294436+0000","last_active":"2024-03-26T06:21:48.721646+0000","last_peered":"2024-03-26T06:21:48.721646+0000","last_clean":"2024-03-26T06:21:48.721646+0000","last_became_active":"2024-03-26T06:20:54.293049+0000","last_became_peered":"2024-03-26T06:20:54.293049+0000","last_unstale":"2024-03-26T06:21:48.721646+0000","last_undegraded":"2024-03-26T06:21:48.721646+0000","last_fullsized":"2024-03-26T06:21:48.721646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:44:02.729046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715930+0000","last_change":"2024-03-26T06:20:54.292560+0000","last_active":"2024-03-26T06:21:48.715930+0000","last_peered":"2024-03-26T06:21:48.715930+0000","last_clean":"2024-03-26T06:21:48.715930+0000","last_became_active":"2024-03-26T06:20:54.292166+0000","last_became_peered":"2024-03-26T06:20:54.292166+0000","last_unstale":"2024-03-26T06:21:48.715930+0000","last_undegraded":"2024-03-26T06:21:48.715930+0000","last_fullsized":"2024-03-26T06:21:48.715930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:45.255912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715710+0000","last_change":"2024-03-26T06:20:54.294350+0000","last_active":"2024-03-26T06:21:48.715710+0000","last_peered":"2024-03-26T06:21:48.715710+0000","last_clean":"2024-03-26T06:21:48.715710+0000","last_became_active":"2024-03-26T06:20:54.293813+0000","last_became_peered":"2024-03-26T06:20:54.293813+0000","last_unstale":"2024-03-26T06:21:48.715710+0000","last_undegraded":"2024-03-26T06:21:48.715710+0000","last_fullsized":"2024-03-26T06:21:48.715710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:44:31.126828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.739898+0000","last_change":"2024-03-26T06:20:54.299151+0000","last_active":"2024-03-26T06:21:48.739898+0000","last_peered":"2024-03-26T06:21:48.739898+0000","last_clean":"2024-03-26T06:21:48.739898+0000","last_became_active":"2024-03-26T06:20:54.298820+0000","last_became_peered":"2024-03-26T06:20:54.298820+0000","last_unstale":"2024-03-26T06:21:48.739898+0000","last_undegraded":"2024-03-26T06:21:48.739898+0000","last_fullsized":"2024-03-26T06:21:48.739898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:47:25.503692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718913+0000","last_change":"2024-03-26T06:20:54.299977+0000","last_active":"2024-03-26T06:21:48.718913+0000","last_peered":"2024-03-26T06:21:48.718913+0000","last_clean":"2024-03-26T06:21:48.718913+0000","last_became_active":"2024-03-26T06:20:54.299641+0000","last_became_peered":"2024-03-26T06:20:54.299641+0000","last_unstale":"2024-03-26T06:21:48.718913+0000","last_undegraded":"2024-03-26T06:21:48.718913+0000","last_fullsized":"2024-03-26T06:21:48.718913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:42:32.478970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.353109+0000","last_change":"2024-03-26T06:20:54.294308+0000","last_active":"2024-03-26T06:21:49.353109+0000","last_peered":"2024-03-26T06:21:49.353109+0000","last_clean":"2024-03-26T06:21:49.353109+0000","last_became_active":"2024-03-26T06:20:54.293882+0000","last_became_peered":"2024-03-26T06:20:54.293882+0000","last_unstale":"2024-03-26T06:21:49.353109+0000","last_undegraded":"2024-03-26T06:21:49.353109+0000","last_fullsized":"2024-03-26T06:21:49.353109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:26:26.014097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716076+0000","last_change":"2024-03-26T06:20:54.301644+0000","last_active":"2024-03-26T06:21:48.716076+0000","last_peered":"2024-03-26T06:21:48.716076+0000","last_clean":"2024-03-26T06:21:48.716076+0000","last_became_active":"2024-03-26T06:20:54.301257+0000","last_became_peered":"2024-03-26T06:20:54.301257+0000","last_unstale":"2024-03-26T06:21:48.716076+0000","last_undegraded":"2024-03-26T06:21:48.716076+0000","last_fullsized":"2024-03-26T06:21:48.716076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:43:27.602288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718499+0000","last_change":"2024-03-26T06:20:54.296421+0000","last_active":"2024-03-26T06:21:48.718499+0000","last_peered":"2024-03-26T06:21:48.718499+0000","last_clean":"2024-03-26T06:21:48.718499+0000","last_became_active":"2024-03-26T06:20:54.296086+0000","last_became_peered":"2024-03-26T06:20:54.296086+0000","last_unstale":"2024-03-26T06:21:48.718499+0000","last_undegraded":"2024-03-26T06:21:48.718499+0000","last_fullsized":"2024-03-26T06:21:48.718499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:22:25.532160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718901+0000","last_change":"2024-03-26T06:20:54.297206+0000","last_active":"2024-03-26T06:21:48.718901+0000","last_peered":"2024-03-26T06:21:48.718901+0000","last_clean":"2024-03-26T06:21:48.718901+0000","last_became_active":"2024-03-26T06:20:54.296849+0000","last_became_peered":"2024-03-26T06:20:54.296849+0000","last_unstale":"2024-03-26T06:21:48.718901+0000","last_undegraded":"2024-03-26T06:21:48.718901+0000","last_fullsized":"2024-03-26T06:21:48.718901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:24:49.972237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.353439+0000","last_change":"2024-03-26T06:20:54.295048+0000","last_active":"2024-03-26T06:21:49.353439+0000","last_peered":"2024-03-26T06:21:49.353439+0000","last_clean":"2024-03-26T06:21:49.353439+0000","last_became_active":"2024-03-26T06:20:54.294161+0000","last_became_peered":"2024-03-26T06:20:54.294161+0000","last_unstale":"2024-03-26T06:21:49.353439+0000","last_undegraded":"2024-03-26T06:21:49.353439+0000","last_fullsized":"2024-03-26T06:21:49.353439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:41:16.779030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716452+0000","last_change":"2024-03-26T06:20:54.300235+0000","last_active":"2024-03-26T06:21:48.716452+0000","last_peered":"2024-03-26T06:21:48.716452+0000","last_clean":"2024-03-26T06:21:48.716452+0000","last_became_active":"2024-03-26T06:20:54.299677+0000","last_became_peered":"2024-03-26T06:20:54.299677+0000","last_unstale":"2024-03-26T06:21:48.716452+0000","last_undegraded":"2024-03-26T06:21:48.716452+0000","last_fullsized":"2024-03-26T06:21:48.716452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:18:57.028840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.739839+0000","last_change":"2024-03-26T06:20:54.303953+0000","last_active":"2024-03-26T06:21:48.739839+0000","last_peered":"2024-03-26T06:21:48.739839+0000","last_clean":"2024-03-26T06:21:48.739839+0000","last_became_active":"2024-03-26T06:20:54.303339+0000","last_became_peered":"2024-03-26T06:20:54.303339+0000","last_unstale":"2024-03-26T06:21:48.739839+0000","last_undegraded":"2024-03-26T06:21:48.739839+0000","last_fullsized":"2024-03-26T06:21:48.739839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:54:38.511221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050284+0000","last_change":"2024-03-26T06:20:54.283535+0000","last_active":"2024-03-26T06:21:49.050284+0000","last_peered":"2024-03-26T06:21:49.050284+0000","last_clean":"2024-03-26T06:21:49.050284+0000","last_became_active":"2024-03-26T06:20:54.282820+0000","last_became_peered":"2024-03-26T06:20:54.282820+0000","last_unstale":"2024-03-26T06:21:49.050284+0000","last_undegraded":"2024-03-26T06:21:49.050284+0000","last_fullsized":"2024-03-26T06:21:49.050284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:13:07.906341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715676+0000","last_change":"2024-03-26T06:20:54.292841+0000","last_active":"2024-03-26T06:21:48.715676+0000","last_peered":"2024-03-26T06:21:48.715676+0000","last_clean":"2024-03-26T06:21:48.715676+0000","last_became_active":"2024-03-26T06:20:54.292530+0000","last_became_peered":"2024-03-26T06:20:54.292530+0000","last_unstale":"2024-03-26T06:21:48.715676+0000","last_undegraded":"2024-03-26T06:21:48.715676+0000","last_fullsized":"2024-03-26T06:21:48.715676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:35:54.592898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715598+0000","last_change":"2024-03-26T06:20:54.306753+0000","last_active":"2024-03-26T06:21:48.715598+0000","last_peered":"2024-03-26T06:21:48.715598+0000","last_clean":"2024-03-26T06:21:48.715598+0000","last_became_active":"2024-03-26T06:20:54.306379+0000","last_became_peered":"2024-03-26T06:20:54.306379+0000","last_unstale":"2024-03-26T06:21:48.715598+0000","last_undegraded":"2024-03-26T06:21:48.715598+0000","last_fullsized":"2024-03-26T06:21:48.715598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:04:19.879733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718360+0000","last_change":"2024-03-26T06:20:54.305719+0000","last_active":"2024-03-26T06:21:48.718360+0000","last_peered":"2024-03-26T06:21:48.718360+0000","last_clean":"2024-03-26T06:21:48.718360+0000","last_became_active":"2024-03-26T06:20:54.305336+0000","last_became_peered":"2024-03-26T06:20:54.305336+0000","last_unstale":"2024-03-26T06:21:48.718360+0000","last_undegraded":"2024-03-26T06:21:48.718360+0000","last_fullsized":"2024-03-26T06:21:48.718360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:22:15.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.721704+0000","last_change":"2024-03-26T06:20:54.293451+0000","last_active":"2024-03-26T06:21:48.721704+0000","last_peered":"2024-03-26T06:21:48.721704+0000","last_clean":"2024-03-26T06:21:48.721704+0000","last_became_active":"2024-03-26T06:20:54.293139+0000","last_became_peered":"2024-03-26T06:20:54.293139+0000","last_unstale":"2024-03-26T06:21:48.721704+0000","last_undegraded":"2024-03-26T06:21:48.721704+0000","last_fullsized":"2024-03-26T06:21:48.721704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:56:04.407687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719165+0000","last_change":"2024-03-26T06:20:54.290256+0000","last_active":"2024-03-26T06:21:48.719165+0000","last_peered":"2024-03-26T06:21:48.719165+0000","last_clean":"2024-03-26T06:21:48.719165+0000","last_became_active":"2024-03-26T06:20:54.289773+0000","last_became_peered":"2024-03-26T06:20:54.289773+0000","last_unstale":"2024-03-26T06:21:48.719165+0000","last_undegraded":"2024-03-26T06:21:48.719165+0000","last_fullsized":"2024-03-26T06:21:48.719165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:46:07.503946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719408+0000","last_change":"2024-03-26T06:20:54.302601+0000","last_active":"2024-03-26T06:21:48.719408+0000","last_peered":"2024-03-26T06:21:48.719408+0000","last_clean":"2024-03-26T06:21:48.719408+0000","last_became_active":"2024-03-26T06:20:54.301523+0000","last_became_peered":"2024-03-26T06:20:54.301523+0000","last_unstale":"2024-03-26T06:21:48.719408+0000","last_undegraded":"2024-03-26T06:21:48.719408+0000","last_fullsized":"2024-03-26T06:21:48.719408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:58:49.055958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":101,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:06.678048+0000","last_change":"2024-03-26T06:20:53.280275+0000","last_active":"2024-03-26T06:22:06.678048+0000","last_peered":"2024-03-26T06:22:06.678048+0000","last_clean":"2024-03-26T06:22:06.678048+0000","last_became_active":"2024-03-26T06:20:53.279916+0000","last_became_peered":"2024-03-26T06:20:53.279916+0000","last_unstale":"2024-03-26T06:22:06.678048+0000","last_undegraded":"2024-03-26T06:22:06.678048+0000","last_fullsized":"2024-03-26T06:22:06.678048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:50.844507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719551+0000","last_change":"2024-03-26T06:20:54.302720+0000","last_active":"2024-03-26T06:21:48.719551+0000","last_peered":"2024-03-26T06:21:48.719551+0000","last_clean":"2024-03-26T06:21:48.719551+0000","last_became_active":"2024-03-26T06:20:54.302171+0000","last_became_peered":"2024-03-26T06:20:54.302171+0000","last_unstale":"2024-03-26T06:21:48.719551+0000","last_undegraded":"2024-03-26T06:21:48.719551+0000","last_fullsized":"2024-03-26T06:21:48.719551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:23:12.887883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719700+0000","last_change":"2024-03-26T06:20:53.282355+0000","last_active":"2024-03-26T06:21:48.719700+0000","last_peered":"2024-03-26T06:21:48.719700+0000","last_clean":"2024-03-26T06:21:48.719700+0000","last_became_active":"2024-03-26T06:20:53.282062+0000","last_became_peered":"2024-03-26T06:20:53.282062+0000","last_unstale":"2024-03-26T06:21:48.719700+0000","last_undegraded":"2024-03-26T06:21:48.719700+0000","last_fullsized":"2024-03-26T06:21:48.719700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:58:33.398589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716131+0000","last_change":"2024-03-26T06:20:54.294924+0000","last_active":"2024-03-26T06:21:48.716131+0000","last_peered":"2024-03-26T06:21:48.716131+0000","last_clean":"2024-03-26T06:21:48.716131+0000","last_became_active":"2024-03-26T06:20:54.293971+0000","last_became_peered":"2024-03-26T06:20:54.293971+0000","last_unstale":"2024-03-26T06:21:48.716131+0000","last_undegraded":"2024-03-26T06:21:48.716131+0000","last_fullsized":"2024-03-26T06:21:48.716131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:54:15.457497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719986+0000","last_change":"2024-03-26T06:20:53.286284+0000","last_active":"2024-03-26T06:21:48.719986+0000","last_peered":"2024-03-26T06:21:48.719986+0000","last_clean":"2024-03-26T06:21:48.719986+0000","last_became_active":"2024-03-26T06:20:53.285979+0000","last_became_peered":"2024-03-26T06:20:53.285979+0000","last_unstale":"2024-03-26T06:21:48.719986+0000","last_undegraded":"2024-03-26T06:21:48.719986+0000","last_fullsized":"2024-03-26T06:21:48.719986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:33:31.989055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718710+0000","last_change":"2024-03-26T06:20:54.290137+0000","last_active":"2024-03-26T06:21:48.718710+0000","last_peered":"2024-03-26T06:21:48.718710+0000","last_clean":"2024-03-26T06:21:48.718710+0000","last_became_active":"2024-03-26T06:20:54.289727+0000","last_became_peered":"2024-03-26T06:20:54.289727+0000","last_unstale":"2024-03-26T06:21:48.718710+0000","last_undegraded":"2024-03-26T06:21:48.718710+0000","last_fullsized":"2024-03-26T06:21:48.718710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:31:26.447358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.733189+0000","last_change":"2024-03-26T06:21:48.733189+0000","last_active":"2024-03-26T06:21:48.733189+0000","last_peered":"2024-03-26T06:21:48.733189+0000","last_clean":"2024-03-26T06:21:48.733189+0000","last_became_active":"2024-03-26T06:21:48.727506+0000","last_became_peered":"2024-03-26T06:21:48.727506+0000","last_unstale":"2024-03-26T06:21:48.733189+0000","last_undegraded":"2024-03-26T06:21:48.733189+0000","last_fullsized":"2024-03-26T06:21:48.733189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:13:38.631171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.739861+0000","last_change":"2024-03-26T06:21:48.739861+0000","last_active":"2024-03-26T06:21:48.739861+0000","last_peered":"2024-03-26T06:21:48.739861+0000","last_clean":"2024-03-26T06:21:48.739861+0000","last_became_active":"2024-03-26T06:21:48.739569+0000","last_became_peered":"2024-03-26T06:21:48.739569+0000","last_unstale":"2024-03-26T06:21:48.739861+0000","last_undegraded":"2024-03-26T06:21:48.739861+0000","last_fullsized":"2024-03-26T06:21:48.739861+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:44:04.878070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:05.688099+0000","last_change":"2024-03-26T06:20:53.287298+0000","last_active":"2024-03-26T06:22:05.688099+0000","last_peered":"2024-03-26T06:22:05.688099+0000","last_clean":"2024-03-26T06:22:05.688099+0000","last_became_active":"2024-03-26T06:20:53.286987+0000","last_became_peered":"2024-03-26T06:20:53.286987+0000","last_unstale":"2024-03-26T06:22:05.688099+0000","last_undegraded":"2024-03-26T06:22:05.688099+0000","last_fullsized":"2024-03-26T06:22:05.688099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:20.343750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718700+0000","last_change":"2024-03-26T06:20:54.291410+0000","last_active":"2024-03-26T06:21:48.718700+0000","last_peered":"2024-03-26T06:21:48.718700+0000","last_clean":"2024-03-26T06:21:48.718700+0000","last_became_active":"2024-03-26T06:20:54.291085+0000","last_became_peered":"2024-03-26T06:20:54.291085+0000","last_unstale":"2024-03-26T06:21:48.718700+0000","last_undegraded":"2024-03-26T06:21:48.718700+0000","last_fullsized":"2024-03-26T06:21:48.718700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:39.727359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.352482+0000","last_change":"2024-03-26T06:20:53.285476+0000","last_active":"2024-03-26T06:21:49.352482+0000","last_peered":"2024-03-26T06:21:49.352482+0000","last_clean":"2024-03-26T06:21:49.352482+0000","last_became_active":"2024-03-26T06:20:53.285009+0000","last_became_peered":"2024-03-26T06:20:53.285009+0000","last_unstale":"2024-03-26T06:21:49.352482+0000","last_undegraded":"2024-03-26T06:21:49.352482+0000","last_fullsized":"2024-03-26T06:21:49.352482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:40.006117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724368+0000","last_change":"2024-03-26T06:20:54.299909+0000","last_active":"2024-03-26T06:21:48.724368+0000","last_peered":"2024-03-26T06:21:48.724368+0000","last_clean":"2024-03-26T06:21:48.724368+0000","last_became_active":"2024-03-26T06:20:54.299582+0000","last_became_peered":"2024-03-26T06:20:54.299582+0000","last_unstale":"2024-03-26T06:21:48.724368+0000","last_undegraded":"2024-03-26T06:21:48.724368+0000","last_fullsized":"2024-03-26T06:21:48.724368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:16:41.110029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050119+0000","last_change":"2024-03-26T06:20:53.281619+0000","last_active":"2024-03-26T06:21:49.050119+0000","last_peered":"2024-03-26T06:21:49.050119+0000","last_clean":"2024-03-26T06:21:49.050119+0000","last_became_active":"2024-03-26T06:20:53.279792+0000","last_became_peered":"2024-03-26T06:20:53.279792+0000","last_unstale":"2024-03-26T06:21:49.050119+0000","last_undegraded":"2024-03-26T06:21:49.050119+0000","last_fullsized":"2024-03-26T06:21:49.050119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:17:17.892132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050994+0000","last_change":"2024-03-26T06:21:49.050994+0000","last_active":"2024-03-26T06:21:49.050994+0000","last_peered":"2024-03-26T06:21:49.050994+0000","last_clean":"2024-03-26T06:21:49.050994+0000","last_became_active":"2024-03-26T06:21:49.050624+0000","last_became_peered":"2024-03-26T06:21:49.050624+0000","last_unstale":"2024-03-26T06:21:49.050994+0000","last_undegraded":"2024-03-26T06:21:49.050994+0000","last_fullsized":"2024-03-26T06:21:49.050994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:34:18.507960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.723902+0000","last_change":"2024-03-26T06:20:53.285367+0000","last_active":"2024-03-26T06:21:48.723902+0000","last_peered":"2024-03-26T06:21:48.723902+0000","last_clean":"2024-03-26T06:21:48.723902+0000","last_became_active":"2024-03-26T06:20:53.285022+0000","last_became_peered":"2024-03-26T06:20:53.285022+0000","last_unstale":"2024-03-26T06:21:48.723902+0000","last_undegraded":"2024-03-26T06:21:48.723902+0000","last_fullsized":"2024-03-26T06:21:48.723902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:01:32.288961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718531+0000","last_change":"2024-03-26T06:20:54.290125+0000","last_active":"2024-03-26T06:21:48.718531+0000","last_peered":"2024-03-26T06:21:48.718531+0000","last_clean":"2024-03-26T06:21:48.718531+0000","last_became_active":"2024-03-26T06:20:54.289739+0000","last_became_peered":"2024-03-26T06:20:54.289739+0000","last_unstale":"2024-03-26T06:21:48.718531+0000","last_undegraded":"2024-03-26T06:21:48.718531+0000","last_fullsized":"2024-03-26T06:21:48.718531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:51:01.087526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":161,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:07.690598+0000","last_change":"2024-03-26T06:20:53.289898+0000","last_active":"2024-03-26T06:22:07.690598+0000","last_peered":"2024-03-26T06:22:07.690598+0000","last_clean":"2024-03-26T06:22:07.690598+0000","last_became_active":"2024-03-26T06:20:53.289592+0000","last_became_peered":"2024-03-26T06:20:53.289592+0000","last_unstale":"2024-03-26T06:22:07.690598+0000","last_undegraded":"2024-03-26T06:22:07.690598+0000","last_fullsized":"2024-03-26T06:22:07.690598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:11:41.249408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718811+0000","last_change":"2024-03-26T06:20:54.290970+0000","last_active":"2024-03-26T06:21:48.718811+0000","last_peered":"2024-03-26T06:21:48.718811+0000","last_clean":"2024-03-26T06:21:48.718811+0000","last_became_active":"2024-03-26T06:20:54.289686+0000","last_became_peered":"2024-03-26T06:20:54.289686+0000","last_unstale":"2024-03-26T06:21:48.718811+0000","last_undegraded":"2024-03-26T06:21:48.718811+0000","last_fullsized":"2024-03-26T06:21:48.718811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:39:42.552786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.745217+0000","last_change":"2024-03-26T06:21:48.745217+0000","last_active":"2024-03-26T06:21:48.745217+0000","last_peered":"2024-03-26T06:21:48.745217+0000","last_clean":"2024-03-26T06:21:48.745217+0000","last_became_active":"2024-03-26T06:21:48.741202+0000","last_became_peered":"2024-03-26T06:21:48.741202+0000","last_unstale":"2024-03-26T06:21:48.745217+0000","last_undegraded":"2024-03-26T06:21:48.745217+0000","last_fullsized":"2024-03-26T06:21:48.745217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:06:04.855755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716119+0000","last_change":"2024-03-26T06:20:54.299129+0000","last_active":"2024-03-26T06:21:48.716119+0000","last_peered":"2024-03-26T06:21:48.716119+0000","last_clean":"2024-03-26T06:21:48.716119+0000","last_became_active":"2024-03-26T06:20:54.298777+0000","last_became_peered":"2024-03-26T06:20:54.298777+0000","last_unstale":"2024-03-26T06:21:48.716119+0000","last_undegraded":"2024-03-26T06:21:48.716119+0000","last_fullsized":"2024-03-26T06:21:48.716119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:29:38.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724463+0000","last_change":"2024-03-26T06:20:53.284988+0000","last_active":"2024-03-26T06:21:48.724463+0000","last_peered":"2024-03-26T06:21:48.724463+0000","last_clean":"2024-03-26T06:21:48.724463+0000","last_became_active":"2024-03-26T06:20:53.284684+0000","last_became_peered":"2024-03-26T06:20:53.284684+0000","last_unstale":"2024-03-26T06:21:48.724463+0000","last_undegraded":"2024-03-26T06:21:48.724463+0000","last_fullsized":"2024-03-26T06:21:48.724463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:31:45.004042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718889+0000","last_change":"2024-03-26T06:20:54.307147+0000","last_active":"2024-03-26T06:21:48.718889+0000","last_peered":"2024-03-26T06:21:48.718889+0000","last_clean":"2024-03-26T06:21:48.718889+0000","last_became_active":"2024-03-26T06:20:54.306797+0000","last_became_peered":"2024-03-26T06:20:54.306797+0000","last_unstale":"2024-03-26T06:21:48.718889+0000","last_undegraded":"2024-03-26T06:21:48.718889+0000","last_fullsized":"2024-03-26T06:21:48.718889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:20:58.904884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'27","reported_seq":189,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:07.705835+0000","last_change":"2024-03-26T06:20:53.284577+0000","last_active":"2024-03-26T06:22:07.705835+0000","last_peered":"2024-03-26T06:22:07.705835+0000","last_clean":"2024-03-26T06:22:07.705835+0000","last_became_active":"2024-03-26T06:20:53.284161+0000","last_became_peered":"2024-03-26T06:20:53.284161+0000","last_unstale":"2024-03-26T06:22:07.705835+0000","last_undegraded":"2024-03-26T06:22:07.705835+0000","last_fullsized":"2024-03-26T06:22:07.705835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:57.765950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":143,"num_read_kb":129,"num_write":29,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719781+0000","last_change":"2024-03-26T06:20:54.301794+0000","last_active":"2024-03-26T06:21:48.719781+0000","last_peered":"2024-03-26T06:21:48.719781+0000","last_clean":"2024-03-26T06:21:48.719781+0000","last_became_active":"2024-03-26T06:20:54.301445+0000","last_became_peered":"2024-03-26T06:20:54.301445+0000","last_unstale":"2024-03-26T06:21:48.719781+0000","last_undegraded":"2024-03-26T06:21:48.719781+0000","last_fullsized":"2024-03-26T06:21:48.719781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:32:18.280992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719138+0000","last_change":"2024-03-26T06:20:53.282394+0000","last_active":"2024-03-26T06:21:48.719138+0000","last_peered":"2024-03-26T06:21:48.719138+0000","last_clean":"2024-03-26T06:21:48.719138+0000","last_became_active":"2024-03-26T06:20:53.282068+0000","last_became_peered":"2024-03-26T06:20:53.282068+0000","last_unstale":"2024-03-26T06:21:48.719138+0000","last_undegraded":"2024-03-26T06:21:48.719138+0000","last_fullsized":"2024-03-26T06:21:48.719138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:04:00.503682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050085+0000","last_change":"2024-03-26T06:20:54.303294+0000","last_active":"2024-03-26T06:21:49.050085+0000","last_peered":"2024-03-26T06:21:49.050085+0000","last_clean":"2024-03-26T06:21:49.050085+0000","last_became_active":"2024-03-26T06:20:54.301342+0000","last_became_peered":"2024-03-26T06:20:54.301342+0000","last_unstale":"2024-03-26T06:21:49.050085+0000","last_undegraded":"2024-03-26T06:21:49.050085+0000","last_fullsized":"2024-03-26T06:21:49.050085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:53:51.012761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724063+0000","last_change":"2024-03-26T06:20:53.286119+0000","last_active":"2024-03-26T06:21:48.724063+0000","last_peered":"2024-03-26T06:21:48.724063+0000","last_clean":"2024-03-26T06:21:48.724063+0000","last_became_active":"2024-03-26T06:20:53.285784+0000","last_became_peered":"2024-03-26T06:20:53.285784+0000","last_unstale":"2024-03-26T06:21:48.724063+0000","last_undegraded":"2024-03-26T06:21:48.724063+0000","last_fullsized":"2024-03-26T06:21:48.724063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:54:36.786966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716316+0000","last_change":"2024-03-26T06:20:54.309706+0000","last_active":"2024-03-26T06:21:48.716316+0000","last_peered":"2024-03-26T06:21:48.716316+0000","last_clean":"2024-03-26T06:21:48.716316+0000","last_became_active":"2024-03-26T06:20:54.309291+0000","last_became_peered":"2024-03-26T06:20:54.309291+0000","last_unstale":"2024-03-26T06:21:48.716316+0000","last_undegraded":"2024-03-26T06:21:48.716316+0000","last_fullsized":"2024-03-26T06:21:48.716316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:51:52.150636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:04.675812+0000","last_change":"2024-03-26T06:20:53.278569+0000","last_active":"2024-03-26T06:22:04.675812+0000","last_peered":"2024-03-26T06:22:04.675812+0000","last_clean":"2024-03-26T06:22:04.675812+0000","last_became_active":"2024-03-26T06:20:53.278207+0000","last_became_peered":"2024-03-26T06:20:53.278207+0000","last_unstale":"2024-03-26T06:22:04.675812+0000","last_undegraded":"2024-03-26T06:22:04.675812+0000","last_fullsized":"2024-03-26T06:22:04.675812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:59:12.345429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716018+0000","last_change":"2024-03-26T06:20:54.308325+0000","last_active":"2024-03-26T06:21:48.716018+0000","last_peered":"2024-03-26T06:21:48.716018+0000","last_clean":"2024-03-26T06:21:48.716018+0000","last_became_active":"2024-03-26T06:20:54.307970+0000","last_became_peered":"2024-03-26T06:20:54.307970+0000","last_unstale":"2024-03-26T06:21:48.716018+0000","last_undegraded":"2024-03-26T06:21:48.716018+0000","last_fullsized":"2024-03-26T06:21:48.716018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:18:26.597073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718555+0000","last_change":"2024-03-26T06:20:53.284675+0000","last_active":"2024-03-26T06:21:48.718555+0000","last_peered":"2024-03-26T06:21:48.718555+0000","last_clean":"2024-03-26T06:21:48.718555+0000","last_became_active":"2024-03-26T06:20:53.284368+0000","last_became_peered":"2024-03-26T06:20:53.284368+0000","last_unstale":"2024-03-26T06:21:48.718555+0000","last_undegraded":"2024-03-26T06:21:48.718555+0000","last_fullsized":"2024-03-26T06:21:48.718555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:15:51.733268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724138+0000","last_change":"2024-03-26T06:20:54.296348+0000","last_active":"2024-03-26T06:21:48.724138+0000","last_peered":"2024-03-26T06:21:48.724138+0000","last_clean":"2024-03-26T06:21:48.724138+0000","last_became_active":"2024-03-26T06:20:54.296019+0000","last_became_peered":"2024-03-26T06:20:54.296019+0000","last_unstale":"2024-03-26T06:21:48.724138+0000","last_undegraded":"2024-03-26T06:21:48.724138+0000","last_fullsized":"2024-03-26T06:21:48.724138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:25:28.498887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719941+0000","last_change":"2024-03-26T06:20:53.281314+0000","last_active":"2024-03-26T06:21:48.719941+0000","last_peered":"2024-03-26T06:21:48.719941+0000","last_clean":"2024-03-26T06:21:48.719941+0000","last_became_active":"2024-03-26T06:20:53.281007+0000","last_became_peered":"2024-03-26T06:20:53.281007+0000","last_unstale":"2024-03-26T06:21:48.719941+0000","last_undegraded":"2024-03-26T06:21:48.719941+0000","last_fullsized":"2024-03-26T06:21:48.719941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:39:25.840470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.721263+0000","last_change":"2024-03-26T06:20:54.290011+0000","last_active":"2024-03-26T06:21:48.721263+0000","last_peered":"2024-03-26T06:21:48.721263+0000","last_clean":"2024-03-26T06:21:48.721263+0000","last_became_active":"2024-03-26T06:20:54.289492+0000","last_became_peered":"2024-03-26T06:20:54.289492+0000","last_unstale":"2024-03-26T06:21:48.721263+0000","last_undegraded":"2024-03-26T06:21:48.721263+0000","last_fullsized":"2024-03-26T06:21:48.721263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:39:16.960414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718559+0000","last_change":"2024-03-26T06:20:53.276374+0000","last_active":"2024-03-26T06:21:48.718559+0000","last_peered":"2024-03-26T06:21:48.718559+0000","last_clean":"2024-03-26T06:21:48.718559+0000","last_became_active":"2024-03-26T06:20:53.276031+0000","last_became_peered":"2024-03-26T06:20:53.276031+0000","last_unstale":"2024-03-26T06:21:48.718559+0000","last_undegraded":"2024-03-26T06:21:48.718559+0000","last_fullsized":"2024-03-26T06:21:48.718559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:14.601841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.353310+0000","last_change":"2024-03-26T06:20:54.304279+0000","last_active":"2024-03-26T06:21:49.353310+0000","last_peered":"2024-03-26T06:21:49.353310+0000","last_clean":"2024-03-26T06:21:49.353310+0000","last_became_active":"2024-03-26T06:20:54.303859+0000","last_became_peered":"2024-03-26T06:20:54.303859+0000","last_unstale":"2024-03-26T06:21:49.353310+0000","last_undegraded":"2024-03-26T06:21:49.353310+0000","last_fullsized":"2024-03-26T06:21:49.353310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:32:37.969077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:05.677321+0000","last_change":"2024-03-26T06:20:53.286038+0000","last_active":"2024-03-26T06:22:05.677321+0000","last_peered":"2024-03-26T06:22:05.677321+0000","last_clean":"2024-03-26T06:22:05.677321+0000","last_became_active":"2024-03-26T06:20:53.285353+0000","last_became_peered":"2024-03-26T06:20:53.285353+0000","last_unstale":"2024-03-26T06:22:05.677321+0000","last_undegraded":"2024-03-26T06:22:05.677321+0000","last_fullsized":"2024-03-26T06:22:05.677321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:43:53.970393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.353081+0000","last_change":"2024-03-26T06:20:54.294425+0000","last_active":"2024-03-26T06:21:49.353081+0000","last_peered":"2024-03-26T06:21:49.353081+0000","last_clean":"2024-03-26T06:21:49.353081+0000","last_became_active":"2024-03-26T06:20:54.293607+0000","last_became_peered":"2024-03-26T06:20:54.293607+0000","last_unstale":"2024-03-26T06:21:49.353081+0000","last_undegraded":"2024-03-26T06:21:49.353081+0000","last_fullsized":"2024-03-26T06:21:49.353081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:12:49.118427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724238+0000","last_change":"2024-03-26T06:20:54.291038+0000","last_active":"2024-03-26T06:21:48.724238+0000","last_peered":"2024-03-26T06:21:48.724238+0000","last_clean":"2024-03-26T06:21:48.724238+0000","last_became_active":"2024-03-26T06:20:54.290678+0000","last_became_peered":"2024-03-26T06:20:54.290678+0000","last_unstale":"2024-03-26T06:21:48.724238+0000","last_undegraded":"2024-03-26T06:21:48.724238+0000","last_fullsized":"2024-03-26T06:21:48.724238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:29:30.528408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:05.676638+0000","last_change":"2024-03-26T06:20:53.288795+0000","last_active":"2024-03-26T06:22:05.676638+0000","last_peered":"2024-03-26T06:22:05.676638+0000","last_clean":"2024-03-26T06:22:05.676638+0000","last_became_active":"2024-03-26T06:20:53.288468+0000","last_became_peered":"2024-03-26T06:20:53.288468+0000","last_unstale":"2024-03-26T06:22:05.676638+0000","last_undegraded":"2024-03-26T06:22:05.676638+0000","last_fullsized":"2024-03-26T06:22:05.676638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:16:05.569407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719380+0000","last_change":"2024-03-26T06:20:54.296706+0000","last_active":"2024-03-26T06:21:48.719380+0000","last_peered":"2024-03-26T06:21:48.719380+0000","last_clean":"2024-03-26T06:21:48.719380+0000","last_became_active":"2024-03-26T06:20:54.296387+0000","last_became_peered":"2024-03-26T06:20:54.296387+0000","last_unstale":"2024-03-26T06:21:48.719380+0000","last_undegraded":"2024-03-26T06:21:48.719380+0000","last_fullsized":"2024-03-26T06:21:48.719380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:23:23.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.739654+0000","last_change":"2024-03-26T06:20:53.282790+0000","last_active":"2024-03-26T06:21:48.739654+0000","last_peered":"2024-03-26T06:21:48.739654+0000","last_clean":"2024-03-26T06:21:48.739654+0000","last_became_active":"2024-03-26T06:20:53.282451+0000","last_became_peered":"2024-03-26T06:20:53.282451+0000","last_unstale":"2024-03-26T06:21:48.739654+0000","last_undegraded":"2024-03-26T06:21:48.739654+0000","last_fullsized":"2024-03-26T06:21:48.739654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:47:56.328354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716073+0000","last_change":"2024-03-26T06:19:42.185355+0000","last_active":"2024-03-26T06:21:48.716073+0000","last_peered":"2024-03-26T06:21:48.716073+0000","last_clean":"2024-03-26T06:21:48.716073+0000","last_became_active":"2024-03-26T06:19:42.184672+0000","last_became_peered":"2024-03-26T06:19:42.184672+0000","last_unstale":"2024-03-26T06:21:48.716073+0000","last_undegraded":"2024-03-26T06:21:48.716073+0000","last_fullsized":"2024-03-26T06:21:48.716073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:16:17.700523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:16:17.700523+0000","last_clean_scrub_stamp":"2024-03-26T06:16:17.700523+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:14:25.637431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.049942+0000","last_change":"2024-03-26T06:20:54.295599+0000","last_active":"2024-03-26T06:21:49.049942+0000","last_peered":"2024-03-26T06:21:49.049942+0000","last_clean":"2024-03-26T06:21:49.049942+0000","last_became_active":"2024-03-26T06:20:54.295270+0000","last_became_peered":"2024-03-26T06:20:54.295270+0000","last_unstale":"2024-03-26T06:21:49.049942+0000","last_undegraded":"2024-03-26T06:21:49.049942+0000","last_fullsized":"2024-03-26T06:21:49.049942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:47.406455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716223+0000","last_change":"2024-03-26T06:20:53.284859+0000","last_active":"2024-03-26T06:21:48.716223+0000","last_peered":"2024-03-26T06:21:48.716223+0000","last_clean":"2024-03-26T06:21:48.716223+0000","last_became_active":"2024-03-26T06:20:53.283671+0000","last_became_peered":"2024-03-26T06:20:53.283671+0000","last_unstale":"2024-03-26T06:21:48.716223+0000","last_undegraded":"2024-03-26T06:21:48.716223+0000","last_fullsized":"2024-03-26T06:21:48.716223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:06:23.676648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.720068+0000","last_change":"2024-03-26T06:20:53.288792+0000","last_active":"2024-03-26T06:21:48.720068+0000","last_peered":"2024-03-26T06:21:48.720068+0000","last_clean":"2024-03-26T06:21:48.720068+0000","last_became_active":"2024-03-26T06:20:53.288463+0000","last_became_peered":"2024-03-26T06:20:53.288463+0000","last_unstale":"2024-03-26T06:21:48.720068+0000","last_undegraded":"2024-03-26T06:21:48.720068+0000","last_fullsized":"2024-03-26T06:21:48.720068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:55:55.014266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050305+0000","last_change":"2024-03-26T06:20:54.288478+0000","last_active":"2024-03-26T06:21:49.050305+0000","last_peered":"2024-03-26T06:21:49.050305+0000","last_clean":"2024-03-26T06:21:49.050305+0000","last_became_active":"2024-03-26T06:20:54.288014+0000","last_became_peered":"2024-03-26T06:20:54.288014+0000","last_unstale":"2024-03-26T06:21:49.050305+0000","last_undegraded":"2024-03-26T06:21:49.050305+0000","last_fullsized":"2024-03-26T06:21:49.050305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:36:16.252703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719283+0000","last_change":"2024-03-26T06:20:53.280055+0000","last_active":"2024-03-26T06:21:48.719283+0000","last_peered":"2024-03-26T06:21:48.719283+0000","last_clean":"2024-03-26T06:21:48.719283+0000","last_became_active":"2024-03-26T06:20:53.279751+0000","last_became_peered":"2024-03-26T06:20:53.279751+0000","last_unstale":"2024-03-26T06:21:48.719283+0000","last_undegraded":"2024-03-26T06:21:48.719283+0000","last_fullsized":"2024-03-26T06:21:48.719283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:32:48.407054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719942+0000","last_change":"2024-03-26T06:20:54.302001+0000","last_active":"2024-03-26T06:21:48.719942+0000","last_peered":"2024-03-26T06:21:48.719942+0000","last_clean":"2024-03-26T06:21:48.719942+0000","last_became_active":"2024-03-26T06:20:54.301659+0000","last_became_peered":"2024-03-26T06:20:54.301659+0000","last_unstale":"2024-03-26T06:21:48.719942+0000","last_undegraded":"2024-03-26T06:21:48.719942+0000","last_fullsized":"2024-03-26T06:21:48.719942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:38:08.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.740176+0000","last_change":"2024-03-26T06:20:53.285561+0000","last_active":"2024-03-26T06:21:48.740176+0000","last_peered":"2024-03-26T06:21:48.740176+0000","last_clean":"2024-03-26T06:21:48.740176+0000","last_became_active":"2024-03-26T06:20:53.285189+0000","last_became_peered":"2024-03-26T06:20:53.285189+0000","last_unstale":"2024-03-26T06:21:48.740176+0000","last_undegraded":"2024-03-26T06:21:48.740176+0000","last_fullsized":"2024-03-26T06:21:48.740176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:53:54.266188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715957+0000","last_change":"2024-03-26T06:20:54.300421+0000","last_active":"2024-03-26T06:21:48.715957+0000","last_peered":"2024-03-26T06:21:48.715957+0000","last_clean":"2024-03-26T06:21:48.715957+0000","last_became_active":"2024-03-26T06:20:54.299292+0000","last_became_peered":"2024-03-26T06:20:54.299292+0000","last_unstale":"2024-03-26T06:21:48.715957+0000","last_undegraded":"2024-03-26T06:21:48.715957+0000","last_fullsized":"2024-03-26T06:21:48.715957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:29:12.685903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.730662+0000","last_change":"2024-03-26T06:21:48.730662+0000","last_active":"2024-03-26T06:21:48.730662+0000","last_peered":"2024-03-26T06:21:48.730662+0000","last_clean":"2024-03-26T06:21:48.730662+0000","last_became_active":"2024-03-26T06:21:48.727946+0000","last_became_peered":"2024-03-26T06:21:48.727946+0000","last_unstale":"2024-03-26T06:21:48.730662+0000","last_undegraded":"2024-03-26T06:21:48.730662+0000","last_fullsized":"2024-03-26T06:21:48.730662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:01:50.526857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718480+0000","last_change":"2024-03-26T06:20:54.295448+0000","last_active":"2024-03-26T06:21:48.718480+0000","last_peered":"2024-03-26T06:21:48.718480+0000","last_clean":"2024-03-26T06:21:48.718480+0000","last_became_active":"2024-03-26T06:20:54.295116+0000","last_became_peered":"2024-03-26T06:20:54.295116+0000","last_unstale":"2024-03-26T06:21:48.718480+0000","last_undegraded":"2024-03-26T06:21:48.718480+0000","last_fullsized":"2024-03-26T06:21:48.718480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:22:07.474387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718967+0000","last_change":"2024-03-26T06:20:53.281317+0000","last_active":"2024-03-26T06:21:48.718967+0000","last_peered":"2024-03-26T06:21:48.718967+0000","last_clean":"2024-03-26T06:21:48.718967+0000","last_became_active":"2024-03-26T06:20:53.280979+0000","last_became_peered":"2024-03-26T06:20:53.280979+0000","last_unstale":"2024-03-26T06:21:48.718967+0000","last_undegraded":"2024-03-26T06:21:48.718967+0000","last_fullsized":"2024-03-26T06:21:48.718967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:02:29.850564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.722350+0000","last_change":"2024-03-26T06:20:54.288963+0000","last_active":"2024-03-26T06:21:48.722350+0000","last_peered":"2024-03-26T06:21:48.722350+0000","last_clean":"2024-03-26T06:21:48.722350+0000","last_became_active":"2024-03-26T06:20:54.288433+0000","last_became_peered":"2024-03-26T06:20:54.288433+0000","last_unstale":"2024-03-26T06:21:48.722350+0000","last_undegraded":"2024-03-26T06:21:48.722350+0000","last_fullsized":"2024-03-26T06:21:48.722350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:33.713471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718826+0000","last_change":"2024-03-26T06:20:53.279874+0000","last_active":"2024-03-26T06:21:48.718826+0000","last_peered":"2024-03-26T06:21:48.718826+0000","last_clean":"2024-03-26T06:21:48.718826+0000","last_became_active":"2024-03-26T06:20:53.279548+0000","last_became_peered":"2024-03-26T06:20:53.279548+0000","last_unstale":"2024-03-26T06:21:48.718826+0000","last_undegraded":"2024-03-26T06:21:48.718826+0000","last_fullsized":"2024-03-26T06:21:48.718826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:14.807650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718829+0000","last_change":"2024-03-26T06:20:54.305353+0000","last_active":"2024-03-26T06:21:48.718829+0000","last_peered":"2024-03-26T06:21:48.718829+0000","last_clean":"2024-03-26T06:21:48.718829+0000","last_became_active":"2024-03-26T06:20:54.304871+0000","last_became_peered":"2024-03-26T06:20:54.304871+0000","last_unstale":"2024-03-26T06:21:48.718829+0000","last_undegraded":"2024-03-26T06:21:48.718829+0000","last_fullsized":"2024-03-26T06:21:48.718829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:24:48.679669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719057+0000","last_change":"2024-03-26T06:20:53.286291+0000","last_active":"2024-03-26T06:21:48.719057+0000","last_peered":"2024-03-26T06:21:48.719057+0000","last_clean":"2024-03-26T06:21:48.719057+0000","last_became_active":"2024-03-26T06:20:53.285913+0000","last_became_peered":"2024-03-26T06:20:53.285913+0000","last_unstale":"2024-03-26T06:21:48.719057+0000","last_undegraded":"2024-03-26T06:21:48.719057+0000","last_fullsized":"2024-03-26T06:21:48.719057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:33.013108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716766+0000","last_change":"2024-03-26T06:20:54.302788+0000","last_active":"2024-03-26T06:21:48.716766+0000","last_peered":"2024-03-26T06:21:48.716766+0000","last_clean":"2024-03-26T06:21:48.716766+0000","last_became_active":"2024-03-26T06:20:54.302299+0000","last_became_peered":"2024-03-26T06:20:54.302299+0000","last_unstale":"2024-03-26T06:21:48.716766+0000","last_undegraded":"2024-03-26T06:21:48.716766+0000","last_fullsized":"2024-03-26T06:21:48.716766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:17:54.683294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:04.686293+0000","last_change":"2024-03-26T06:20:53.280496+0000","last_active":"2024-03-26T06:22:04.686293+0000","last_peered":"2024-03-26T06:22:04.686293+0000","last_clean":"2024-03-26T06:22:04.686293+0000","last_became_active":"2024-03-26T06:20:53.280178+0000","last_became_peered":"2024-03-26T06:20:53.280178+0000","last_unstale":"2024-03-26T06:22:04.686293+0000","last_undegraded":"2024-03-26T06:22:04.686293+0000","last_fullsized":"2024-03-26T06:22:04.686293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.095552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716495+0000","last_change":"2024-03-26T06:20:54.299702+0000","last_active":"2024-03-26T06:21:48.716495+0000","last_peered":"2024-03-26T06:21:48.716495+0000","last_clean":"2024-03-26T06:21:48.716495+0000","last_became_active":"2024-03-26T06:20:54.299069+0000","last_became_peered":"2024-03-26T06:20:54.299069+0000","last_unstale":"2024-03-26T06:21:48.716495+0000","last_undegraded":"2024-03-26T06:21:48.716495+0000","last_fullsized":"2024-03-26T06:21:48.716495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:20:56.019540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724256+0000","last_change":"2024-03-26T06:20:53.284304+0000","last_active":"2024-03-26T06:21:48.724256+0000","last_peered":"2024-03-26T06:21:48.724256+0000","last_clean":"2024-03-26T06:21:48.724256+0000","last_became_active":"2024-03-26T06:20:53.283988+0000","last_became_peered":"2024-03-26T06:20:53.283988+0000","last_unstale":"2024-03-26T06:21:48.724256+0000","last_undegraded":"2024-03-26T06:21:48.724256+0000","last_fullsized":"2024-03-26T06:21:48.724256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:07:21.495880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716291+0000","last_change":"2024-03-26T06:20:54.305751+0000","last_active":"2024-03-26T06:21:48.716291+0000","last_peered":"2024-03-26T06:21:48.716291+0000","last_clean":"2024-03-26T06:21:48.716291+0000","last_became_active":"2024-03-26T06:20:54.303904+0000","last_became_peered":"2024-03-26T06:20:54.303904+0000","last_unstale":"2024-03-26T06:21:48.716291+0000","last_undegraded":"2024-03-26T06:21:48.716291+0000","last_fullsized":"2024-03-26T06:21:48.716291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:28:43.597279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050284+0000","last_change":"2024-03-26T06:20:53.283452+0000","last_active":"2024-03-26T06:21:49.050284+0000","last_peered":"2024-03-26T06:21:49.050284+0000","last_clean":"2024-03-26T06:21:49.050284+0000","last_became_active":"2024-03-26T06:20:53.282950+0000","last_became_peered":"2024-03-26T06:20:53.282950+0000","last_unstale":"2024-03-26T06:21:49.050284+0000","last_undegraded":"2024-03-26T06:21:49.050284+0000","last_fullsized":"2024-03-26T06:21:49.050284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:59:15.161087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716550+0000","last_change":"2024-03-26T06:20:54.300176+0000","last_active":"2024-03-26T06:21:48.716550+0000","last_peered":"2024-03-26T06:21:48.716550+0000","last_clean":"2024-03-26T06:21:48.716550+0000","last_became_active":"2024-03-26T06:20:54.299785+0000","last_became_peered":"2024-03-26T06:20:54.299785+0000","last_unstale":"2024-03-26T06:21:48.716550+0000","last_undegraded":"2024-03-26T06:21:48.716550+0000","last_fullsized":"2024-03-26T06:21:48.716550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:10.091612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.722461+0000","last_change":"2024-03-26T06:20:54.288774+0000","last_active":"2024-03-26T06:21:48.722461+0000","last_peered":"2024-03-26T06:21:48.722461+0000","last_clean":"2024-03-26T06:21:48.722461+0000","last_became_active":"2024-03-26T06:20:54.287986+0000","last_became_peered":"2024-03-26T06:20:54.287986+0000","last_unstale":"2024-03-26T06:21:48.722461+0000","last_undegraded":"2024-03-26T06:21:48.722461+0000","last_fullsized":"2024-03-26T06:21:48.722461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:49:04.418838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.051916+0000","last_change":"2024-03-26T06:21:49.051916+0000","last_active":"2024-03-26T06:21:49.051916+0000","last_peered":"2024-03-26T06:21:49.051916+0000","last_clean":"2024-03-26T06:21:49.051916+0000","last_became_active":"2024-03-26T06:21:49.051408+0000","last_became_peered":"2024-03-26T06:21:49.051408+0000","last_unstale":"2024-03-26T06:21:49.051916+0000","last_undegraded":"2024-03-26T06:21:49.051916+0000","last_fullsized":"2024-03-26T06:21:49.051916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:19:15.373133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718680+0000","last_change":"2024-03-26T06:20:54.287935+0000","last_active":"2024-03-26T06:21:48.718680+0000","last_peered":"2024-03-26T06:21:48.718680+0000","last_clean":"2024-03-26T06:21:48.718680+0000","last_became_active":"2024-03-26T06:20:54.287575+0000","last_became_peered":"2024-03-26T06:20:54.287575+0000","last_unstale":"2024-03-26T06:21:48.718680+0000","last_undegraded":"2024-03-26T06:21:48.718680+0000","last_fullsized":"2024-03-26T06:21:48.718680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:41:00.110749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715793+0000","last_change":"2024-03-26T06:20:54.299188+0000","last_active":"2024-03-26T06:21:48.715793+0000","last_peered":"2024-03-26T06:21:48.715793+0000","last_clean":"2024-03-26T06:21:48.715793+0000","last_became_active":"2024-03-26T06:20:54.298822+0000","last_became_peered":"2024-03-26T06:20:54.298822+0000","last_unstale":"2024-03-26T06:21:48.715793+0000","last_undegraded":"2024-03-26T06:21:48.715793+0000","last_fullsized":"2024-03-26T06:21:48.715793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:03:17.694808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050274+0000","last_change":"2024-03-26T06:20:54.295974+0000","last_active":"2024-03-26T06:21:49.050274+0000","last_peered":"2024-03-26T06:21:49.050274+0000","last_clean":"2024-03-26T06:21:49.050274+0000","last_became_active":"2024-03-26T06:20:54.295111+0000","last_became_peered":"2024-03-26T06:20:54.295111+0000","last_unstale":"2024-03-26T06:21:49.050274+0000","last_undegraded":"2024-03-26T06:21:49.050274+0000","last_fullsized":"2024-03-26T06:21:49.050274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:53:45.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.721067+0000","last_change":"2024-03-26T06:20:54.282386+0000","last_active":"2024-03-26T06:21:48.721067+0000","last_peered":"2024-03-26T06:21:48.721067+0000","last_clean":"2024-03-26T06:21:48.721067+0000","last_became_active":"2024-03-26T06:20:54.282017+0000","last_became_peered":"2024-03-26T06:20:54.282017+0000","last_unstale":"2024-03-26T06:21:48.721067+0000","last_undegraded":"2024-03-26T06:21:48.721067+0000","last_fullsized":"2024-03-26T06:21:48.721067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:09:14.663629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719476+0000","last_change":"2024-03-26T06:20:54.297876+0000","last_active":"2024-03-26T06:21:48.719476+0000","last_peered":"2024-03-26T06:21:48.719476+0000","last_clean":"2024-03-26T06:21:48.719476+0000","last_became_active":"2024-03-26T06:20:54.297545+0000","last_became_peered":"2024-03-26T06:20:54.297545+0000","last_unstale":"2024-03-26T06:21:48.719476+0000","last_undegraded":"2024-03-26T06:21:48.719476+0000","last_fullsized":"2024-03-26T06:21:48.719476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:15:51.786210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.722722+0000","last_change":"2024-03-26T06:21:48.722722+0000","last_active":"2024-03-26T06:21:48.722722+0000","last_peered":"2024-03-26T06:21:48.722722+0000","last_clean":"2024-03-26T06:21:48.722722+0000","last_became_active":"2024-03-26T06:21:48.722062+0000","last_became_peered":"2024-03-26T06:21:48.722062+0000","last_unstale":"2024-03-26T06:21:48.722722+0000","last_undegraded":"2024-03-26T06:21:48.722722+0000","last_fullsized":"2024-03-26T06:21:48.722722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:01:44.959387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.723982+0000","last_change":"2024-03-26T06:20:54.289849+0000","last_active":"2024-03-26T06:21:48.723982+0000","last_peered":"2024-03-26T06:21:48.723982+0000","last_clean":"2024-03-26T06:21:48.723982+0000","last_became_active":"2024-03-26T06:20:54.287372+0000","last_became_peered":"2024-03-26T06:20:54.287372+0000","last_unstale":"2024-03-26T06:21:48.723982+0000","last_undegraded":"2024-03-26T06:21:48.723982+0000","last_fullsized":"2024-03-26T06:21:48.723982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:48.537975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718835+0000","last_change":"2024-03-26T06:20:54.285737+0000","last_active":"2024-03-26T06:21:48.718835+0000","last_peered":"2024-03-26T06:21:48.718835+0000","last_clean":"2024-03-26T06:21:48.718835+0000","last_became_active":"2024-03-26T06:20:54.285423+0000","last_became_peered":"2024-03-26T06:20:54.285423+0000","last_unstale":"2024-03-26T06:21:48.718835+0000","last_undegraded":"2024-03-26T06:21:48.718835+0000","last_fullsized":"2024-03-26T06:21:48.718835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:55:59.095785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719004+0000","last_change":"2024-03-26T06:20:54.301849+0000","last_active":"2024-03-26T06:21:48.719004+0000","last_peered":"2024-03-26T06:21:48.719004+0000","last_clean":"2024-03-26T06:21:48.719004+0000","last_became_active":"2024-03-26T06:20:54.301444+0000","last_became_peered":"2024-03-26T06:20:54.301444+0000","last_unstale":"2024-03-26T06:21:48.719004+0000","last_undegraded":"2024-03-26T06:21:48.719004+0000","last_fullsized":"2024-03-26T06:21:48.719004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:57:32.365781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050384+0000","last_change":"2024-03-26T06:20:54.302979+0000","last_active":"2024-03-26T06:21:49.050384+0000","last_peered":"2024-03-26T06:21:49.050384+0000","last_clean":"2024-03-26T06:21:49.050384+0000","last_became_active":"2024-03-26T06:20:54.301428+0000","last_became_peered":"2024-03-26T06:20:54.301428+0000","last_unstale":"2024-03-26T06:21:49.050384+0000","last_undegraded":"2024-03-26T06:21:49.050384+0000","last_fullsized":"2024-03-26T06:21:49.050384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:49:29.519202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.725345+0000","last_change":"2024-03-26T06:21:48.725345+0000","last_active":"2024-03-26T06:21:48.725345+0000","last_peered":"2024-03-26T06:21:48.725345+0000","last_clean":"2024-03-26T06:21:48.725345+0000","last_became_active":"2024-03-26T06:21:48.724945+0000","last_became_peered":"2024-03-26T06:21:48.724945+0000","last_unstale":"2024-03-26T06:21:48.725345+0000","last_undegraded":"2024-03-26T06:21:48.725345+0000","last_fullsized":"2024-03-26T06:21:48.725345+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:53:19.086435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65725,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":715,"num_read_kb":432,"num_write":233,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":197175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383707,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":457397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1650688,"data_stored":1107147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":456745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":456541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712440,"statfs":{"total":95995035648,"available":95961538560,"internally_reserved":0,"allocated":1568768,"data_stored":1049641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":516287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":978944,"data_stored":457801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":516507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":456287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1048928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T06:22:09.922 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T06:22:10.171 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:22:10.551 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:10 smithi037 ceph-mon[31974]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:22:10.551 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:10 smithi037 ceph-mon[31974]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:22:10.551 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:10 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:22:10.551 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:10 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:22:10.551 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:10 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:22:10.551 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:10 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:22:10.552 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:10 smithi037 ceph-mon[31974]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:22:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:10 smithi066 ceph-mon[36504]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:22:10.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:10 smithi066 ceph-mon[36504]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:22:10.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:10 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:22:10.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:10 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:22:10.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:10 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:22:10.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:10 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:22:10.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:10 smithi066 ceph-mon[36504]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:22:10.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:10 smithi149 ceph-mon[36681]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:22:10.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:10 smithi149 ceph-mon[36681]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:22:10.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:10 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:22:10.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:10 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:22:10.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:10 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:22:10.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:10 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:22:10.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:10 smithi149 ceph-mon[36681]: from='mgr.14152 172.21.15.37:0/25986207' entity='mgr.x' 2024-03-26T06:22:11.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.853+0000 7f98fee4a700 1 -- 172.21.15.37:0/2035963086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98f81027f0 msgr2=0x7f98f810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:11.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.853+0000 7f98fee4a700 1 --2- 172.21.15.37:0/2035963086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98f81027f0 0x7f98f810ec30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f98f000b0d0 tx=0x7f98f000b3e0 comp rx=0 tx=0).stop 2024-03-26T06:22:11.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.853+0000 7f98fee4a700 1 -- 172.21.15.37:0/2035963086 shutdown_connections 2024-03-26T06:22:11.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.853+0000 7f98fee4a700 1 --2- 172.21.15.37:0/2035963086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98f81027f0 0x7f98f810ec30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:11.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.853+0000 7f98fee4a700 1 --2- 172.21.15.37:0/2035963086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98f8107e10 0x7f98f81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.853+0000 7f98fee4a700 1 --2- 172.21.15.37:0/2035963086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98f8101e30 0x7f98f81022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.853+0000 7f98fee4a700 1 -- 172.21.15.37:0/2035963086 >> 172.21.15.37:0/2035963086 conn(0x7f98f8078f10 msgr2=0x7f98f8079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.853+0000 7f98fee4a700 1 -- 172.21.15.37:0/2035963086 shutdown_connections 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.853+0000 7f98fee4a700 1 -- 172.21.15.37:0/2035963086 wait complete. 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98fee4a700 1 Processor -- start 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98fee4a700 1 -- start start 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98fee4a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98f8101e30 0x7f98f80740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98fee4a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98f81027f0 0x7f98f8074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98fee4a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98f8107e10 0x7f98f8074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98fee4a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f8112110 con 0x7f98f81027f0 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98fee4a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f8111f90 con 0x7f98f8107e10 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98fee4a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f8112290 con 0x7f98f8101e30 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98f7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98f81027f0 0x7f98f8074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:11.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98f7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98f81027f0 0x7f98f8074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52794/0 (socket says 172.21.15.37:52794) 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98f7fff700 1 -- 172.21.15.37:0/2138356059 learned_addr learned my addr 172.21.15.37:0/2138356059 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98fcbe6700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98f8101e30 0x7f98f80740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98fd3e7700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98f8107e10 0x7f98f8074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98f7fff700 1 -- 172.21.15.37:0/2138356059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98f8101e30 msgr2=0x7f98f80740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98f7fff700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98f8101e30 0x7f98f80740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98f7fff700 1 -- 172.21.15.37:0/2138356059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98f8107e10 msgr2=0x7f98f8074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98f7fff700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98f8107e10 0x7f98f8074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98f7fff700 1 -- 172.21.15.37:0/2138356059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f81a69d0 con 0x7f98f81027f0 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.854+0000 7f98fd3e7700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98f8107e10 0x7f98f8074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.855+0000 7f98f7fff700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98f81027f0 0x7f98f8074620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f98e400b530 tx=0x7f98e400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.855+0000 7f98f5ffb700 1 -- 172.21.15.37:0/2138356059 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4015020 con 0x7f98f81027f0 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.855+0000 7f98f5ffb700 1 -- 172.21.15.37:0/2138356059 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98e4011d10 con 0x7f98f81027f0 2024-03-26T06:22:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.855+0000 7f98fee4a700 1 -- 172.21.15.37:0/2138356059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f81a6cc0 con 0x7f98f81027f0 2024-03-26T06:22:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.855+0000 7f98f5ffb700 1 -- 172.21.15.37:0/2138356059 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4010e50 con 0x7f98f81027f0 2024-03-26T06:22:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.855+0000 7f98fee4a700 1 -- 172.21.15.37:0/2138356059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f81a71f0 con 0x7f98f81027f0 2024-03-26T06:22:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.856+0000 7f98fee4a700 1 -- 172.21.15.37:0/2138356059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f8068aa0 con 0x7f98f81027f0 2024-03-26T06:22:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.857+0000 7f98f5ffb700 1 -- 172.21.15.37:0/2138356059 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f98e4018020 con 0x7f98f81027f0 2024-03-26T06:22:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.857+0000 7f98f5ffb700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f98e80a2b40 0x7f98e80a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:11.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.858+0000 7f98fcbe6700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f98e80a2b40 0x7f98e80a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:11.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.858+0000 7f98f5ffb700 1 -- 172.21.15.37:0/2138356059 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f98e40ce7e0 con 0x7f98f81027f0 2024-03-26T06:22:11.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.858+0000 7f98fcbe6700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f98e80a2b40 0x7f98e80a5000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f98ec005fd0 tx=0x7f98ec00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:11.861+0000 7f98f5ffb700 1 -- 172.21.15.37:0/2138356059 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f98e401f020 con 0x7f98f81027f0 2024-03-26T06:22:12.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.060+0000 7f98fee4a700 1 -- 172.21.15.37:0/2138356059 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98f8062220 con 0x7f98e80a2b40 2024-03-26T06:22:12.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.068+0000 7f98f5ffb700 1 -- 172.21.15.37:0/2138356059 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258763 (secure 0 0 0) 0x7f98f8062220 con 0x7f98e80a2b40 2024-03-26T06:22:12.073 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:12.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.070+0000 7f98fee4a700 1 -- 172.21.15.37:0/2138356059 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f98e80a2b40 msgr2=0x7f98e80a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.071+0000 7f98fee4a700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f98e80a2b40 0x7f98e80a5000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f98ec005fd0 tx=0x7f98ec00a560 comp rx=0 tx=0).stop 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.071+0000 7f98fee4a700 1 -- 172.21.15.37:0/2138356059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98f81027f0 msgr2=0x7f98f8074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.071+0000 7f98fee4a700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98f81027f0 0x7f98f8074620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f98e400b530 tx=0x7f98e400b8f0 comp rx=0 tx=0).stop 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.071+0000 7f98fee4a700 1 -- 172.21.15.37:0/2138356059 shutdown_connections 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.071+0000 7f98fee4a700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f98e80a2b40 0x7f98e80a5000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.071+0000 7f98fee4a700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98f8107e10 0x7f98f8074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.071+0000 7f98fee4a700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98f8101e30 0x7f98f80740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.071+0000 7f98fee4a700 1 --2- 172.21.15.37:0/2138356059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98f81027f0 0x7f98f8074620 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.071+0000 7f98fee4a700 1 -- 172.21.15.37:0/2138356059 >> 172.21.15.37:0/2138356059 conn(0x7f98f8078f10 msgr2=0x7f98f8100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.071+0000 7f98fee4a700 1 -- 172.21.15.37:0/2138356059 shutdown_connections 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:12.071+0000 7f98fee4a700 1 -- 172.21.15.37:0/2138356059 wait complete. 2024-03-26T06:22:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-26T06:22:12.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:12 smithi037 ceph-mon[31974]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:22:12.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:12 smithi066 ceph-mon[36504]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:22:12.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:12 smithi149 ceph-mon[36681]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:22:12.791 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-03-26T06:22:10.763642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656327,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":851,"num_read_kb":551,"num_write":416,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366684,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575140,"statfs":{"total":1151940427776,"available":1151564943360,"internally_reserved":0,"allocated":13627392,"data_stored":7436457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003036"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.721646+0000","last_change":"2024-03-26T06:20:54.294436+0000","last_active":"2024-03-26T06:21:48.721646+0000","last_peered":"2024-03-26T06:21:48.721646+0000","last_clean":"2024-03-26T06:21:48.721646+0000","last_became_active":"2024-03-26T06:20:54.293049+0000","last_became_peered":"2024-03-26T06:20:54.293049+0000","last_unstale":"2024-03-26T06:21:48.721646+0000","last_undegraded":"2024-03-26T06:21:48.721646+0000","last_fullsized":"2024-03-26T06:21:48.721646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:44:02.729046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715930+0000","last_change":"2024-03-26T06:20:54.292560+0000","last_active":"2024-03-26T06:21:48.715930+0000","last_peered":"2024-03-26T06:21:48.715930+0000","last_clean":"2024-03-26T06:21:48.715930+0000","last_became_active":"2024-03-26T06:20:54.292166+0000","last_became_peered":"2024-03-26T06:20:54.292166+0000","last_unstale":"2024-03-26T06:21:48.715930+0000","last_undegraded":"2024-03-26T06:21:48.715930+0000","last_fullsized":"2024-03-26T06:21:48.715930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:45.255912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715710+0000","last_change":"2024-03-26T06:20:54.294350+0000","last_active":"2024-03-26T06:21:48.715710+0000","last_peered":"2024-03-26T06:21:48.715710+0000","last_clean":"2024-03-26T06:21:48.715710+0000","last_became_active":"2024-03-26T06:20:54.293813+0000","last_became_peered":"2024-03-26T06:20:54.293813+0000","last_unstale":"2024-03-26T06:21:48.715710+0000","last_undegraded":"2024-03-26T06:21:48.715710+0000","last_fullsized":"2024-03-26T06:21:48.715710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:44:31.126828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.739898+0000","last_change":"2024-03-26T06:20:54.299151+0000","last_active":"2024-03-26T06:21:48.739898+0000","last_peered":"2024-03-26T06:21:48.739898+0000","last_clean":"2024-03-26T06:21:48.739898+0000","last_became_active":"2024-03-26T06:20:54.298820+0000","last_became_peered":"2024-03-26T06:20:54.298820+0000","last_unstale":"2024-03-26T06:21:48.739898+0000","last_undegraded":"2024-03-26T06:21:48.739898+0000","last_fullsized":"2024-03-26T06:21:48.739898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:47:25.503692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718913+0000","last_change":"2024-03-26T06:20:54.299977+0000","last_active":"2024-03-26T06:21:48.718913+0000","last_peered":"2024-03-26T06:21:48.718913+0000","last_clean":"2024-03-26T06:21:48.718913+0000","last_became_active":"2024-03-26T06:20:54.299641+0000","last_became_peered":"2024-03-26T06:20:54.299641+0000","last_unstale":"2024-03-26T06:21:48.718913+0000","last_undegraded":"2024-03-26T06:21:48.718913+0000","last_fullsized":"2024-03-26T06:21:48.718913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:42:32.478970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.353109+0000","last_change":"2024-03-26T06:20:54.294308+0000","last_active":"2024-03-26T06:21:49.353109+0000","last_peered":"2024-03-26T06:21:49.353109+0000","last_clean":"2024-03-26T06:21:49.353109+0000","last_became_active":"2024-03-26T06:20:54.293882+0000","last_became_peered":"2024-03-26T06:20:54.293882+0000","last_unstale":"2024-03-26T06:21:49.353109+0000","last_undegraded":"2024-03-26T06:21:49.353109+0000","last_fullsized":"2024-03-26T06:21:49.353109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:26:26.014097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716076+0000","last_change":"2024-03-26T06:20:54.301644+0000","last_active":"2024-03-26T06:21:48.716076+0000","last_peered":"2024-03-26T06:21:48.716076+0000","last_clean":"2024-03-26T06:21:48.716076+0000","last_became_active":"2024-03-26T06:20:54.301257+0000","last_became_peered":"2024-03-26T06:20:54.301257+0000","last_unstale":"2024-03-26T06:21:48.716076+0000","last_undegraded":"2024-03-26T06:21:48.716076+0000","last_fullsized":"2024-03-26T06:21:48.716076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:43:27.602288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718499+0000","last_change":"2024-03-26T06:20:54.296421+0000","last_active":"2024-03-26T06:21:48.718499+0000","last_peered":"2024-03-26T06:21:48.718499+0000","last_clean":"2024-03-26T06:21:48.718499+0000","last_became_active":"2024-03-26T06:20:54.296086+0000","last_became_peered":"2024-03-26T06:20:54.296086+0000","last_unstale":"2024-03-26T06:21:48.718499+0000","last_undegraded":"2024-03-26T06:21:48.718499+0000","last_fullsized":"2024-03-26T06:21:48.718499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:22:25.532160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718901+0000","last_change":"2024-03-26T06:20:54.297206+0000","last_active":"2024-03-26T06:21:48.718901+0000","last_peered":"2024-03-26T06:21:48.718901+0000","last_clean":"2024-03-26T06:21:48.718901+0000","last_became_active":"2024-03-26T06:20:54.296849+0000","last_became_peered":"2024-03-26T06:20:54.296849+0000","last_unstale":"2024-03-26T06:21:48.718901+0000","last_undegraded":"2024-03-26T06:21:48.718901+0000","last_fullsized":"2024-03-26T06:21:48.718901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:24:49.972237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.353439+0000","last_change":"2024-03-26T06:20:54.295048+0000","last_active":"2024-03-26T06:21:49.353439+0000","last_peered":"2024-03-26T06:21:49.353439+0000","last_clean":"2024-03-26T06:21:49.353439+0000","last_became_active":"2024-03-26T06:20:54.294161+0000","last_became_peered":"2024-03-26T06:20:54.294161+0000","last_unstale":"2024-03-26T06:21:49.353439+0000","last_undegraded":"2024-03-26T06:21:49.353439+0000","last_fullsized":"2024-03-26T06:21:49.353439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:41:16.779030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716452+0000","last_change":"2024-03-26T06:20:54.300235+0000","last_active":"2024-03-26T06:21:48.716452+0000","last_peered":"2024-03-26T06:21:48.716452+0000","last_clean":"2024-03-26T06:21:48.716452+0000","last_became_active":"2024-03-26T06:20:54.299677+0000","last_became_peered":"2024-03-26T06:20:54.299677+0000","last_unstale":"2024-03-26T06:21:48.716452+0000","last_undegraded":"2024-03-26T06:21:48.716452+0000","last_fullsized":"2024-03-26T06:21:48.716452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:18:57.028840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.739839+0000","last_change":"2024-03-26T06:20:54.303953+0000","last_active":"2024-03-26T06:21:48.739839+0000","last_peered":"2024-03-26T06:21:48.739839+0000","last_clean":"2024-03-26T06:21:48.739839+0000","last_became_active":"2024-03-26T06:20:54.303339+0000","last_became_peered":"2024-03-26T06:20:54.303339+0000","last_unstale":"2024-03-26T06:21:48.739839+0000","last_undegraded":"2024-03-26T06:21:48.739839+0000","last_fullsized":"2024-03-26T06:21:48.739839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:54:38.511221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050284+0000","last_change":"2024-03-26T06:20:54.283535+0000","last_active":"2024-03-26T06:21:49.050284+0000","last_peered":"2024-03-26T06:21:49.050284+0000","last_clean":"2024-03-26T06:21:49.050284+0000","last_became_active":"2024-03-26T06:20:54.282820+0000","last_became_peered":"2024-03-26T06:20:54.282820+0000","last_unstale":"2024-03-26T06:21:49.050284+0000","last_undegraded":"2024-03-26T06:21:49.050284+0000","last_fullsized":"2024-03-26T06:21:49.050284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:13:07.906341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715676+0000","last_change":"2024-03-26T06:20:54.292841+0000","last_active":"2024-03-26T06:21:48.715676+0000","last_peered":"2024-03-26T06:21:48.715676+0000","last_clean":"2024-03-26T06:21:48.715676+0000","last_became_active":"2024-03-26T06:20:54.292530+0000","last_became_peered":"2024-03-26T06:20:54.292530+0000","last_unstale":"2024-03-26T06:21:48.715676+0000","last_undegraded":"2024-03-26T06:21:48.715676+0000","last_fullsized":"2024-03-26T06:21:48.715676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:35:54.592898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715598+0000","last_change":"2024-03-26T06:20:54.306753+0000","last_active":"2024-03-26T06:21:48.715598+0000","last_peered":"2024-03-26T06:21:48.715598+0000","last_clean":"2024-03-26T06:21:48.715598+0000","last_became_active":"2024-03-26T06:20:54.306379+0000","last_became_peered":"2024-03-26T06:20:54.306379+0000","last_unstale":"2024-03-26T06:21:48.715598+0000","last_undegraded":"2024-03-26T06:21:48.715598+0000","last_fullsized":"2024-03-26T06:21:48.715598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:04:19.879733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718360+0000","last_change":"2024-03-26T06:20:54.305719+0000","last_active":"2024-03-26T06:21:48.718360+0000","last_peered":"2024-03-26T06:21:48.718360+0000","last_clean":"2024-03-26T06:21:48.718360+0000","last_became_active":"2024-03-26T06:20:54.305336+0000","last_became_peered":"2024-03-26T06:20:54.305336+0000","last_unstale":"2024-03-26T06:21:48.718360+0000","last_undegraded":"2024-03-26T06:21:48.718360+0000","last_fullsized":"2024-03-26T06:21:48.718360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:22:15.837575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.721704+0000","last_change":"2024-03-26T06:20:54.293451+0000","last_active":"2024-03-26T06:21:48.721704+0000","last_peered":"2024-03-26T06:21:48.721704+0000","last_clean":"2024-03-26T06:21:48.721704+0000","last_became_active":"2024-03-26T06:20:54.293139+0000","last_became_peered":"2024-03-26T06:20:54.293139+0000","last_unstale":"2024-03-26T06:21:48.721704+0000","last_undegraded":"2024-03-26T06:21:48.721704+0000","last_fullsized":"2024-03-26T06:21:48.721704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:56:04.407687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719165+0000","last_change":"2024-03-26T06:20:54.290256+0000","last_active":"2024-03-26T06:21:48.719165+0000","last_peered":"2024-03-26T06:21:48.719165+0000","last_clean":"2024-03-26T06:21:48.719165+0000","last_became_active":"2024-03-26T06:20:54.289773+0000","last_became_peered":"2024-03-26T06:20:54.289773+0000","last_unstale":"2024-03-26T06:21:48.719165+0000","last_undegraded":"2024-03-26T06:21:48.719165+0000","last_fullsized":"2024-03-26T06:21:48.719165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:46:07.503946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719408+0000","last_change":"2024-03-26T06:20:54.302601+0000","last_active":"2024-03-26T06:21:48.719408+0000","last_peered":"2024-03-26T06:21:48.719408+0000","last_clean":"2024-03-26T06:21:48.719408+0000","last_became_active":"2024-03-26T06:20:54.301523+0000","last_became_peered":"2024-03-26T06:20:54.301523+0000","last_unstale":"2024-03-26T06:21:48.719408+0000","last_undegraded":"2024-03-26T06:21:48.719408+0000","last_fullsized":"2024-03-26T06:21:48.719408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:58:49.055958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":101,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:06.678048+0000","last_change":"2024-03-26T06:20:53.280275+0000","last_active":"2024-03-26T06:22:06.678048+0000","last_peered":"2024-03-26T06:22:06.678048+0000","last_clean":"2024-03-26T06:22:06.678048+0000","last_became_active":"2024-03-26T06:20:53.279916+0000","last_became_peered":"2024-03-26T06:20:53.279916+0000","last_unstale":"2024-03-26T06:22:06.678048+0000","last_undegraded":"2024-03-26T06:22:06.678048+0000","last_fullsized":"2024-03-26T06:22:06.678048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:50.844507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719551+0000","last_change":"2024-03-26T06:20:54.302720+0000","last_active":"2024-03-26T06:21:48.719551+0000","last_peered":"2024-03-26T06:21:48.719551+0000","last_clean":"2024-03-26T06:21:48.719551+0000","last_became_active":"2024-03-26T06:20:54.302171+0000","last_became_peered":"2024-03-26T06:20:54.302171+0000","last_unstale":"2024-03-26T06:21:48.719551+0000","last_undegraded":"2024-03-26T06:21:48.719551+0000","last_fullsized":"2024-03-26T06:21:48.719551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:23:12.887883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719700+0000","last_change":"2024-03-26T06:20:53.282355+0000","last_active":"2024-03-26T06:21:48.719700+0000","last_peered":"2024-03-26T06:21:48.719700+0000","last_clean":"2024-03-26T06:21:48.719700+0000","last_became_active":"2024-03-26T06:20:53.282062+0000","last_became_peered":"2024-03-26T06:20:53.282062+0000","last_unstale":"2024-03-26T06:21:48.719700+0000","last_undegraded":"2024-03-26T06:21:48.719700+0000","last_fullsized":"2024-03-26T06:21:48.719700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:58:33.398589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716131+0000","last_change":"2024-03-26T06:20:54.294924+0000","last_active":"2024-03-26T06:21:48.716131+0000","last_peered":"2024-03-26T06:21:48.716131+0000","last_clean":"2024-03-26T06:21:48.716131+0000","last_became_active":"2024-03-26T06:20:54.293971+0000","last_became_peered":"2024-03-26T06:20:54.293971+0000","last_unstale":"2024-03-26T06:21:48.716131+0000","last_undegraded":"2024-03-26T06:21:48.716131+0000","last_fullsized":"2024-03-26T06:21:48.716131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:54:15.457497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719986+0000","last_change":"2024-03-26T06:20:53.286284+0000","last_active":"2024-03-26T06:21:48.719986+0000","last_peered":"2024-03-26T06:21:48.719986+0000","last_clean":"2024-03-26T06:21:48.719986+0000","last_became_active":"2024-03-26T06:20:53.285979+0000","last_became_peered":"2024-03-26T06:20:53.285979+0000","last_unstale":"2024-03-26T06:21:48.719986+0000","last_undegraded":"2024-03-26T06:21:48.719986+0000","last_fullsized":"2024-03-26T06:21:48.719986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:33:31.989055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718710+0000","last_change":"2024-03-26T06:20:54.290137+0000","last_active":"2024-03-26T06:21:48.718710+0000","last_peered":"2024-03-26T06:21:48.718710+0000","last_clean":"2024-03-26T06:21:48.718710+0000","last_became_active":"2024-03-26T06:20:54.289727+0000","last_became_peered":"2024-03-26T06:20:54.289727+0000","last_unstale":"2024-03-26T06:21:48.718710+0000","last_undegraded":"2024-03-26T06:21:48.718710+0000","last_fullsized":"2024-03-26T06:21:48.718710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:31:26.447358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.733189+0000","last_change":"2024-03-26T06:21:48.733189+0000","last_active":"2024-03-26T06:21:48.733189+0000","last_peered":"2024-03-26T06:21:48.733189+0000","last_clean":"2024-03-26T06:21:48.733189+0000","last_became_active":"2024-03-26T06:21:48.727506+0000","last_became_peered":"2024-03-26T06:21:48.727506+0000","last_unstale":"2024-03-26T06:21:48.733189+0000","last_undegraded":"2024-03-26T06:21:48.733189+0000","last_fullsized":"2024-03-26T06:21:48.733189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:13:38.631171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.739861+0000","last_change":"2024-03-26T06:21:48.739861+0000","last_active":"2024-03-26T06:21:48.739861+0000","last_peered":"2024-03-26T06:21:48.739861+0000","last_clean":"2024-03-26T06:21:48.739861+0000","last_became_active":"2024-03-26T06:21:48.739569+0000","last_became_peered":"2024-03-26T06:21:48.739569+0000","last_unstale":"2024-03-26T06:21:48.739861+0000","last_undegraded":"2024-03-26T06:21:48.739861+0000","last_fullsized":"2024-03-26T06:21:48.739861+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:44:04.878070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:05.688099+0000","last_change":"2024-03-26T06:20:53.287298+0000","last_active":"2024-03-26T06:22:05.688099+0000","last_peered":"2024-03-26T06:22:05.688099+0000","last_clean":"2024-03-26T06:22:05.688099+0000","last_became_active":"2024-03-26T06:20:53.286987+0000","last_became_peered":"2024-03-26T06:20:53.286987+0000","last_unstale":"2024-03-26T06:22:05.688099+0000","last_undegraded":"2024-03-26T06:22:05.688099+0000","last_fullsized":"2024-03-26T06:22:05.688099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:20.343750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718700+0000","last_change":"2024-03-26T06:20:54.291410+0000","last_active":"2024-03-26T06:21:48.718700+0000","last_peered":"2024-03-26T06:21:48.718700+0000","last_clean":"2024-03-26T06:21:48.718700+0000","last_became_active":"2024-03-26T06:20:54.291085+0000","last_became_peered":"2024-03-26T06:20:54.291085+0000","last_unstale":"2024-03-26T06:21:48.718700+0000","last_undegraded":"2024-03-26T06:21:48.718700+0000","last_fullsized":"2024-03-26T06:21:48.718700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:39.727359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.352482+0000","last_change":"2024-03-26T06:20:53.285476+0000","last_active":"2024-03-26T06:21:49.352482+0000","last_peered":"2024-03-26T06:21:49.352482+0000","last_clean":"2024-03-26T06:21:49.352482+0000","last_became_active":"2024-03-26T06:20:53.285009+0000","last_became_peered":"2024-03-26T06:20:53.285009+0000","last_unstale":"2024-03-26T06:21:49.352482+0000","last_undegraded":"2024-03-26T06:21:49.352482+0000","last_fullsized":"2024-03-26T06:21:49.352482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:40.006117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724368+0000","last_change":"2024-03-26T06:20:54.299909+0000","last_active":"2024-03-26T06:21:48.724368+0000","last_peered":"2024-03-26T06:21:48.724368+0000","last_clean":"2024-03-26T06:21:48.724368+0000","last_became_active":"2024-03-26T06:20:54.299582+0000","last_became_peered":"2024-03-26T06:20:54.299582+0000","last_unstale":"2024-03-26T06:21:48.724368+0000","last_undegraded":"2024-03-26T06:21:48.724368+0000","last_fullsized":"2024-03-26T06:21:48.724368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:16:41.110029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050119+0000","last_change":"2024-03-26T06:20:53.281619+0000","last_active":"2024-03-26T06:21:49.050119+0000","last_peered":"2024-03-26T06:21:49.050119+0000","last_clean":"2024-03-26T06:21:49.050119+0000","last_became_active":"2024-03-26T06:20:53.279792+0000","last_became_peered":"2024-03-26T06:20:53.279792+0000","last_unstale":"2024-03-26T06:21:49.050119+0000","last_undegraded":"2024-03-26T06:21:49.050119+0000","last_fullsized":"2024-03-26T06:21:49.050119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:17:17.892132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050994+0000","last_change":"2024-03-26T06:21:49.050994+0000","last_active":"2024-03-26T06:21:49.050994+0000","last_peered":"2024-03-26T06:21:49.050994+0000","last_clean":"2024-03-26T06:21:49.050994+0000","last_became_active":"2024-03-26T06:21:49.050624+0000","last_became_peered":"2024-03-26T06:21:49.050624+0000","last_unstale":"2024-03-26T06:21:49.050994+0000","last_undegraded":"2024-03-26T06:21:49.050994+0000","last_fullsized":"2024-03-26T06:21:49.050994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:34:18.507960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.723902+0000","last_change":"2024-03-26T06:20:53.285367+0000","last_active":"2024-03-26T06:21:48.723902+0000","last_peered":"2024-03-26T06:21:48.723902+0000","last_clean":"2024-03-26T06:21:48.723902+0000","last_became_active":"2024-03-26T06:20:53.285022+0000","last_became_peered":"2024-03-26T06:20:53.285022+0000","last_unstale":"2024-03-26T06:21:48.723902+0000","last_undegraded":"2024-03-26T06:21:48.723902+0000","last_fullsized":"2024-03-26T06:21:48.723902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:01:32.288961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718531+0000","last_change":"2024-03-26T06:20:54.290125+0000","last_active":"2024-03-26T06:21:48.718531+0000","last_peered":"2024-03-26T06:21:48.718531+0000","last_clean":"2024-03-26T06:21:48.718531+0000","last_became_active":"2024-03-26T06:20:54.289739+0000","last_became_peered":"2024-03-26T06:20:54.289739+0000","last_unstale":"2024-03-26T06:21:48.718531+0000","last_undegraded":"2024-03-26T06:21:48.718531+0000","last_fullsized":"2024-03-26T06:21:48.718531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:51:01.087526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":161,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:07.690598+0000","last_change":"2024-03-26T06:20:53.289898+0000","last_active":"2024-03-26T06:22:07.690598+0000","last_peered":"2024-03-26T06:22:07.690598+0000","last_clean":"2024-03-26T06:22:07.690598+0000","last_became_active":"2024-03-26T06:20:53.289592+0000","last_became_peered":"2024-03-26T06:20:53.289592+0000","last_unstale":"2024-03-26T06:22:07.690598+0000","last_undegraded":"2024-03-26T06:22:07.690598+0000","last_fullsized":"2024-03-26T06:22:07.690598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:11:41.249408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718811+0000","last_change":"2024-03-26T06:20:54.290970+0000","last_active":"2024-03-26T06:21:48.718811+0000","last_peered":"2024-03-26T06:21:48.718811+0000","last_clean":"2024-03-26T06:21:48.718811+0000","last_became_active":"2024-03-26T06:20:54.289686+0000","last_became_peered":"2024-03-26T06:20:54.289686+0000","last_unstale":"2024-03-26T06:21:48.718811+0000","last_undegraded":"2024-03-26T06:21:48.718811+0000","last_fullsized":"2024-03-26T06:21:48.718811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:39:42.552786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.745217+0000","last_change":"2024-03-26T06:21:48.745217+0000","last_active":"2024-03-26T06:21:48.745217+0000","last_peered":"2024-03-26T06:21:48.745217+0000","last_clean":"2024-03-26T06:21:48.745217+0000","last_became_active":"2024-03-26T06:21:48.741202+0000","last_became_peered":"2024-03-26T06:21:48.741202+0000","last_unstale":"2024-03-26T06:21:48.745217+0000","last_undegraded":"2024-03-26T06:21:48.745217+0000","last_fullsized":"2024-03-26T06:21:48.745217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:06:04.855755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716119+0000","last_change":"2024-03-26T06:20:54.299129+0000","last_active":"2024-03-26T06:21:48.716119+0000","last_peered":"2024-03-26T06:21:48.716119+0000","last_clean":"2024-03-26T06:21:48.716119+0000","last_became_active":"2024-03-26T06:20:54.298777+0000","last_became_peered":"2024-03-26T06:20:54.298777+0000","last_unstale":"2024-03-26T06:21:48.716119+0000","last_undegraded":"2024-03-26T06:21:48.716119+0000","last_fullsized":"2024-03-26T06:21:48.716119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:29:38.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724463+0000","last_change":"2024-03-26T06:20:53.284988+0000","last_active":"2024-03-26T06:21:48.724463+0000","last_peered":"2024-03-26T06:21:48.724463+0000","last_clean":"2024-03-26T06:21:48.724463+0000","last_became_active":"2024-03-26T06:20:53.284684+0000","last_became_peered":"2024-03-26T06:20:53.284684+0000","last_unstale":"2024-03-26T06:21:48.724463+0000","last_undegraded":"2024-03-26T06:21:48.724463+0000","last_fullsized":"2024-03-26T06:21:48.724463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:31:45.004042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718889+0000","last_change":"2024-03-26T06:20:54.307147+0000","last_active":"2024-03-26T06:21:48.718889+0000","last_peered":"2024-03-26T06:21:48.718889+0000","last_clean":"2024-03-26T06:21:48.718889+0000","last_became_active":"2024-03-26T06:20:54.306797+0000","last_became_peered":"2024-03-26T06:20:54.306797+0000","last_unstale":"2024-03-26T06:21:48.718889+0000","last_undegraded":"2024-03-26T06:21:48.718889+0000","last_fullsized":"2024-03-26T06:21:48.718889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:20:58.904884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'27","reported_seq":189,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:07.705835+0000","last_change":"2024-03-26T06:20:53.284577+0000","last_active":"2024-03-26T06:22:07.705835+0000","last_peered":"2024-03-26T06:22:07.705835+0000","last_clean":"2024-03-26T06:22:07.705835+0000","last_became_active":"2024-03-26T06:20:53.284161+0000","last_became_peered":"2024-03-26T06:20:53.284161+0000","last_unstale":"2024-03-26T06:22:07.705835+0000","last_undegraded":"2024-03-26T06:22:07.705835+0000","last_fullsized":"2024-03-26T06:22:07.705835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:57.765950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":143,"num_read_kb":129,"num_write":29,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719781+0000","last_change":"2024-03-26T06:20:54.301794+0000","last_active":"2024-03-26T06:21:48.719781+0000","last_peered":"2024-03-26T06:21:48.719781+0000","last_clean":"2024-03-26T06:21:48.719781+0000","last_became_active":"2024-03-26T06:20:54.301445+0000","last_became_peered":"2024-03-26T06:20:54.301445+0000","last_unstale":"2024-03-26T06:21:48.719781+0000","last_undegraded":"2024-03-26T06:21:48.719781+0000","last_fullsized":"2024-03-26T06:21:48.719781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:32:18.280992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719138+0000","last_change":"2024-03-26T06:20:53.282394+0000","last_active":"2024-03-26T06:21:48.719138+0000","last_peered":"2024-03-26T06:21:48.719138+0000","last_clean":"2024-03-26T06:21:48.719138+0000","last_became_active":"2024-03-26T06:20:53.282068+0000","last_became_peered":"2024-03-26T06:20:53.282068+0000","last_unstale":"2024-03-26T06:21:48.719138+0000","last_undegraded":"2024-03-26T06:21:48.719138+0000","last_fullsized":"2024-03-26T06:21:48.719138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:04:00.503682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050085+0000","last_change":"2024-03-26T06:20:54.303294+0000","last_active":"2024-03-26T06:21:49.050085+0000","last_peered":"2024-03-26T06:21:49.050085+0000","last_clean":"2024-03-26T06:21:49.050085+0000","last_became_active":"2024-03-26T06:20:54.301342+0000","last_became_peered":"2024-03-26T06:20:54.301342+0000","last_unstale":"2024-03-26T06:21:49.050085+0000","last_undegraded":"2024-03-26T06:21:49.050085+0000","last_fullsized":"2024-03-26T06:21:49.050085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:53:51.012761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724063+0000","last_change":"2024-03-26T06:20:53.286119+0000","last_active":"2024-03-26T06:21:48.724063+0000","last_peered":"2024-03-26T06:21:48.724063+0000","last_clean":"2024-03-26T06:21:48.724063+0000","last_became_active":"2024-03-26T06:20:53.285784+0000","last_became_peered":"2024-03-26T06:20:53.285784+0000","last_unstale":"2024-03-26T06:21:48.724063+0000","last_undegraded":"2024-03-26T06:21:48.724063+0000","last_fullsized":"2024-03-26T06:21:48.724063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:54:36.786966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716316+0000","last_change":"2024-03-26T06:20:54.309706+0000","last_active":"2024-03-26T06:21:48.716316+0000","last_peered":"2024-03-26T06:21:48.716316+0000","last_clean":"2024-03-26T06:21:48.716316+0000","last_became_active":"2024-03-26T06:20:54.309291+0000","last_became_peered":"2024-03-26T06:20:54.309291+0000","last_unstale":"2024-03-26T06:21:48.716316+0000","last_undegraded":"2024-03-26T06:21:48.716316+0000","last_fullsized":"2024-03-26T06:21:48.716316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:51:52.150636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:04.675812+0000","last_change":"2024-03-26T06:20:53.278569+0000","last_active":"2024-03-26T06:22:04.675812+0000","last_peered":"2024-03-26T06:22:04.675812+0000","last_clean":"2024-03-26T06:22:04.675812+0000","last_became_active":"2024-03-26T06:20:53.278207+0000","last_became_peered":"2024-03-26T06:20:53.278207+0000","last_unstale":"2024-03-26T06:22:04.675812+0000","last_undegraded":"2024-03-26T06:22:04.675812+0000","last_fullsized":"2024-03-26T06:22:04.675812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:59:12.345429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716018+0000","last_change":"2024-03-26T06:20:54.308325+0000","last_active":"2024-03-26T06:21:48.716018+0000","last_peered":"2024-03-26T06:21:48.716018+0000","last_clean":"2024-03-26T06:21:48.716018+0000","last_became_active":"2024-03-26T06:20:54.307970+0000","last_became_peered":"2024-03-26T06:20:54.307970+0000","last_unstale":"2024-03-26T06:21:48.716018+0000","last_undegraded":"2024-03-26T06:21:48.716018+0000","last_fullsized":"2024-03-26T06:21:48.716018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:18:26.597073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718555+0000","last_change":"2024-03-26T06:20:53.284675+0000","last_active":"2024-03-26T06:21:48.718555+0000","last_peered":"2024-03-26T06:21:48.718555+0000","last_clean":"2024-03-26T06:21:48.718555+0000","last_became_active":"2024-03-26T06:20:53.284368+0000","last_became_peered":"2024-03-26T06:20:53.284368+0000","last_unstale":"2024-03-26T06:21:48.718555+0000","last_undegraded":"2024-03-26T06:21:48.718555+0000","last_fullsized":"2024-03-26T06:21:48.718555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:15:51.733268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724138+0000","last_change":"2024-03-26T06:20:54.296348+0000","last_active":"2024-03-26T06:21:48.724138+0000","last_peered":"2024-03-26T06:21:48.724138+0000","last_clean":"2024-03-26T06:21:48.724138+0000","last_became_active":"2024-03-26T06:20:54.296019+0000","last_became_peered":"2024-03-26T06:20:54.296019+0000","last_unstale":"2024-03-26T06:21:48.724138+0000","last_undegraded":"2024-03-26T06:21:48.724138+0000","last_fullsized":"2024-03-26T06:21:48.724138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:25:28.498887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719941+0000","last_change":"2024-03-26T06:20:53.281314+0000","last_active":"2024-03-26T06:21:48.719941+0000","last_peered":"2024-03-26T06:21:48.719941+0000","last_clean":"2024-03-26T06:21:48.719941+0000","last_became_active":"2024-03-26T06:20:53.281007+0000","last_became_peered":"2024-03-26T06:20:53.281007+0000","last_unstale":"2024-03-26T06:21:48.719941+0000","last_undegraded":"2024-03-26T06:21:48.719941+0000","last_fullsized":"2024-03-26T06:21:48.719941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:39:25.840470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.721263+0000","last_change":"2024-03-26T06:20:54.290011+0000","last_active":"2024-03-26T06:21:48.721263+0000","last_peered":"2024-03-26T06:21:48.721263+0000","last_clean":"2024-03-26T06:21:48.721263+0000","last_became_active":"2024-03-26T06:20:54.289492+0000","last_became_peered":"2024-03-26T06:20:54.289492+0000","last_unstale":"2024-03-26T06:21:48.721263+0000","last_undegraded":"2024-03-26T06:21:48.721263+0000","last_fullsized":"2024-03-26T06:21:48.721263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:39:16.960414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718559+0000","last_change":"2024-03-26T06:20:53.276374+0000","last_active":"2024-03-26T06:21:48.718559+0000","last_peered":"2024-03-26T06:21:48.718559+0000","last_clean":"2024-03-26T06:21:48.718559+0000","last_became_active":"2024-03-26T06:20:53.276031+0000","last_became_peered":"2024-03-26T06:20:53.276031+0000","last_unstale":"2024-03-26T06:21:48.718559+0000","last_undegraded":"2024-03-26T06:21:48.718559+0000","last_fullsized":"2024-03-26T06:21:48.718559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:14.601841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.353310+0000","last_change":"2024-03-26T06:20:54.304279+0000","last_active":"2024-03-26T06:21:49.353310+0000","last_peered":"2024-03-26T06:21:49.353310+0000","last_clean":"2024-03-26T06:21:49.353310+0000","last_became_active":"2024-03-26T06:20:54.303859+0000","last_became_peered":"2024-03-26T06:20:54.303859+0000","last_unstale":"2024-03-26T06:21:49.353310+0000","last_undegraded":"2024-03-26T06:21:49.353310+0000","last_fullsized":"2024-03-26T06:21:49.353310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:32:37.969077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:05.677321+0000","last_change":"2024-03-26T06:20:53.286038+0000","last_active":"2024-03-26T06:22:05.677321+0000","last_peered":"2024-03-26T06:22:05.677321+0000","last_clean":"2024-03-26T06:22:05.677321+0000","last_became_active":"2024-03-26T06:20:53.285353+0000","last_became_peered":"2024-03-26T06:20:53.285353+0000","last_unstale":"2024-03-26T06:22:05.677321+0000","last_undegraded":"2024-03-26T06:22:05.677321+0000","last_fullsized":"2024-03-26T06:22:05.677321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:43:53.970393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.353081+0000","last_change":"2024-03-26T06:20:54.294425+0000","last_active":"2024-03-26T06:21:49.353081+0000","last_peered":"2024-03-26T06:21:49.353081+0000","last_clean":"2024-03-26T06:21:49.353081+0000","last_became_active":"2024-03-26T06:20:54.293607+0000","last_became_peered":"2024-03-26T06:20:54.293607+0000","last_unstale":"2024-03-26T06:21:49.353081+0000","last_undegraded":"2024-03-26T06:21:49.353081+0000","last_fullsized":"2024-03-26T06:21:49.353081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:12:49.118427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724238+0000","last_change":"2024-03-26T06:20:54.291038+0000","last_active":"2024-03-26T06:21:48.724238+0000","last_peered":"2024-03-26T06:21:48.724238+0000","last_clean":"2024-03-26T06:21:48.724238+0000","last_became_active":"2024-03-26T06:20:54.290678+0000","last_became_peered":"2024-03-26T06:20:54.290678+0000","last_unstale":"2024-03-26T06:21:48.724238+0000","last_undegraded":"2024-03-26T06:21:48.724238+0000","last_fullsized":"2024-03-26T06:21:48.724238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:29:30.528408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:05.676638+0000","last_change":"2024-03-26T06:20:53.288795+0000","last_active":"2024-03-26T06:22:05.676638+0000","last_peered":"2024-03-26T06:22:05.676638+0000","last_clean":"2024-03-26T06:22:05.676638+0000","last_became_active":"2024-03-26T06:20:53.288468+0000","last_became_peered":"2024-03-26T06:20:53.288468+0000","last_unstale":"2024-03-26T06:22:05.676638+0000","last_undegraded":"2024-03-26T06:22:05.676638+0000","last_fullsized":"2024-03-26T06:22:05.676638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:16:05.569407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719380+0000","last_change":"2024-03-26T06:20:54.296706+0000","last_active":"2024-03-26T06:21:48.719380+0000","last_peered":"2024-03-26T06:21:48.719380+0000","last_clean":"2024-03-26T06:21:48.719380+0000","last_became_active":"2024-03-26T06:20:54.296387+0000","last_became_peered":"2024-03-26T06:20:54.296387+0000","last_unstale":"2024-03-26T06:21:48.719380+0000","last_undegraded":"2024-03-26T06:21:48.719380+0000","last_fullsized":"2024-03-26T06:21:48.719380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:23:23.813878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.739654+0000","last_change":"2024-03-26T06:20:53.282790+0000","last_active":"2024-03-26T06:21:48.739654+0000","last_peered":"2024-03-26T06:21:48.739654+0000","last_clean":"2024-03-26T06:21:48.739654+0000","last_became_active":"2024-03-26T06:20:53.282451+0000","last_became_peered":"2024-03-26T06:20:53.282451+0000","last_unstale":"2024-03-26T06:21:48.739654+0000","last_undegraded":"2024-03-26T06:21:48.739654+0000","last_fullsized":"2024-03-26T06:21:48.739654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:47:56.328354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716073+0000","last_change":"2024-03-26T06:19:42.185355+0000","last_active":"2024-03-26T06:21:48.716073+0000","last_peered":"2024-03-26T06:21:48.716073+0000","last_clean":"2024-03-26T06:21:48.716073+0000","last_became_active":"2024-03-26T06:19:42.184672+0000","last_became_peered":"2024-03-26T06:19:42.184672+0000","last_unstale":"2024-03-26T06:21:48.716073+0000","last_undegraded":"2024-03-26T06:21:48.716073+0000","last_fullsized":"2024-03-26T06:21:48.716073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:16:17.700523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:16:17.700523+0000","last_clean_scrub_stamp":"2024-03-26T06:16:17.700523+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:14:25.637431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.049942+0000","last_change":"2024-03-26T06:20:54.295599+0000","last_active":"2024-03-26T06:21:49.049942+0000","last_peered":"2024-03-26T06:21:49.049942+0000","last_clean":"2024-03-26T06:21:49.049942+0000","last_became_active":"2024-03-26T06:20:54.295270+0000","last_became_peered":"2024-03-26T06:20:54.295270+0000","last_unstale":"2024-03-26T06:21:49.049942+0000","last_undegraded":"2024-03-26T06:21:49.049942+0000","last_fullsized":"2024-03-26T06:21:49.049942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:47.406455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716223+0000","last_change":"2024-03-26T06:20:53.284859+0000","last_active":"2024-03-26T06:21:48.716223+0000","last_peered":"2024-03-26T06:21:48.716223+0000","last_clean":"2024-03-26T06:21:48.716223+0000","last_became_active":"2024-03-26T06:20:53.283671+0000","last_became_peered":"2024-03-26T06:20:53.283671+0000","last_unstale":"2024-03-26T06:21:48.716223+0000","last_undegraded":"2024-03-26T06:21:48.716223+0000","last_fullsized":"2024-03-26T06:21:48.716223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:06:23.676648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.720068+0000","last_change":"2024-03-26T06:20:53.288792+0000","last_active":"2024-03-26T06:21:48.720068+0000","last_peered":"2024-03-26T06:21:48.720068+0000","last_clean":"2024-03-26T06:21:48.720068+0000","last_became_active":"2024-03-26T06:20:53.288463+0000","last_became_peered":"2024-03-26T06:20:53.288463+0000","last_unstale":"2024-03-26T06:21:48.720068+0000","last_undegraded":"2024-03-26T06:21:48.720068+0000","last_fullsized":"2024-03-26T06:21:48.720068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:55:55.014266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050305+0000","last_change":"2024-03-26T06:20:54.288478+0000","last_active":"2024-03-26T06:21:49.050305+0000","last_peered":"2024-03-26T06:21:49.050305+0000","last_clean":"2024-03-26T06:21:49.050305+0000","last_became_active":"2024-03-26T06:20:54.288014+0000","last_became_peered":"2024-03-26T06:20:54.288014+0000","last_unstale":"2024-03-26T06:21:49.050305+0000","last_undegraded":"2024-03-26T06:21:49.050305+0000","last_fullsized":"2024-03-26T06:21:49.050305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:36:16.252703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719283+0000","last_change":"2024-03-26T06:20:53.280055+0000","last_active":"2024-03-26T06:21:48.719283+0000","last_peered":"2024-03-26T06:21:48.719283+0000","last_clean":"2024-03-26T06:21:48.719283+0000","last_became_active":"2024-03-26T06:20:53.279751+0000","last_became_peered":"2024-03-26T06:20:53.279751+0000","last_unstale":"2024-03-26T06:21:48.719283+0000","last_undegraded":"2024-03-26T06:21:48.719283+0000","last_fullsized":"2024-03-26T06:21:48.719283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:32:48.407054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719942+0000","last_change":"2024-03-26T06:20:54.302001+0000","last_active":"2024-03-26T06:21:48.719942+0000","last_peered":"2024-03-26T06:21:48.719942+0000","last_clean":"2024-03-26T06:21:48.719942+0000","last_became_active":"2024-03-26T06:20:54.301659+0000","last_became_peered":"2024-03-26T06:20:54.301659+0000","last_unstale":"2024-03-26T06:21:48.719942+0000","last_undegraded":"2024-03-26T06:21:48.719942+0000","last_fullsized":"2024-03-26T06:21:48.719942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:38:08.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.740176+0000","last_change":"2024-03-26T06:20:53.285561+0000","last_active":"2024-03-26T06:21:48.740176+0000","last_peered":"2024-03-26T06:21:48.740176+0000","last_clean":"2024-03-26T06:21:48.740176+0000","last_became_active":"2024-03-26T06:20:53.285189+0000","last_became_peered":"2024-03-26T06:20:53.285189+0000","last_unstale":"2024-03-26T06:21:48.740176+0000","last_undegraded":"2024-03-26T06:21:48.740176+0000","last_fullsized":"2024-03-26T06:21:48.740176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:53:54.266188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715957+0000","last_change":"2024-03-26T06:20:54.300421+0000","last_active":"2024-03-26T06:21:48.715957+0000","last_peered":"2024-03-26T06:21:48.715957+0000","last_clean":"2024-03-26T06:21:48.715957+0000","last_became_active":"2024-03-26T06:20:54.299292+0000","last_became_peered":"2024-03-26T06:20:54.299292+0000","last_unstale":"2024-03-26T06:21:48.715957+0000","last_undegraded":"2024-03-26T06:21:48.715957+0000","last_fullsized":"2024-03-26T06:21:48.715957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:29:12.685903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.730662+0000","last_change":"2024-03-26T06:21:48.730662+0000","last_active":"2024-03-26T06:21:48.730662+0000","last_peered":"2024-03-26T06:21:48.730662+0000","last_clean":"2024-03-26T06:21:48.730662+0000","last_became_active":"2024-03-26T06:21:48.727946+0000","last_became_peered":"2024-03-26T06:21:48.727946+0000","last_unstale":"2024-03-26T06:21:48.730662+0000","last_undegraded":"2024-03-26T06:21:48.730662+0000","last_fullsized":"2024-03-26T06:21:48.730662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:01:50.526857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718480+0000","last_change":"2024-03-26T06:20:54.295448+0000","last_active":"2024-03-26T06:21:48.718480+0000","last_peered":"2024-03-26T06:21:48.718480+0000","last_clean":"2024-03-26T06:21:48.718480+0000","last_became_active":"2024-03-26T06:20:54.295116+0000","last_became_peered":"2024-03-26T06:20:54.295116+0000","last_unstale":"2024-03-26T06:21:48.718480+0000","last_undegraded":"2024-03-26T06:21:48.718480+0000","last_fullsized":"2024-03-26T06:21:48.718480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:22:07.474387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718967+0000","last_change":"2024-03-26T06:20:53.281317+0000","last_active":"2024-03-26T06:21:48.718967+0000","last_peered":"2024-03-26T06:21:48.718967+0000","last_clean":"2024-03-26T06:21:48.718967+0000","last_became_active":"2024-03-26T06:20:53.280979+0000","last_became_peered":"2024-03-26T06:20:53.280979+0000","last_unstale":"2024-03-26T06:21:48.718967+0000","last_undegraded":"2024-03-26T06:21:48.718967+0000","last_fullsized":"2024-03-26T06:21:48.718967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:02:29.850564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.722350+0000","last_change":"2024-03-26T06:20:54.288963+0000","last_active":"2024-03-26T06:21:48.722350+0000","last_peered":"2024-03-26T06:21:48.722350+0000","last_clean":"2024-03-26T06:21:48.722350+0000","last_became_active":"2024-03-26T06:20:54.288433+0000","last_became_peered":"2024-03-26T06:20:54.288433+0000","last_unstale":"2024-03-26T06:21:48.722350+0000","last_undegraded":"2024-03-26T06:21:48.722350+0000","last_fullsized":"2024-03-26T06:21:48.722350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:33.713471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718826+0000","last_change":"2024-03-26T06:20:53.279874+0000","last_active":"2024-03-26T06:21:48.718826+0000","last_peered":"2024-03-26T06:21:48.718826+0000","last_clean":"2024-03-26T06:21:48.718826+0000","last_became_active":"2024-03-26T06:20:53.279548+0000","last_became_peered":"2024-03-26T06:20:53.279548+0000","last_unstale":"2024-03-26T06:21:48.718826+0000","last_undegraded":"2024-03-26T06:21:48.718826+0000","last_fullsized":"2024-03-26T06:21:48.718826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:14.807650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718829+0000","last_change":"2024-03-26T06:20:54.305353+0000","last_active":"2024-03-26T06:21:48.718829+0000","last_peered":"2024-03-26T06:21:48.718829+0000","last_clean":"2024-03-26T06:21:48.718829+0000","last_became_active":"2024-03-26T06:20:54.304871+0000","last_became_peered":"2024-03-26T06:20:54.304871+0000","last_unstale":"2024-03-26T06:21:48.718829+0000","last_undegraded":"2024-03-26T06:21:48.718829+0000","last_fullsized":"2024-03-26T06:21:48.718829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:24:48.679669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719057+0000","last_change":"2024-03-26T06:20:53.286291+0000","last_active":"2024-03-26T06:21:48.719057+0000","last_peered":"2024-03-26T06:21:48.719057+0000","last_clean":"2024-03-26T06:21:48.719057+0000","last_became_active":"2024-03-26T06:20:53.285913+0000","last_became_peered":"2024-03-26T06:20:53.285913+0000","last_unstale":"2024-03-26T06:21:48.719057+0000","last_undegraded":"2024-03-26T06:21:48.719057+0000","last_fullsized":"2024-03-26T06:21:48.719057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:33.013108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716766+0000","last_change":"2024-03-26T06:20:54.302788+0000","last_active":"2024-03-26T06:21:48.716766+0000","last_peered":"2024-03-26T06:21:48.716766+0000","last_clean":"2024-03-26T06:21:48.716766+0000","last_became_active":"2024-03-26T06:20:54.302299+0000","last_became_peered":"2024-03-26T06:20:54.302299+0000","last_unstale":"2024-03-26T06:21:48.716766+0000","last_undegraded":"2024-03-26T06:21:48.716766+0000","last_fullsized":"2024-03-26T06:21:48.716766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:17:54.683294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:22:04.686293+0000","last_change":"2024-03-26T06:20:53.280496+0000","last_active":"2024-03-26T06:22:04.686293+0000","last_peered":"2024-03-26T06:22:04.686293+0000","last_clean":"2024-03-26T06:22:04.686293+0000","last_became_active":"2024-03-26T06:20:53.280178+0000","last_became_peered":"2024-03-26T06:20:53.280178+0000","last_unstale":"2024-03-26T06:22:04.686293+0000","last_undegraded":"2024-03-26T06:22:04.686293+0000","last_fullsized":"2024-03-26T06:22:04.686293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.095552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716495+0000","last_change":"2024-03-26T06:20:54.299702+0000","last_active":"2024-03-26T06:21:48.716495+0000","last_peered":"2024-03-26T06:21:48.716495+0000","last_clean":"2024-03-26T06:21:48.716495+0000","last_became_active":"2024-03-26T06:20:54.299069+0000","last_became_peered":"2024-03-26T06:20:54.299069+0000","last_unstale":"2024-03-26T06:21:48.716495+0000","last_undegraded":"2024-03-26T06:21:48.716495+0000","last_fullsized":"2024-03-26T06:21:48.716495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:20:56.019540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.724256+0000","last_change":"2024-03-26T06:20:53.284304+0000","last_active":"2024-03-26T06:21:48.724256+0000","last_peered":"2024-03-26T06:21:48.724256+0000","last_clean":"2024-03-26T06:21:48.724256+0000","last_became_active":"2024-03-26T06:20:53.283988+0000","last_became_peered":"2024-03-26T06:20:53.283988+0000","last_unstale":"2024-03-26T06:21:48.724256+0000","last_undegraded":"2024-03-26T06:21:48.724256+0000","last_fullsized":"2024-03-26T06:21:48.724256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:07:21.495880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716291+0000","last_change":"2024-03-26T06:20:54.305751+0000","last_active":"2024-03-26T06:21:48.716291+0000","last_peered":"2024-03-26T06:21:48.716291+0000","last_clean":"2024-03-26T06:21:48.716291+0000","last_became_active":"2024-03-26T06:20:54.303904+0000","last_became_peered":"2024-03-26T06:20:54.303904+0000","last_unstale":"2024-03-26T06:21:48.716291+0000","last_undegraded":"2024-03-26T06:21:48.716291+0000","last_fullsized":"2024-03-26T06:21:48.716291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:28:43.597279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050284+0000","last_change":"2024-03-26T06:20:53.283452+0000","last_active":"2024-03-26T06:21:49.050284+0000","last_peered":"2024-03-26T06:21:49.050284+0000","last_clean":"2024-03-26T06:21:49.050284+0000","last_became_active":"2024-03-26T06:20:53.282950+0000","last_became_peered":"2024-03-26T06:20:53.282950+0000","last_unstale":"2024-03-26T06:21:49.050284+0000","last_undegraded":"2024-03-26T06:21:49.050284+0000","last_fullsized":"2024-03-26T06:21:49.050284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:52.257524+0000","last_clean_scrub_stamp":"2024-03-26T06:20:52.257524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:59:15.161087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.716550+0000","last_change":"2024-03-26T06:20:54.300176+0000","last_active":"2024-03-26T06:21:48.716550+0000","last_peered":"2024-03-26T06:21:48.716550+0000","last_clean":"2024-03-26T06:21:48.716550+0000","last_became_active":"2024-03-26T06:20:54.299785+0000","last_became_peered":"2024-03-26T06:20:54.299785+0000","last_unstale":"2024-03-26T06:21:48.716550+0000","last_undegraded":"2024-03-26T06:21:48.716550+0000","last_fullsized":"2024-03-26T06:21:48.716550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:10.091612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.722461+0000","last_change":"2024-03-26T06:20:54.288774+0000","last_active":"2024-03-26T06:21:48.722461+0000","last_peered":"2024-03-26T06:21:48.722461+0000","last_clean":"2024-03-26T06:21:48.722461+0000","last_became_active":"2024-03-26T06:20:54.287986+0000","last_became_peered":"2024-03-26T06:20:54.287986+0000","last_unstale":"2024-03-26T06:21:48.722461+0000","last_undegraded":"2024-03-26T06:21:48.722461+0000","last_fullsized":"2024-03-26T06:21:48.722461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:49:04.418838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.051916+0000","last_change":"2024-03-26T06:21:49.051916+0000","last_active":"2024-03-26T06:21:49.051916+0000","last_peered":"2024-03-26T06:21:49.051916+0000","last_clean":"2024-03-26T06:21:49.051916+0000","last_became_active":"2024-03-26T06:21:49.051408+0000","last_became_peered":"2024-03-26T06:21:49.051408+0000","last_unstale":"2024-03-26T06:21:49.051916+0000","last_undegraded":"2024-03-26T06:21:49.051916+0000","last_fullsized":"2024-03-26T06:21:49.051916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:19:15.373133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718680+0000","last_change":"2024-03-26T06:20:54.287935+0000","last_active":"2024-03-26T06:21:48.718680+0000","last_peered":"2024-03-26T06:21:48.718680+0000","last_clean":"2024-03-26T06:21:48.718680+0000","last_became_active":"2024-03-26T06:20:54.287575+0000","last_became_peered":"2024-03-26T06:20:54.287575+0000","last_unstale":"2024-03-26T06:21:48.718680+0000","last_undegraded":"2024-03-26T06:21:48.718680+0000","last_fullsized":"2024-03-26T06:21:48.718680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:41:00.110749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.715793+0000","last_change":"2024-03-26T06:20:54.299188+0000","last_active":"2024-03-26T06:21:48.715793+0000","last_peered":"2024-03-26T06:21:48.715793+0000","last_clean":"2024-03-26T06:21:48.715793+0000","last_became_active":"2024-03-26T06:20:54.298822+0000","last_became_peered":"2024-03-26T06:20:54.298822+0000","last_unstale":"2024-03-26T06:21:48.715793+0000","last_undegraded":"2024-03-26T06:21:48.715793+0000","last_fullsized":"2024-03-26T06:21:48.715793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:03:17.694808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050274+0000","last_change":"2024-03-26T06:20:54.295974+0000","last_active":"2024-03-26T06:21:49.050274+0000","last_peered":"2024-03-26T06:21:49.050274+0000","last_clean":"2024-03-26T06:21:49.050274+0000","last_became_active":"2024-03-26T06:20:54.295111+0000","last_became_peered":"2024-03-26T06:20:54.295111+0000","last_unstale":"2024-03-26T06:21:49.050274+0000","last_undegraded":"2024-03-26T06:21:49.050274+0000","last_fullsized":"2024-03-26T06:21:49.050274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:53:45.126931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.721067+0000","last_change":"2024-03-26T06:20:54.282386+0000","last_active":"2024-03-26T06:21:48.721067+0000","last_peered":"2024-03-26T06:21:48.721067+0000","last_clean":"2024-03-26T06:21:48.721067+0000","last_became_active":"2024-03-26T06:20:54.282017+0000","last_became_peered":"2024-03-26T06:20:54.282017+0000","last_unstale":"2024-03-26T06:21:48.721067+0000","last_undegraded":"2024-03-26T06:21:48.721067+0000","last_fullsized":"2024-03-26T06:21:48.721067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:09:14.663629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719476+0000","last_change":"2024-03-26T06:20:54.297876+0000","last_active":"2024-03-26T06:21:48.719476+0000","last_peered":"2024-03-26T06:21:48.719476+0000","last_clean":"2024-03-26T06:21:48.719476+0000","last_became_active":"2024-03-26T06:20:54.297545+0000","last_became_peered":"2024-03-26T06:20:54.297545+0000","last_unstale":"2024-03-26T06:21:48.719476+0000","last_undegraded":"2024-03-26T06:21:48.719476+0000","last_fullsized":"2024-03-26T06:21:48.719476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:15:51.786210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.722722+0000","last_change":"2024-03-26T06:21:48.722722+0000","last_active":"2024-03-26T06:21:48.722722+0000","last_peered":"2024-03-26T06:21:48.722722+0000","last_clean":"2024-03-26T06:21:48.722722+0000","last_became_active":"2024-03-26T06:21:48.722062+0000","last_became_peered":"2024-03-26T06:21:48.722062+0000","last_unstale":"2024-03-26T06:21:48.722722+0000","last_undegraded":"2024-03-26T06:21:48.722722+0000","last_fullsized":"2024-03-26T06:21:48.722722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:01:44.959387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.723982+0000","last_change":"2024-03-26T06:20:54.289849+0000","last_active":"2024-03-26T06:21:48.723982+0000","last_peered":"2024-03-26T06:21:48.723982+0000","last_clean":"2024-03-26T06:21:48.723982+0000","last_became_active":"2024-03-26T06:20:54.287372+0000","last_became_peered":"2024-03-26T06:20:54.287372+0000","last_unstale":"2024-03-26T06:21:48.723982+0000","last_undegraded":"2024-03-26T06:21:48.723982+0000","last_fullsized":"2024-03-26T06:21:48.723982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:48.537975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.718835+0000","last_change":"2024-03-26T06:20:54.285737+0000","last_active":"2024-03-26T06:21:48.718835+0000","last_peered":"2024-03-26T06:21:48.718835+0000","last_clean":"2024-03-26T06:21:48.718835+0000","last_became_active":"2024-03-26T06:20:54.285423+0000","last_became_peered":"2024-03-26T06:20:54.285423+0000","last_unstale":"2024-03-26T06:21:48.718835+0000","last_undegraded":"2024-03-26T06:21:48.718835+0000","last_fullsized":"2024-03-26T06:21:48.718835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:55:59.095785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.719004+0000","last_change":"2024-03-26T06:20:54.301849+0000","last_active":"2024-03-26T06:21:48.719004+0000","last_peered":"2024-03-26T06:21:48.719004+0000","last_clean":"2024-03-26T06:21:48.719004+0000","last_became_active":"2024-03-26T06:20:54.301444+0000","last_became_peered":"2024-03-26T06:20:54.301444+0000","last_unstale":"2024-03-26T06:21:48.719004+0000","last_undegraded":"2024-03-26T06:21:48.719004+0000","last_fullsized":"2024-03-26T06:21:48.719004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:57:32.365781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:49.050384+0000","last_change":"2024-03-26T06:20:54.302979+0000","last_active":"2024-03-26T06:21:49.050384+0000","last_peered":"2024-03-26T06:21:49.050384+0000","last_clean":"2024-03-26T06:21:49.050384+0000","last_became_active":"2024-03-26T06:20:54.301428+0000","last_became_peered":"2024-03-26T06:20:54.301428+0000","last_unstale":"2024-03-26T06:21:49.050384+0000","last_undegraded":"2024-03-26T06:21:49.050384+0000","last_fullsized":"2024-03-26T06:21:49.050384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:49:29.519202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T06:21:48.725345+0000","last_change":"2024-03-26T06:21:48.725345+0000","last_active":"2024-03-26T06:21:48.725345+0000","last_peered":"2024-03-26T06:21:48.725345+0000","last_clean":"2024-03-26T06:21:48.725345+0000","last_became_active":"2024-03-26T06:21:48.724945+0000","last_became_peered":"2024-03-26T06:21:48.724945+0000","last_unstale":"2024-03-26T06:21:48.725345+0000","last_undegraded":"2024-03-26T06:21:48.725345+0000","last_fullsized":"2024-03-26T06:21:48.725345+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T06:20:53.261535+0000","last_clean_scrub_stamp":"2024-03-26T06:20:53.261535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:53:19.086435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65725,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":715,"num_read_kb":432,"num_write":233,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":197175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383707,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":457397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1650688,"data_stored":1107147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":456745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":456541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712440,"statfs":{"total":95995035648,"available":95961538560,"internally_reserved":0,"allocated":1568768,"data_stored":1049641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":516287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":978944,"data_stored":457801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":516507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":456287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1048928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T06:22:12.795 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-26T06:22:12.795 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-26T06:22:12.796 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-26T06:22:12.796 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-26T06:22:13.038 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:22:13.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:13 smithi037 ceph-mon[31974]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:22:13.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:13 smithi066 ceph-mon[36504]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:22:13.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:13 smithi149 ceph-mon[36681]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T06:22:14.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:14 smithi037 ceph-mon[31974]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-26T06:22:14.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:14 smithi066 ceph-mon[36504]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-26T06:22:14.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:14 smithi149 ceph-mon[36681]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-26T06:22:14.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.664+0000 7f1830782700 1 -- 172.21.15.37:0/800444098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828100330 msgr2=0x7f18281007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:14.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.664+0000 7f1830782700 1 --2- 172.21.15.37:0/800444098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828100330 0x7f18281007b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f181c0099c0 tx=0x7f181c009cd0 comp rx=0 tx=0).stop 2024-03-26T06:22:14.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 -- 172.21.15.37:0/800444098 shutdown_connections 2024-03-26T06:22:14.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 --2- 172.21.15.37:0/800444098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1828100cf0 0x7f182810ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:14.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 --2- 172.21.15.37:0/800444098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1828107f30 0x7f1828108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:14.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 --2- 172.21.15.37:0/800444098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828100330 0x7f18281007b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:14.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 -- 172.21.15.37:0/800444098 >> 172.21.15.37:0/800444098 conn(0x7f18280772b0 msgr2=0x7f18280fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:14.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 -- 172.21.15.37:0/800444098 shutdown_connections 2024-03-26T06:22:14.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 -- 172.21.15.37:0/800444098 wait complete. 2024-03-26T06:22:14.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 Processor -- start 2024-03-26T06:22:14.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 -- start start 2024-03-26T06:22:14.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1828100330 0x7f1828101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:14.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828100cf0 0x7f1828101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1828107f30 0x7f1828101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18281121a0 con 0x7f1828100cf0 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1828112020 con 0x7f1828107f30 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.665+0000 7f1830782700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1828112320 con 0x7f1828100330 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182dd1d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828100cf0 0x7f1828101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182dd1d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828100cf0 0x7f1828101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52802/0 (socket says 172.21.15.37:52802) 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182dd1d700 1 -- 172.21.15.37:0/551354000 learned_addr learned my addr 172.21.15.37:0/551354000 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182e51e700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1828100330 0x7f1828101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182ed1f700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1828107f30 0x7f1828101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182dd1d700 1 -- 172.21.15.37:0/551354000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1828100330 msgr2=0x7f1828101510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182dd1d700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1828100330 0x7f1828101510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:14.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182dd1d700 1 -- 172.21.15.37:0/551354000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1828107f30 msgr2=0x7f1828101f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182dd1d700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1828107f30 0x7f1828101f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182dd1d700 1 -- 172.21.15.37:0/551354000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18281a6bc0 con 0x7f1828100cf0 2024-03-26T06:22:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182ed1f700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1828107f30 0x7f1828101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f182dd1d700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828100cf0 0x7f1828101a50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f181c006010 tx=0x7f181c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f18277fe700 1 -- 172.21.15.37:0/551354000 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c027070 con 0x7f1828100cf0 2024-03-26T06:22:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f18277fe700 1 -- 172.21.15.37:0/551354000 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f181c003780 con 0x7f1828100cf0 2024-03-26T06:22:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f1830782700 1 -- 172.21.15.37:0/551354000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18281a6e50 con 0x7f1828100cf0 2024-03-26T06:22:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f18277fe700 1 -- 172.21.15.37:0/551354000 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c022da0 con 0x7f1828100cf0 2024-03-26T06:22:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.666+0000 7f1830782700 1 -- 172.21.15.37:0/551354000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18281a7240 con 0x7f1828100cf0 2024-03-26T06:22:14.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.667+0000 7f1830782700 1 -- 172.21.15.37:0/551354000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1828068aa0 con 0x7f1828100cf0 2024-03-26T06:22:14.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.669+0000 7f18277fe700 1 -- 172.21.15.37:0/551354000 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f181c02a070 con 0x7f1828100cf0 2024-03-26T06:22:14.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.669+0000 7f18277fe700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18100a2920 0x7f18100a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:14.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.670+0000 7f182e51e700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18100a2920 0x7f18100a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:14.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.670+0000 7f18277fe700 1 -- 172.21.15.37:0/551354000 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f181c0cfc20 con 0x7f1828100cf0 2024-03-26T06:22:14.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.670+0000 7f182e51e700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18100a2920 0x7f18100a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1820005950 tx=0x7f182000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:14.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.673+0000 7f18277fe700 1 -- 172.21.15.37:0/551354000 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f181c031080 con 0x7f1828100cf0 2024-03-26T06:22:14.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.922+0000 7f1830782700 1 -- 172.21.15.37:0/551354000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f182802d830 con 0x7f1828100cf0 2024-03-26T06:22:14.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.922+0000 7f18277fe700 1 -- 172.21.15.37:0/551354000 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f181c013070 con 0x7f1828100cf0 2024-03-26T06:22:14.924 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:14.924 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-26T06:22:14.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.924+0000 7f1830782700 1 -- 172.21.15.37:0/551354000 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18100a2920 msgr2=0x7f18100a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:14.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.924+0000 7f1830782700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18100a2920 0x7f18100a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1820005950 tx=0x7f182000b410 comp rx=0 tx=0).stop 2024-03-26T06:22:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.924+0000 7f1830782700 1 -- 172.21.15.37:0/551354000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828100cf0 msgr2=0x7f1828101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.924+0000 7f1830782700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828100cf0 0x7f1828101a50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f181c006010 tx=0x7f181c009f80 comp rx=0 tx=0).stop 2024-03-26T06:22:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.924+0000 7f1830782700 1 -- 172.21.15.37:0/551354000 shutdown_connections 2024-03-26T06:22:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.924+0000 7f1830782700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f18100a2920 0x7f18100a4de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.924+0000 7f1830782700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1828107f30 0x7f1828101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.924+0000 7f1830782700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1828100330 0x7f1828101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.924+0000 7f1830782700 1 --2- 172.21.15.37:0/551354000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1828100cf0 0x7f1828101a50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.924+0000 7f1830782700 1 -- 172.21.15.37:0/551354000 >> 172.21.15.37:0/551354000 conn(0x7f18280772b0 msgr2=0x7f1828104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.925+0000 7f1830782700 1 -- 172.21.15.37:0/551354000 shutdown_connections 2024-03-26T06:22:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:14.925+0000 7f1830782700 1 -- 172.21.15.37:0/551354000 wait complete. 2024-03-26T06:22:15.518 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:15 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/551354000' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T06:22:15.545 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-26T06:22:15.546 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:22:15.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:15 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/551354000' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T06:22:15.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.654+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/2897274373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef0101de0 msgr2=0x7f2ef0108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:15.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.654+0000 7f2ef6ab2640 1 --2- 172.21.15.37:0/2897274373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef0101de0 0x7f2ef0108670 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009ca0 tx=0x7f2ee802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.654+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/2897274373 shutdown_connections 2024-03-26T06:22:15.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.654+0000 7f2ef6ab2640 1 --2- 172.21.15.37:0/2897274373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef0101de0 0x7f2ef0108670 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.654+0000 7f2ef6ab2640 1 --2- 172.21.15.37:0/2897274373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ef0101420 0x7f2ef01018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.654+0000 7f2ef6ab2640 1 --2- 172.21.15.37:0/2897274373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef0100230 0x7f2ef0100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.654+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/2897274373 >> 172.21.15.37:0/2897274373 conn(0x7f2ef00fb7a0 msgr2=0x7f2ef00fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.654+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/2897274373 shutdown_connections 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.654+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/2897274373 wait complete. 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.654+0000 7f2ef6ab2640 1 Processor -- start 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.654+0000 7f2ef6ab2640 1 -- start start 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2ef6ab2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ef0100230 0x7f2ef01964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2ef6ab2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef0101420 0x7f2ef0196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2ef6ab2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef0101de0 0x7f2ef019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2ef6ab2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef010bec0 con 0x7f2ef0101420 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2ef6ab2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef010bd40 con 0x7f2ef0101de0 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2ef6ab2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef010c040 con 0x7f2ef0100230 2024-03-26T06:22:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eeffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef0101420 0x7f2ef0196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eeffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef0101420 0x7f2ef0196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52826/0 (socket says 172.21.15.37:52826) 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eeffff640 1 -- 172.21.15.37:0/1163989085 learned_addr learned my addr 172.21.15.37:0/1163989085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2ef504f640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef0101de0 0x7f2ef019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2ef484e640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ef0100230 0x7f2ef01964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eeffff640 1 -- 172.21.15.37:0/1163989085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ef0100230 msgr2=0x7f2ef01964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eeffff640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ef0100230 0x7f2ef01964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eeffff640 1 -- 172.21.15.37:0/1163989085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef0101de0 msgr2=0x7f2ef019daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eeffff640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef0101de0 0x7f2ef019daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eeffff640 1 -- 172.21.15.37:0/1163989085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef019e110 con 0x7f2ef0101420 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eeffff640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef0101420 0x7f2ef0196a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00c970 tx=0x7f2edc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eedffb640 1 -- 172.21.15.37:0/1163989085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc007bb0 con 0x7f2ef0101420 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eedffb640 1 -- 172.21.15.37:0/1163989085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2edc007d50 con 0x7f2ef0101420 2024-03-26T06:22:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/1163989085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef019e400 con 0x7f2ef0101420 2024-03-26T06:22:15.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.655+0000 7f2eedffb640 1 -- 172.21.15.37:0/1163989085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00f450 con 0x7f2ef0101420 2024-03-26T06:22:15.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.656+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/1163989085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef019e900 con 0x7f2ef0101420 2024-03-26T06:22:15.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:15 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/551354000' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T06:22:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.657+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/1163989085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef01074f0 con 0x7f2ef0101420 2024-03-26T06:22:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.657+0000 7f2eedffb640 1 -- 172.21.15.37:0/1163989085 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2edc005020 con 0x7f2ef0101420 2024-03-26T06:22:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.658+0000 7f2eedffb640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ee00a2890 0x7f2ee00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.658+0000 7f2ef484e640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ee00a2890 0x7f2ee00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:15.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.658+0000 7f2eedffb640 1 -- 172.21.15.37:0/1163989085 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2edc0cea10 con 0x7f2ef0101420 2024-03-26T06:22:15.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.658+0000 7f2ef484e640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ee00a2890 0x7f2ee00a4d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4005fd0 tx=0x7f2ee4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:15.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.663+0000 7f2eedffb640 1 -- 172.21.15.37:0/1163989085 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2edc014030 con 0x7f2ef0101420 2024-03-26T06:22:15.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.846+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/1163989085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ef002d020 con 0x7f2ef0101420 2024-03-26T06:22:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.848+0000 7f2eedffb640 1 -- 172.21.15.37:0/1163989085 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22580 (secure 0 0 0) 0x7f2edc09b900 con 0x7f2ef0101420 2024-03-26T06:22:15.849 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:15.849 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":87,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:21:48.711822+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:22:15.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/1163989085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ee00a2890 msgr2=0x7f2ee00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:15.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ee00a2890 0x7f2ee00a4d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4005fd0 tx=0x7f2ee4005850 comp rx=0 tx=0).stop 2024-03-26T06:22:15.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/1163989085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef0101420 msgr2=0x7f2ef0196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:15.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef0101420 0x7f2ef0196a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00c970 tx=0x7f2edc00ce30 comp rx=0 tx=0).stop 2024-03-26T06:22:15.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/1163989085 shutdown_connections 2024-03-26T06:22:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f2ee00a2890 0x7f2ee00a4d50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ef0101de0 0x7f2ef019daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef0101420 0x7f2ef0196a30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 --2- 172.21.15.37:0/1163989085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ef0100230 0x7f2ef01964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/1163989085 >> 172.21.15.37:0/1163989085 conn(0x7f2ef00fb7a0 msgr2=0x7f2ef01046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/1163989085 shutdown_connections 2024-03-26T06:22:15.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.850+0000 7f2ef6ab2640 1 -- 172.21.15.37:0/1163989085 wait complete. 2024-03-26T06:22:15.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T06:16:17.005303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T06:20:51.878592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T06:20:52.836733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T06:22:15.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T06:22:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.968+0000 7f4cf6420640 1 -- 172.21.15.37:0/421751173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf0106ff0 msgr2=0x7f4cf0109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.968+0000 7f4cf6420640 1 --2- 172.21.15.37:0/421751173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf0106ff0 0x7f4cf0109440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009ca0 tx=0x7f4ce802c100 comp rx=0 tx=0).stop 2024-03-26T06:22:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.968+0000 7f4cf6420640 1 -- 172.21.15.37:0/421751173 shutdown_connections 2024-03-26T06:22:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.968+0000 7f4cf6420640 1 --2- 172.21.15.37:0/421751173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf0106ff0 0x7f4cf0109440 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.968+0000 7f4cf6420640 1 --2- 172.21.15.37:0/421751173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cf01046c0 0x7f4cf0106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.968+0000 7f4cf6420640 1 --2- 172.21.15.37:0/421751173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cf0101d90 0x7f4cf0104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.968+0000 7f4cf6420640 1 -- 172.21.15.37:0/421751173 >> 172.21.15.37:0/421751173 conn(0x7f4cf00fb7a0 msgr2=0x7f4cf00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.968+0000 7f4cf6420640 1 -- 172.21.15.37:0/421751173 shutdown_connections 2024-03-26T06:22:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.968+0000 7f4cf6420640 1 -- 172.21.15.37:0/421751173 wait complete. 2024-03-26T06:22:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.968+0000 7f4cf6420640 1 Processor -- start 2024-03-26T06:22:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.968+0000 7f4cf6420640 1 -- start start 2024-03-26T06:22:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cf6420640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cf0101d90 0x7f4cf01974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cf6420640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf01046c0 0x7f4cf01979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cf6420640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cf0106ff0 0x7f4cf019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cf6420640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf010cfa0 con 0x7f4cf01046c0 2024-03-26T06:22:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cf6420640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf010ce20 con 0x7f4cf0101d90 2024-03-26T06:22:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cf6420640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf010d120 con 0x7f4cf0106ff0 2024-03-26T06:22:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf01046c0 0x7f4cf01979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf01046c0 0x7f4cf01979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52844/0 (socket says 172.21.15.37:52844) 2024-03-26T06:22:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cef7fe640 1 -- 172.21.15.37:0/296200295 learned_addr learned my addr 172.21.15.37:0/296200295 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cf49bd640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cf0106ff0 0x7f4cf019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4ceffff640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cf0101d90 0x7f4cf01974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cef7fe640 1 -- 172.21.15.37:0/296200295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cf0106ff0 msgr2=0x7f4cf019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cef7fe640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cf0106ff0 0x7f4cf019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cef7fe640 1 -- 172.21.15.37:0/296200295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cf0101d90 msgr2=0x7f4cf01974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cef7fe640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cf0101d90 0x7f4cf01974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cef7fe640 1 -- 172.21.15.37:0/296200295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf019f0d0 con 0x7f4cf01046c0 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cf49bd640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cf0106ff0 0x7f4cf019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4ceffff640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cf0101d90 0x7f4cf01974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cef7fe640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf01046c0 0x7f4cf01979f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400c970 tx=0x7f4ce400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4ced7fa640 1 -- 172.21.15.37:0/296200295 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4014070 con 0x7f4cf01046c0 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4ced7fa640 1 -- 172.21.15.37:0/296200295 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ce4004440 con 0x7f4cf01046c0 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cf6420640 1 -- 172.21.15.37:0/296200295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf019f3c0 con 0x7f4cf01046c0 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4ced7fa640 1 -- 172.21.15.37:0/296200295 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce400f450 con 0x7f4cf01046c0 2024-03-26T06:22:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.969+0000 7f4cf6420640 1 -- 172.21.15.37:0/296200295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf019f910 con 0x7f4cf01046c0 2024-03-26T06:22:15.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.970+0000 7f4cf6420640 1 -- 172.21.15.37:0/296200295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf00fd290 con 0x7f4cf01046c0 2024-03-26T06:22:15.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.972+0000 7f4ced7fa640 1 -- 172.21.15.37:0/296200295 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f4ce4017070 con 0x7f4cf01046c0 2024-03-26T06:22:15.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.972+0000 7f4ced7fa640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4cd80a2890 0x7f4cd80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:15.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.972+0000 7f4ceffff640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4cd80a2890 0x7f4cd80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:15.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.972+0000 7f4ced7fa640 1 -- 172.21.15.37:0/296200295 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4ce40cef00 con 0x7f4cf01046c0 2024-03-26T06:22:15.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.973+0000 7f4ceffff640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4cd80a2890 0x7f4cd80a4d50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000abe0 tx=0x7f4ce000a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:15.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:15.976+0000 7f4ced7fa640 1 -- 172.21.15.37:0/296200295 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4ce401e020 con 0x7f4cf01046c0 2024-03-26T06:22:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.157+0000 7f4cf6420640 1 -- 172.21.15.37:0/296200295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cf002d100 con 0x7f4cf01046c0 2024-03-26T06:22:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.157+0000 7f4ced7fa640 1 -- 172.21.15.37:0/296200295 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ce409be50 con 0x7f4cf01046c0 2024-03-26T06:22:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.159+0000 7f4cf6420640 1 -- 172.21.15.37:0/296200295 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4cd80a2890 msgr2=0x7f4cd80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.159+0000 7f4cf6420640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4cd80a2890 0x7f4cd80a4d50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000abe0 tx=0x7f4ce000a5c0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.159+0000 7f4cf6420640 1 -- 172.21.15.37:0/296200295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf01046c0 msgr2=0x7f4cf01979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.159+0000 7f4cf6420640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf01046c0 0x7f4cf01979f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400c970 tx=0x7f4ce400ce30 comp rx=0 tx=0).stop 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.159+0000 7f4cf6420640 1 -- 172.21.15.37:0/296200295 shutdown_connections 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.159+0000 7f4cf6420640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4cd80a2890 0x7f4cd80a4d50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.159+0000 7f4cf6420640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cf0106ff0 0x7f4cf019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.159+0000 7f4cf6420640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf01046c0 0x7f4cf01979f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.159+0000 7f4cf6420640 1 --2- 172.21.15.37:0/296200295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cf0101d90 0x7f4cf01974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.159+0000 7f4cf6420640 1 -- 172.21.15.37:0/296200295 >> 172.21.15.37:0/296200295 conn(0x7f4cf00fb7a0 msgr2=0x7f4cf01026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.160+0000 7f4cf6420640 1 -- 172.21.15.37:0/296200295 shutdown_connections 2024-03-26T06:22:16.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.160+0000 7f4cf6420640 1 -- 172.21.15.37:0/296200295 wait complete. 2024-03-26T06:22:16.172 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T06:22:16.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.277+0000 7fd5c7590640 1 -- 172.21.15.37:0/572056400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c0101da0 msgr2=0x7fd5c0108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.277+0000 7fd5c7590640 1 --2- 172.21.15.37:0/572056400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c0101da0 0x7fd5c0108630 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009ca0 tx=0x7fd5b002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.277+0000 7fd5c7590640 1 -- 172.21.15.37:0/572056400 shutdown_connections 2024-03-26T06:22:16.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.277+0000 7fd5c7590640 1 --2- 172.21.15.37:0/572056400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c0101da0 0x7fd5c0108630 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.277+0000 7fd5c7590640 1 --2- 172.21.15.37:0/572056400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c01013e0 0x7fd5c0101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.277+0000 7fd5c7590640 1 --2- 172.21.15.37:0/572056400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c01001f0 0x7fd5c0100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.277+0000 7fd5c7590640 1 -- 172.21.15.37:0/572056400 >> 172.21.15.37:0/572056400 conn(0x7fd5c00fb780 msgr2=0x7fd5c00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:16.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.277+0000 7fd5c7590640 1 -- 172.21.15.37:0/572056400 shutdown_connections 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.277+0000 7fd5c7590640 1 -- 172.21.15.37:0/572056400 wait complete. 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.277+0000 7fd5c7590640 1 Processor -- start 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.277+0000 7fd5c7590640 1 -- start start 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c7590640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c01001f0 0x7fd5c0104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c7590640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c01013e0 0x7fd5c01027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c7590640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0101da0 0x7fd5c0102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c7590640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c010be80 con 0x7fd5c01001f0 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c7590640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c010bd00 con 0x7fd5c0101da0 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c7590640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c010c000 con 0x7fd5c01013e0 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c532c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c01001f0 0x7fd5c0104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c532c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c01001f0 0x7fd5c0104180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52872/0 (socket says 172.21.15.37:52872) 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c532c640 1 -- 172.21.15.37:0/3608042005 learned_addr learned my addr 172.21.15.37:0/3608042005 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:16.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c532c640 1 -- 172.21.15.37:0/3608042005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c01013e0 msgr2=0x7fd5c01027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c532c640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c01013e0 0x7fd5c01027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c532c640 1 -- 172.21.15.37:0/3608042005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0101da0 msgr2=0x7fd5c0102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c5b2d640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0101da0 0x7fd5c0102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c532c640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0101da0 0x7fd5c0102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c532c640 1 -- 172.21.15.37:0/3608042005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c0103590 con 0x7fd5c01001f0 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c5b2d640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0101da0 0x7fd5c0102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c532c640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c01001f0 0x7fd5c0104180 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e9c0 tx=0x7fd5b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5be7fc640 1 -- 172.21.15.37:0/3608042005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800cd80 con 0x7fd5c01001f0 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5be7fc640 1 -- 172.21.15.37:0/3608042005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5b80035f0 con 0x7fd5c01001f0 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c7590640 1 -- 172.21.15.37:0/3608042005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c01a16e0 con 0x7fd5c01001f0 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5be7fc640 1 -- 172.21.15.37:0/3608042005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8010690 con 0x7fd5c01001f0 2024-03-26T06:22:16.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.278+0000 7fd5c7590640 1 -- 172.21.15.37:0/3608042005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c01a1c30 con 0x7fd5c01001f0 2024-03-26T06:22:16.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.279+0000 7fd5c7590640 1 -- 172.21.15.37:0/3608042005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c01a1f50 con 0x7fd5c01001f0 2024-03-26T06:22:16.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.281+0000 7fd5be7fc640 1 -- 172.21.15.37:0/3608042005 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fd5b80037a0 con 0x7fd5c01001f0 2024-03-26T06:22:16.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.281+0000 7fd5be7fc640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd5a80a2910 0x7fd5a80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.282+0000 7fd5c4b2b640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd5a80a2910 0x7fd5a80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:16.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.282+0000 7fd5be7fc640 1 -- 172.21.15.37:0/3608042005 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd5b80cf100 con 0x7fd5c01001f0 2024-03-26T06:22:16.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.282+0000 7fd5c4b2b640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd5a80a2910 0x7fd5a80a4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4005fd0 tx=0x7fd5b4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:16.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.285+0000 7fd5be7fc640 1 -- 172.21.15.37:0/3608042005 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd5b809c050 con 0x7fd5c01001f0 2024-03-26T06:22:16.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:16 smithi037 ceph-mon[31974]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:22:16.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:16 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1163989085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:16.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:16 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/296200295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:16.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.468+0000 7fd5c7590640 1 -- 172.21.15.37:0/3608042005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5c01a2120 con 0x7fd5c01001f0 2024-03-26T06:22:16.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.468+0000 7fd5be7fc640 1 -- 172.21.15.37:0/3608042005 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5b809b950 con 0x7fd5c01001f0 2024-03-26T06:22:16.469 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 -- 172.21.15.37:0/3608042005 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd5a80a2910 msgr2=0x7fd5a80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd5a80a2910 0x7fd5a80a4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4005fd0 tx=0x7fd5b4005850 comp rx=0 tx=0).stop 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 -- 172.21.15.37:0/3608042005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c01001f0 msgr2=0x7fd5c0104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c01001f0 0x7fd5c0104180 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800e9c0 tx=0x7fd5b800ee80 comp rx=0 tx=0).stop 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 -- 172.21.15.37:0/3608042005 shutdown_connections 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd5a80a2910 0x7fd5a80a4dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5c0101da0 0x7fd5c0102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5c01013e0 0x7fd5c01027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 --2- 172.21.15.37:0/3608042005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c01001f0 0x7fd5c0104180 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 -- 172.21.15.37:0/3608042005 >> 172.21.15.37:0/3608042005 conn(0x7fd5c00fb780 msgr2=0x7fd5c00fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 -- 172.21.15.37:0/3608042005 shutdown_connections 2024-03-26T06:22:16.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.470+0000 7fd5c7590640 1 -- 172.21.15.37:0/3608042005 wait complete. 2024-03-26T06:22:16.482 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T06:22:16.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.587+0000 7ff99e77e640 1 -- 172.21.15.37:0/554659345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff998101da0 msgr2=0x7ff998108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.587+0000 7ff99e77e640 1 --2- 172.21.15.37:0/554659345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff998101da0 0x7ff998108630 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff990009ca0 tx=0x7ff99002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.587+0000 7ff99e77e640 1 -- 172.21.15.37:0/554659345 shutdown_connections 2024-03-26T06:22:16.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.587+0000 7ff99e77e640 1 --2- 172.21.15.37:0/554659345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff998101da0 0x7ff998108630 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.587+0000 7ff99e77e640 1 --2- 172.21.15.37:0/554659345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9981013e0 0x7ff998101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.587+0000 7ff99e77e640 1 --2- 172.21.15.37:0/554659345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9981001f0 0x7ff998100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.587+0000 7ff99e77e640 1 -- 172.21.15.37:0/554659345 >> 172.21.15.37:0/554659345 conn(0x7ff9980fb7a0 msgr2=0x7ff9980fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.587+0000 7ff99e77e640 1 -- 172.21.15.37:0/554659345 shutdown_connections 2024-03-26T06:22:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.587+0000 7ff99e77e640 1 -- 172.21.15.37:0/554659345 wait complete. 2024-03-26T06:22:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.587+0000 7ff99e77e640 1 Processor -- start 2024-03-26T06:22:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.587+0000 7ff99e77e640 1 -- start start 2024-03-26T06:22:16.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99e77e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9981001f0 0x7ff9981964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99e77e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9981013e0 0x7ff9981969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99e77e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff998101da0 0x7ff99819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99e77e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99810be80 con 0x7ff998101da0 2024-03-26T06:22:16.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99e77e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99810bd00 con 0x7ff9981001f0 2024-03-26T06:22:16.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99e77e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99810c000 con 0x7ff9981013e0 2024-03-26T06:22:16.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99cd1b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff998101da0 0x7ff99819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:16.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99cd1b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff998101da0 0x7ff99819da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52882/0 (socket says 172.21.15.37:52882) 2024-03-26T06:22:16.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99cd1b640 1 -- 172.21.15.37:0/1140652854 learned_addr learned my addr 172.21.15.37:0/1140652854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff9977fe640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9981013e0 0x7ff9981969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff997fff640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9981001f0 0x7ff9981964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99cd1b640 1 -- 172.21.15.37:0/1140652854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9981013e0 msgr2=0x7ff9981969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99cd1b640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9981013e0 0x7ff9981969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99cd1b640 1 -- 172.21.15.37:0/1140652854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9981001f0 msgr2=0x7ff9981964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99cd1b640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9981001f0 0x7ff9981964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99cd1b640 1 -- 172.21.15.37:0/1140652854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99819e0c0 con 0x7ff998101da0 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff997fff640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9981001f0 0x7ff9981964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff9977fe640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9981013e0 0x7ff9981969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99cd1b640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff998101da0 0x7ff99819da50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff990037040 tx=0x7ff990032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff9957fa640 1 -- 172.21.15.37:0/1140652854 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990039040 con 0x7ff998101da0 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff9957fa640 1 -- 172.21.15.37:0/1140652854 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff990044070 con 0x7ff998101da0 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99e77e640 1 -- 172.21.15.37:0/1140652854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99819e350 con 0x7ff998101da0 2024-03-26T06:22:16.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff9957fa640 1 -- 172.21.15.37:0/1140652854 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990004330 con 0x7ff998101da0 2024-03-26T06:22:16.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.588+0000 7ff99e77e640 1 -- 172.21.15.37:0/1140652854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99819e890 con 0x7ff998101da0 2024-03-26T06:22:16.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.590+0000 7ff99e77e640 1 -- 172.21.15.37:0/1140652854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff998107510 con 0x7ff998101da0 2024-03-26T06:22:16.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.591+0000 7ff9957fa640 1 -- 172.21.15.37:0/1140652854 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff990003a00 con 0x7ff998101da0 2024-03-26T06:22:16.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.591+0000 7ff9957fa640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff9840a2910 0x7ff9840a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.591+0000 7ff997fff640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff9840a2910 0x7ff9840a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:16.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.591+0000 7ff9957fa640 1 -- 172.21.15.37:0/1140652854 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff9900f12f0 con 0x7ff998101da0 2024-03-26T06:22:16.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.592+0000 7ff997fff640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff9840a2910 0x7ff9840a4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9880058c0 tx=0x7ff988005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:16.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.596+0000 7ff9957fa640 1 -- 172.21.15.37:0/1140652854 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff9900be350 con 0x7ff998101da0 2024-03-26T06:22:16.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:16 smithi066 ceph-mon[36504]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:22:16.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:16 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1163989085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:16.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:16 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/296200295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:16.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:16 smithi149 ceph-mon[36681]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:22:16.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:16 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1163989085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:16.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:16 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/296200295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:16.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.807+0000 7ff99e77e640 1 -- 172.21.15.37:0/1140652854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff998061ef0 con 0x7ff998101da0 2024-03-26T06:22:16.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.807+0000 7ff9957fa640 1 -- 172.21.15.37:0/1140652854 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff9900bdc50 con 0x7ff998101da0 2024-03-26T06:22:16.809 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-26T06:22:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.809+0000 7ff99e77e640 1 -- 172.21.15.37:0/1140652854 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff9840a2910 msgr2=0x7ff9840a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.809+0000 7ff99e77e640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff9840a2910 0x7ff9840a4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9880058c0 tx=0x7ff988005d50 comp rx=0 tx=0).stop 2024-03-26T06:22:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.809+0000 7ff99e77e640 1 -- 172.21.15.37:0/1140652854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff998101da0 msgr2=0x7ff99819da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.809+0000 7ff99e77e640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff998101da0 0x7ff99819da50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff990037040 tx=0x7ff990032f90 comp rx=0 tx=0).stop 2024-03-26T06:22:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.810+0000 7ff99e77e640 1 -- 172.21.15.37:0/1140652854 shutdown_connections 2024-03-26T06:22:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.810+0000 7ff99e77e640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff9840a2910 0x7ff9840a4dd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.810+0000 7ff99e77e640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff998101da0 0x7ff99819da50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.810+0000 7ff99e77e640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9981013e0 0x7ff9981969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.810+0000 7ff99e77e640 1 --2- 172.21.15.37:0/1140652854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9981001f0 0x7ff9981964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.810+0000 7ff99e77e640 1 -- 172.21.15.37:0/1140652854 >> 172.21.15.37:0/1140652854 conn(0x7ff9980fb7a0 msgr2=0x7ff9981046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.810+0000 7ff99e77e640 1 -- 172.21.15.37:0/1140652854 shutdown_connections 2024-03-26T06:22:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.810+0000 7ff99e77e640 1 -- 172.21.15.37:0/1140652854 wait complete. 2024-03-26T06:22:16.822 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:22:16.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.923+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1532660887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64c4101400 msgr2=0x7f64c4101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.923+0000 7f64cbaf1640 1 --2- 172.21.15.37:0/1532660887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64c4101400 0x7f64c4101880 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009900 tx=0x7f64b402bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:16.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.923+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1532660887 shutdown_connections 2024-03-26T06:22:16.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.923+0000 7f64cbaf1640 1 --2- 172.21.15.37:0/1532660887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c4101dc0 0x7f64c4108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.923+0000 7f64cbaf1640 1 --2- 172.21.15.37:0/1532660887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64c4101400 0x7f64c4101880 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.923+0000 7f64cbaf1640 1 --2- 172.21.15.37:0/1532660887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c4100210 0x7f64c4100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.923+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1532660887 >> 172.21.15.37:0/1532660887 conn(0x7f64c40fb780 msgr2=0x7f64c40fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:16.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1532660887 shutdown_connections 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1532660887 wait complete. 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64cbaf1640 1 Processor -- start 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64cbaf1640 1 -- start start 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64cbaf1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64c4100210 0x7f64c41041a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64cbaf1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c4101400 0x7f64c41027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64cbaf1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c4101dc0 0x7f64c4102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64cbaf1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c410bea0 con 0x7f64c4100210 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64cbaf1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c410bd20 con 0x7f64c4101dc0 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64cbaf1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c410c020 con 0x7f64c4101400 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64c988d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64c4100210 0x7f64c41041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64c988d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64c4100210 0x7f64c41041a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52892/0 (socket says 172.21.15.37:52892) 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64c988d640 1 -- 172.21.15.37:0/1709902194 learned_addr learned my addr 172.21.15.37:0/1709902194 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.924+0000 7f64c908c640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c4101400 0x7f64c41027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:16.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64ca08e640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c4101dc0 0x7f64c4102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64c988d640 1 -- 172.21.15.37:0/1709902194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c4101400 msgr2=0x7f64c41027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64c988d640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c4101400 0x7f64c41027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64c988d640 1 -- 172.21.15.37:0/1709902194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c4101dc0 msgr2=0x7f64c4102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64c988d640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c4101dc0 0x7f64c4102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64c988d640 1 -- 172.21.15.37:0/1709902194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c41035b0 con 0x7f64c4100210 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64ca08e640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c4101dc0 0x7f64c4102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64c988d640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64c4100210 0x7f64c41041a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00d950 tx=0x7f64bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64baffd640 1 -- 172.21.15.37:0/1709902194 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc017070 con 0x7f64c4100210 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64baffd640 1 -- 172.21.15.37:0/1709902194 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64bc009c30 con 0x7f64c4100210 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1709902194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c41a1760 con 0x7f64c4100210 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64baffd640 1 -- 172.21.15.37:0/1709902194 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00f600 con 0x7f64c4100210 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.925+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1709902194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c41a1c80 con 0x7f64c4100210 2024-03-26T06:22:16.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.926+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1709902194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c41a1f40 con 0x7f64c4100210 2024-03-26T06:22:16.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.927+0000 7f64baffd640 1 -- 172.21.15.37:0/1709902194 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f64bc01a070 con 0x7f64c4100210 2024-03-26T06:22:16.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.928+0000 7f64baffd640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64b00a2910 0x7f64b00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:16.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.928+0000 7f64c908c640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64b00a2910 0x7f64b00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:16.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.928+0000 7f64baffd640 1 -- 172.21.15.37:0/1709902194 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f64bc0ce680 con 0x7f64c4100210 2024-03-26T06:22:16.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.928+0000 7f64c908c640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64b00a2910 0x7f64b00a4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f64b402c7b0 tx=0x7f64b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:16.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:16.931+0000 7f64baffd640 1 -- 172.21.15.37:0/1709902194 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f64bc009de0 con 0x7f64c4100210 2024-03-26T06:22:17.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.153+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1709902194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64c41a2110 con 0x7f64c4100210 2024-03-26T06:22:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.154+0000 7f64baffd640 1 -- 172.21.15.37:0/1709902194 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11385 (secure 0 0 0) 0x7f64bc09b5f0 con 0x7f64c4100210 2024-03-26T06:22:17.155 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:17.155 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T06:20:53.631953+0000","modified":"2024-03-26T06:21:00.587013+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24494,"mds_2":24470,"mds_3":24469,"mds_4":14682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14670":{"gid":14670,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/1944002831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1944002831},{"type":"v1","addr":"172.21.15.37:6839","nonce":1944002831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14682":{"gid":14682,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6841/933565564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":933565564},{"type":"v1","addr":"172.21.15.37:6841","nonce":933565564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24394":{"gid":24394,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/1420321367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1420321367},{"type":"v1","addr":"172.21.15.66:6839","nonce":1420321367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-26T06:22:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-03-26T06:22:17.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1709902194 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64b00a2910 msgr2=0x7f64b00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64b00a2910 0x7f64b00a4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f64b402c7b0 tx=0x7f64b4005fd0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1709902194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64c4100210 msgr2=0x7f64c41041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64c4100210 0x7f64c41041a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00d950 tx=0x7f64bc00de10 comp rx=0 tx=0).stop 2024-03-26T06:22:17.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1709902194 shutdown_connections 2024-03-26T06:22:17.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64b00a2910 0x7f64b00a4dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c4101dc0 0x7f64c4102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c4101400 0x7f64c41027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 --2- 172.21.15.37:0/1709902194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64c4100210 0x7f64c41041a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1709902194 >> 172.21.15.37:0/1709902194 conn(0x7f64c40fb780 msgr2=0x7f64c40fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:17.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1709902194 shutdown_connections 2024-03-26T06:22:17.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.156+0000 7f64cbaf1640 1 -- 172.21.15.37:0/1709902194 wait complete. 2024-03-26T06:22:17.168 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:22:17.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.273+0000 7f8f6866e640 1 -- 172.21.15.37:0/940709370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f60102ec0 msgr2=0x7f8f60109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.273+0000 7f8f6866e640 1 --2- 172.21.15.37:0/940709370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f60102ec0 0x7f8f60109750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009ca0 tx=0x7f8f5802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.273+0000 7f8f6866e640 1 -- 172.21.15.37:0/940709370 shutdown_connections 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.273+0000 7f8f6866e640 1 --2- 172.21.15.37:0/940709370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f60102ec0 0x7f8f60109750 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.273+0000 7f8f6866e640 1 --2- 172.21.15.37:0/940709370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f60102500 0x7f8f60102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.273+0000 7f8f6866e640 1 --2- 172.21.15.37:0/940709370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f60101310 0x7f8f60101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.273+0000 7f8f6866e640 1 -- 172.21.15.37:0/940709370 >> 172.21.15.37:0/940709370 conn(0x7f8f600fc8a0 msgr2=0x7f8f600fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.273+0000 7f8f6866e640 1 -- 172.21.15.37:0/940709370 shutdown_connections 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.273+0000 7f8f6866e640 1 -- 172.21.15.37:0/940709370 wait complete. 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f6866e640 1 Processor -- start 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f6866e640 1 -- start start 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f6866e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f60101310 0x7f8f60197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f6866e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f60102500 0x7f8f60197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f6866e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f60102ec0 0x7f8f6019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f6866e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6010cfa0 con 0x7f8f60102ec0 2024-03-26T06:22:17.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f6866e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6010ce20 con 0x7f8f60101310 2024-03-26T06:22:17.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f6866e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6010d120 con 0x7f8f60102500 2024-03-26T06:22:17.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f66c0b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f60102ec0 0x7f8f6019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:17.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f66c0b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f60102ec0 0x7f8f6019eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52902/0 (socket says 172.21.15.37:52902) 2024-03-26T06:22:17.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f66c0b640 1 -- 172.21.15.37:0/3935715013 learned_addr learned my addr 172.21.15.37:0/3935715013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:17.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f65c09640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f60102500 0x7f8f60197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:17.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f6640a640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f60101310 0x7f8f60197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:17.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f66c0b640 1 -- 172.21.15.37:0/3935715013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f60102500 msgr2=0x7f8f60197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f66c0b640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f60102500 0x7f8f60197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f66c0b640 1 -- 172.21.15.37:0/3935715013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f60101310 msgr2=0x7f8f60197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f66c0b640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f60101310 0x7f8f60197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f66c0b640 1 -- 172.21.15.37:0/3935715013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6019f1b0 con 0x7f8f60102ec0 2024-03-26T06:22:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f6640a640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f60101310 0x7f8f60197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.274+0000 7f8f66c0b640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f60102ec0 0x7f8f6019eb40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f58037040 tx=0x7f8f58032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.275+0000 7f8f577fe640 1 -- 172.21.15.37:0/3935715013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58041020 con 0x7f8f60102ec0 2024-03-26T06:22:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.275+0000 7f8f577fe640 1 -- 172.21.15.37:0/3935715013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f58034d90 con 0x7f8f60102ec0 2024-03-26T06:22:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.275+0000 7f8f6866e640 1 -- 172.21.15.37:0/3935715013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6019f440 con 0x7f8f60102ec0 2024-03-26T06:22:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.275+0000 7f8f577fe640 1 -- 172.21.15.37:0/3935715013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58039040 con 0x7f8f60102ec0 2024-03-26T06:22:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.275+0000 7f8f6866e640 1 -- 172.21.15.37:0/3935715013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6019f930 con 0x7f8f60102ec0 2024-03-26T06:22:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.276+0000 7f8f6866e640 1 -- 172.21.15.37:0/3935715013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f60108630 con 0x7f8f60102ec0 2024-03-26T06:22:17.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.277+0000 7f8f577fe640 1 -- 172.21.15.37:0/3935715013 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f8f58044020 con 0x7f8f60102ec0 2024-03-26T06:22:17.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.278+0000 7f8f577fe640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8f4c0a2890 0x7f8f4c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.278+0000 7f8f6640a640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8f4c0a2890 0x7f8f4c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:17.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.278+0000 7f8f577fe640 1 -- 172.21.15.37:0/3935715013 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8f580f0f10 con 0x7f8f60102ec0 2024-03-26T06:22:17.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.278+0000 7f8f6640a640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8f4c0a2890 0x7f8f4c0a4d50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c0058c0 tx=0x7f8f5c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:17.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.281+0000 7f8f577fe640 1 -- 172.21.15.37:0/3935715013 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8f5804b020 con 0x7f8f60102ec0 2024-03-26T06:22:17.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:17 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3608042005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:17.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:17 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1140652854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:17.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:17 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1709902194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:17.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.465+0000 7f8f6866e640 1 -- 172.21.15.37:0/3935715013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f60061ef0 con 0x7f8f60102ec0 2024-03-26T06:22:17.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.467+0000 7f8f577fe640 1 -- 172.21.15.37:0/3935715013 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22580 (secure 0 0 0) 0x7f8f580bdf10 con 0x7f8f60102ec0 2024-03-26T06:22:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:17.468 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":87,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:21:48.711822+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:22:17.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 -- 172.21.15.37:0/3935715013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8f4c0a2890 msgr2=0x7f8f4c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8f4c0a2890 0x7f8f4c0a4d50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c0058c0 tx=0x7f8f5c005850 comp rx=0 tx=0).stop 2024-03-26T06:22:17.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 -- 172.21.15.37:0/3935715013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f60102ec0 msgr2=0x7f8f6019eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f60102ec0 0x7f8f6019eb40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f58037040 tx=0x7f8f58032f20 comp rx=0 tx=0).stop 2024-03-26T06:22:17.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 -- 172.21.15.37:0/3935715013 shutdown_connections 2024-03-26T06:22:17.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8f4c0a2890 0x7f8f4c0a4d50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f60102ec0 0x7f8f6019eb40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f60102500 0x7f8f60197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 --2- 172.21.15.37:0/3935715013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f60101310 0x7f8f60197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 -- 172.21.15.37:0/3935715013 >> 172.21.15.37:0/3935715013 conn(0x7f8f600fc8a0 msgr2=0x7f8f60105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:17.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 -- 172.21.15.37:0/3935715013 shutdown_connections 2024-03-26T06:22:17.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.469+0000 7f8f6866e640 1 -- 172.21.15.37:0/3935715013 wait complete. 2024-03-26T06:22:17.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T06:16:17.005303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T06:20:51.878592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T06:20:52.836733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T06:22:17.482 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T06:22:17.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.586+0000 7f7cabfd4640 1 -- 172.21.15.37:0/2987664002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca4102eb0 msgr2=0x7f7ca4109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.586+0000 7f7cabfd4640 1 --2- 172.21.15.37:0/2987664002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca4102eb0 0x7f7ca4109740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009ca0 tx=0x7f7c9c02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.586+0000 7f7cabfd4640 1 -- 172.21.15.37:0/2987664002 shutdown_connections 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.586+0000 7f7cabfd4640 1 --2- 172.21.15.37:0/2987664002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca4102eb0 0x7f7ca4109740 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.586+0000 7f7cabfd4640 1 --2- 172.21.15.37:0/2987664002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca41024f0 0x7f7ca4102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.586+0000 7f7cabfd4640 1 --2- 172.21.15.37:0/2987664002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4101300 0x7f7ca4101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.586+0000 7f7cabfd4640 1 -- 172.21.15.37:0/2987664002 >> 172.21.15.37:0/2987664002 conn(0x7f7ca40fc890 msgr2=0x7f7ca40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7cabfd4640 1 -- 172.21.15.37:0/2987664002 shutdown_connections 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7cabfd4640 1 -- 172.21.15.37:0/2987664002 wait complete. 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7cabfd4640 1 Processor -- start 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7cabfd4640 1 -- start start 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7cabfd4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4101300 0x7f7ca4197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7cabfd4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca41024f0 0x7f7ca4197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7cabfd4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca4102eb0 0x7f7ca419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7cabfd4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca410cff0 con 0x7f7ca41024f0 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7cabfd4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca410ce70 con 0x7f7ca4102eb0 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7cabfd4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca410d170 con 0x7f7ca4101300 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7ca956f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca41024f0 0x7f7ca4197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7ca956f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca41024f0 0x7f7ca4197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52918/0 (socket says 172.21.15.37:52918) 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7ca956f640 1 -- 172.21.15.37:0/661597766 learned_addr learned my addr 172.21.15.37:0/661597766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.587+0000 7f7ca9d70640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4101300 0x7f7ca4197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.588+0000 7f7ca956f640 1 -- 172.21.15.37:0/661597766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4101300 msgr2=0x7f7ca4197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.588+0000 7f7ca956f640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4101300 0x7f7ca4197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.588+0000 7f7ca956f640 1 -- 172.21.15.37:0/661597766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca4102eb0 msgr2=0x7f7ca419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.588+0000 7f7ca956f640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca4102eb0 0x7f7ca419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.588+0000 7f7ca956f640 1 -- 172.21.15.37:0/661597766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca419f220 con 0x7f7ca41024f0 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.588+0000 7f7ca956f640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca41024f0 0x7f7ca4197b40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7c9400b4a0 tx=0x7f7c9400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.588+0000 7f7c9affd640 1 -- 172.21.15.37:0/661597766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c94014070 con 0x7f7ca41024f0 2024-03-26T06:22:17.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.588+0000 7f7c9affd640 1 -- 172.21.15.37:0/661597766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c94010af0 con 0x7f7ca41024f0 2024-03-26T06:22:17.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.588+0000 7f7cabfd4640 1 -- 172.21.15.37:0/661597766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca419f510 con 0x7f7ca41024f0 2024-03-26T06:22:17.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.588+0000 7f7c9affd640 1 -- 172.21.15.37:0/661597766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c94017d10 con 0x7f7ca41024f0 2024-03-26T06:22:17.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.588+0000 7f7cabfd4640 1 -- 172.21.15.37:0/661597766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca419fa60 con 0x7f7ca41024f0 2024-03-26T06:22:17.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.589+0000 7f7cabfd4640 1 -- 172.21.15.37:0/661597766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca41086f0 con 0x7f7ca41024f0 2024-03-26T06:22:17.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.590+0000 7f7c9affd640 1 -- 172.21.15.37:0/661597766 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f7c94017070 con 0x7f7ca41024f0 2024-03-26T06:22:17.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.590+0000 7f7c9affd640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7c900a2890 0x7f7c900a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.591+0000 7f7ca9d70640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7c900a2890 0x7f7c900a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:17.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.591+0000 7f7c9affd640 1 -- 172.21.15.37:0/661597766 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7c940cde40 con 0x7f7ca41024f0 2024-03-26T06:22:17.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.591+0000 7f7ca9d70640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7c900a2890 0x7f7c900a4d50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0005fd0 tx=0x7f7ca000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:17.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.595+0000 7f7c9affd640 1 -- 172.21.15.37:0/661597766 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7c9409aea0 con 0x7f7ca41024f0 2024-03-26T06:22:17.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:17 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3608042005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:17.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:17 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1140652854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:17.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:17 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1709902194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:17.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:17 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3608042005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:17.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:17 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1140652854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:17.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:17 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1709902194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:17.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.777+0000 7f7cabfd4640 1 -- 172.21.15.37:0/661597766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ca402d0e0 con 0x7f7ca41024f0 2024-03-26T06:22:17.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.777+0000 7f7c9affd640 1 -- 172.21.15.37:0/661597766 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c9409a7a0 con 0x7f7ca41024f0 2024-03-26T06:22:17.778 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 -- 172.21.15.37:0/661597766 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7c900a2890 msgr2=0x7f7c900a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7c900a2890 0x7f7c900a4d50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0005fd0 tx=0x7f7ca000b390 comp rx=0 tx=0).stop 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 -- 172.21.15.37:0/661597766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca41024f0 msgr2=0x7f7ca4197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca41024f0 0x7f7ca4197b40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7c9400b4a0 tx=0x7f7c9400b960 comp rx=0 tx=0).stop 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 -- 172.21.15.37:0/661597766 shutdown_connections 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f7c900a2890 0x7f7c900a4d50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca4102eb0 0x7f7ca419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca41024f0 0x7f7ca4197b40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 --2- 172.21.15.37:0/661597766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4101300 0x7f7ca4197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 -- 172.21.15.37:0/661597766 >> 172.21.15.37:0/661597766 conn(0x7f7ca40fc890 msgr2=0x7f7ca41058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 -- 172.21.15.37:0/661597766 shutdown_connections 2024-03-26T06:22:17.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.779+0000 7f7cabfd4640 1 -- 172.21.15.37:0/661597766 wait complete. 2024-03-26T06:22:17.792 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T06:22:17.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 -- 172.21.15.37:0/3254409847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4718101460 msgr2=0x7f47181018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 --2- 172.21.15.37:0/3254409847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4718101460 0x7f47181018e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4708009ca0 tx=0x7f470802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 -- 172.21.15.37:0/3254409847 shutdown_connections 2024-03-26T06:22:17.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 --2- 172.21.15.37:0/3254409847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4718101e20 0x7f47181086b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 --2- 172.21.15.37:0/3254409847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4718101460 0x7f47181018e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 --2- 172.21.15.37:0/3254409847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718100750 0x7f4718100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 -- 172.21.15.37:0/3254409847 >> 172.21.15.37:0/3254409847 conn(0x7f47180faf20 msgr2=0x7f47180fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:17.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 -- 172.21.15.37:0/3254409847 shutdown_connections 2024-03-26T06:22:17.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 -- 172.21.15.37:0/3254409847 wait complete. 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 Processor -- start 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 -- start start 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4718100750 0x7f47180ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718101460 0x7f471819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4718101e20 0x7f471819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471810bf30 con 0x7f4718101e20 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471810bdb0 con 0x7f4718100750 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471ead1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471810c0b0 con 0x7f4718101460 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.896+0000 7f471d06e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4718101e20 0x7f471819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f471d06e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4718101e20 0x7f471819e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59386/0 (socket says 172.21.15.37:59386) 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f471d06e640 1 -- 172.21.15.37:0/3619509067 learned_addr learned my addr 172.21.15.37:0/3619509067 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f4717fff640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718101460 0x7f471819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f471d06e640 1 -- 172.21.15.37:0/3619509067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718101460 msgr2=0x7f471819c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f471d06e640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718101460 0x7f471819c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f471d06e640 1 -- 172.21.15.37:0/3619509067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4718100750 msgr2=0x7f47180ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:17.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f471d06e640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4718100750 0x7f47180ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:17.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f471d06e640 1 -- 172.21.15.37:0/3619509067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471819f2f0 con 0x7f4718101e20 2024-03-26T06:22:17.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f471d06e640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4718101e20 0x7f471819e9e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4700009fd0 tx=0x7f470000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:17.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f4715ffb640 1 -- 172.21.15.37:0/3619509067 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700011830 con 0x7f4718101e20 2024-03-26T06:22:17.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f4715ffb640 1 -- 172.21.15.37:0/3619509067 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4700016070 con 0x7f4718101e20 2024-03-26T06:22:17.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f471ead1640 1 -- 172.21.15.37:0/3619509067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471819f580 con 0x7f4718101e20 2024-03-26T06:22:17.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f4715ffb640 1 -- 172.21.15.37:0/3619509067 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470000f2e0 con 0x7f4718101e20 2024-03-26T06:22:17.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.897+0000 7f471ead1640 1 -- 172.21.15.37:0/3619509067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471819fa50 con 0x7f4718101e20 2024-03-26T06:22:17.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.898+0000 7f471ead1640 1 -- 172.21.15.37:0/3619509067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4718107590 con 0x7f4718101e20 2024-03-26T06:22:17.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.899+0000 7f4715ffb640 1 -- 172.21.15.37:0/3619509067 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f47000119d0 con 0x7f4718101e20 2024-03-26T06:22:17.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.900+0000 7f4715ffb640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f47040a2890 0x7f47040a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:17.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.900+0000 7f4715ffb640 1 -- 172.21.15.37:0/3619509067 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f47000ce250 con 0x7f4718101e20 2024-03-26T06:22:17.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.900+0000 7f471c86d640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f47040a2890 0x7f47040a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:17.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.900+0000 7f471c86d640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f47040a2890 0x7f47040a4d50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4710005fd0 tx=0x7f471000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:17.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:17.904+0000 7f4715ffb640 1 -- 172.21.15.37:0/3619509067 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f470009b2b0 con 0x7f4718101e20 2024-03-26T06:22:18.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.086+0000 7f471ead1640 1 -- 172.21.15.37:0/3619509067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f471802d040 con 0x7f4718101e20 2024-03-26T06:22:18.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.086+0000 7f4715ffb640 1 -- 172.21.15.37:0/3619509067 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f470009abb0 con 0x7f4718101e20 2024-03-26T06:22:18.087 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-26T06:22:18.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 -- 172.21.15.37:0/3619509067 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f47040a2890 msgr2=0x7f47040a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f47040a2890 0x7f47040a4d50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4710005fd0 tx=0x7f471000b410 comp rx=0 tx=0).stop 2024-03-26T06:22:18.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 -- 172.21.15.37:0/3619509067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4718101e20 msgr2=0x7f471819e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4718101e20 0x7f471819e9e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4700009fd0 tx=0x7f470000b8e0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 -- 172.21.15.37:0/3619509067 shutdown_connections 2024-03-26T06:22:18.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f47040a2890 0x7f47040a4d50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4718101e20 0x7f471819e9e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718101460 0x7f471819c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 --2- 172.21.15.37:0/3619509067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4718100750 0x7f47180ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 -- 172.21.15.37:0/3619509067 >> 172.21.15.37:0/3619509067 conn(0x7f47180faf20 msgr2=0x7f4718104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 -- 172.21.15.37:0/3619509067 shutdown_connections 2024-03-26T06:22:18.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.088+0000 7f471ead1640 1 -- 172.21.15.37:0/3619509067 wait complete. 2024-03-26T06:22:18.101 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T06:22:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.205+0000 7fb10a261640 1 -- 172.21.15.37:0/3004287771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1040ff5d0 msgr2=0x7fb1040ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.205+0000 7fb10a261640 1 --2- 172.21.15.37:0/3004287771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1040ff5d0 0x7fb1040ff9f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009cd0 tx=0x7fb0f402c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.205+0000 7fb10a261640 1 -- 172.21.15.37:0/3004287771 shutdown_connections 2024-03-26T06:22:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.205+0000 7fb10a261640 1 --2- 172.21.15.37:0/3004287771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb104102f60 0x7fb1041097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.205+0000 7fb10a261640 1 --2- 172.21.15.37:0/3004287771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1041025a0 0x7fb104102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.205+0000 7fb10a261640 1 --2- 172.21.15.37:0/3004287771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1040ff5d0 0x7fb1040ff9f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.205+0000 7fb10a261640 1 -- 172.21.15.37:0/3004287771 >> 172.21.15.37:0/3004287771 conn(0x7fb1040faee0 msgr2=0x7fb1040fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.205+0000 7fb10a261640 1 -- 172.21.15.37:0/3004287771 shutdown_connections 2024-03-26T06:22:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.205+0000 7fb10a261640 1 -- 172.21.15.37:0/3004287771 wait complete. 2024-03-26T06:22:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.206+0000 7fb10a261640 1 Processor -- start 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.206+0000 7fb10a261640 1 -- start start 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.206+0000 7fb10a261640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1040ff5d0 0x7fb104197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.206+0000 7fb10a261640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1041025a0 0x7fb104197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.206+0000 7fb10a261640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb104102f60 0x7fb10419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.206+0000 7fb10a261640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10410cfc0 con 0x7fb1041025a0 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.206+0000 7fb10a261640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10410ce40 con 0x7fb104102f60 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.206+0000 7fb10a261640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10410d140 con 0x7fb1040ff5d0 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.206+0000 7fb102ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1041025a0 0x7fb104197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.206+0000 7fb102ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1041025a0 0x7fb104197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59410/0 (socket says 172.21.15.37:59410) 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.206+0000 7fb102ffd640 1 -- 172.21.15.37:0/3520630371 learned_addr learned my addr 172.21.15.37:0/3520630371 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb1037fe640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1040ff5d0 0x7fb104197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb102ffd640 1 -- 172.21.15.37:0/3520630371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1040ff5d0 msgr2=0x7fb104197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb102ffd640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1040ff5d0 0x7fb104197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb102ffd640 1 -- 172.21.15.37:0/3520630371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb104102f60 msgr2=0x7fb10419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb102ffd640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb104102f60 0x7fb10419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb102ffd640 1 -- 172.21.15.37:0/3520630371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10419f250 con 0x7fb1041025a0 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb1037fe640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1040ff5d0 0x7fb104197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb102ffd640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1041025a0 0x7fb104197b70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d950 tx=0x7fb0f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb100ff9640 1 -- 172.21.15.37:0/3520630371 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8017070 con 0x7fb1041025a0 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb10a261640 1 -- 172.21.15.37:0/3520630371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10419f540 con 0x7fb1041025a0 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb100ff9640 1 -- 172.21.15.37:0/3520630371 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0f8009c30 con 0x7fb1041025a0 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb100ff9640 1 -- 172.21.15.37:0/3520630371 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800f6b0 con 0x7fb1041025a0 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.207+0000 7fb10a261640 1 -- 172.21.15.37:0/3520630371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10419fa90 con 0x7fb1041025a0 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.208+0000 7fb10a261640 1 -- 172.21.15.37:0/3520630371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1041086d0 con 0x7fb1041025a0 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.209+0000 7fb100ff9640 1 -- 172.21.15.37:0/3520630371 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fb0f80109d0 con 0x7fb1041025a0 2024-03-26T06:22:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.210+0000 7fb100ff9640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb0ec0a2890 0x7fb0ec0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.210+0000 7fb1037fe640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb0ec0a2890 0x7fb0ec0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:18.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.210+0000 7fb100ff9640 1 -- 172.21.15.37:0/3520630371 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb0f80ce8c0 con 0x7fb1041025a0 2024-03-26T06:22:18.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.210+0000 7fb1037fe640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb0ec0a2890 0x7fb0ec0a4d50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4006010 tx=0x7fb0f4009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:18.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.214+0000 7fb100ff9640 1 -- 172.21.15.37:0/3520630371 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb0f8009de0 con 0x7fb1041025a0 2024-03-26T06:22:18.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.263+0000 7fb100ff9640 1 -- 172.21.15.37:0/3520630371 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb0f8097780 con 0x7fb1041025a0 2024-03-26T06:22:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.399+0000 7fb10a261640 1 -- 172.21.15.37:0/3520630371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb10402d090 con 0x7fb1041025a0 2024-03-26T06:22:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.399+0000 7fb100ff9640 1 -- 172.21.15.37:0/3520630371 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0f809b810 con 0x7fb1041025a0 2024-03-26T06:22:18.401 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-26T06:22:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 -- 172.21.15.37:0/3520630371 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb0ec0a2890 msgr2=0x7fb0ec0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb0ec0a2890 0x7fb0ec0a4d50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4006010 tx=0x7fb0f4009940 comp rx=0 tx=0).stop 2024-03-26T06:22:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 -- 172.21.15.37:0/3520630371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1041025a0 msgr2=0x7fb104197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1041025a0 0x7fb104197b70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d950 tx=0x7fb0f800de10 comp rx=0 tx=0).stop 2024-03-26T06:22:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 -- 172.21.15.37:0/3520630371 shutdown_connections 2024-03-26T06:22:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb0ec0a2890 0x7fb0ec0a4d50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb104102f60 0x7fb10419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1041025a0 0x7fb104197b70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 --2- 172.21.15.37:0/3520630371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1040ff5d0 0x7fb104197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 -- 172.21.15.37:0/3520630371 >> 172.21.15.37:0/3520630371 conn(0x7fb1040faee0 msgr2=0x7fb1041058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 -- 172.21.15.37:0/3520630371 shutdown_connections 2024-03-26T06:22:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.401+0000 7fb10a261640 1 -- 172.21.15.37:0/3520630371 wait complete. 2024-03-26T06:22:18.413 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:18 smithi037 ceph-mon[31974]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:18.413 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:18 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3935715013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:18.414 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:18 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/661597766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:18.414 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:18 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3619509067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:18.415 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:22:18.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 -- 172.21.15.37:0/4149718474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c100290 msgr2=0x7f622c1006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 --2- 172.21.15.37:0/4149718474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c100290 0x7f622c1006b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f621c009900 tx=0x7f621c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:18.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 -- 172.21.15.37:0/4149718474 shutdown_connections 2024-03-26T06:22:18.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 --2- 172.21.15.37:0/4149718474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f622c101e40 0x7f622c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 --2- 172.21.15.37:0/4149718474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f622c101480 0x7f622c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 --2- 172.21.15.37:0/4149718474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c100290 0x7f622c1006b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 -- 172.21.15.37:0/4149718474 >> 172.21.15.37:0/4149718474 conn(0x7f622c0fb820 msgr2=0x7f622c0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 -- 172.21.15.37:0/4149718474 shutdown_connections 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 -- 172.21.15.37:0/4149718474 wait complete. 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 Processor -- start 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 -- start start 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f622c100290 0x7f622c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c101480 0x7f622c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f622c101e40 0x7f622c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c10bef0 con 0x7f622c101480 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c10bd70 con 0x7f622c100290 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.517+0000 7f6231353640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c10c070 con 0x7f622c101e40 2024-03-26T06:22:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6223fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c101480 0x7f622c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6223fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c101480 0x7f622c19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59426/0 (socket says 172.21.15.37:59426) 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6223fff640 1 -- 172.21.15.37:0/1507215504 learned_addr learned my addr 172.21.15.37:0/1507215504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f622b7fe640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f622c101e40 0x7f622c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f622affd640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f622c100290 0x7f622c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6223fff640 1 -- 172.21.15.37:0/1507215504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f622c101e40 msgr2=0x7f622c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6223fff640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f622c101e40 0x7f622c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6223fff640 1 -- 172.21.15.37:0/1507215504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f622c100290 msgr2=0x7f622c06a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6223fff640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f622c100290 0x7f622c06a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6223fff640 1 -- 172.21.15.37:0/1507215504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c19f200 con 0x7f622c101480 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f622b7fe640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f622c101e40 0x7f622c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6223fff640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c101480 0x7f622c19c300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f621400c920 tx=0x7f621400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6228ff9640 1 -- 172.21.15.37:0/1507215504 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214007c70 con 0x7f622c101480 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6228ff9640 1 -- 172.21.15.37:0/1507215504 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6214007e10 con 0x7f622c101480 2024-03-26T06:22:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6228ff9640 1 -- 172.21.15.37:0/1507215504 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214018890 con 0x7f622c101480 2024-03-26T06:22:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6231353640 1 -- 172.21.15.37:0/1507215504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c19f4f0 con 0x7f622c101480 2024-03-26T06:22:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.518+0000 7f6231353640 1 -- 172.21.15.37:0/1507215504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c19fa40 con 0x7f622c101480 2024-03-26T06:22:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.519+0000 7f6231353640 1 -- 172.21.15.37:0/1507215504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c107550 con 0x7f622c101480 2024-03-26T06:22:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.520+0000 7f6228ff9640 1 -- 172.21.15.37:0/1507215504 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6214018a50 con 0x7f622c101480 2024-03-26T06:22:18.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.521+0000 7f6228ff9640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f62180a2800 0x7f62180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.521+0000 7f622affd640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f62180a2800 0x7f62180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:18.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.521+0000 7f6228ff9640 1 -- 172.21.15.37:0/1507215504 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f62140cef00 con 0x7f622c101480 2024-03-26T06:22:18.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.521+0000 7f622affd640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f62180a2800 0x7f62180a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f621c02c8e0 tx=0x7f621c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.524+0000 7f6228ff9640 1 -- 172.21.15.37:0/1507215504 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6214014030 con 0x7f622c101480 2024-03-26T06:22:18.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:18 smithi066 ceph-mon[36504]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:18.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:18 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3935715013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:18.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:18 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/661597766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:18.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:18 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3619509067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:18.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:18 smithi149 ceph-mon[36681]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:18.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:18 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3935715013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:18.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:18 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/661597766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:18.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:18 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3619509067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:18.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.747+0000 7f6231353640 1 -- 172.21.15.37:0/1507215504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f622c02d0c0 con 0x7f622c101480 2024-03-26T06:22:18.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.747+0000 7f6228ff9640 1 -- 172.21.15.37:0/1507215504 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11385 (secure 0 0 0) 0x7f621409c040 con 0x7f622c101480 2024-03-26T06:22:18.749 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:18.750 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T06:20:53.631953+0000","modified":"2024-03-26T06:21:00.587013+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24494,"mds_2":24470,"mds_3":24469,"mds_4":14682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14670":{"gid":14670,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/1944002831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1944002831},{"type":"v1","addr":"172.21.15.37:6839","nonce":1944002831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14682":{"gid":14682,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6841/933565564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":933565564},{"type":"v1","addr":"172.21.15.37:6841","nonce":933565564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24394":{"gid":24394,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/1420321367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1420321367},{"type":"v1","addr":"172.21.15.66:6839","nonce":1420321367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-26T06:22:18.750 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-03-26T06:22:18.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 -- 172.21.15.37:0/1507215504 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f62180a2800 msgr2=0x7f62180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f62180a2800 0x7f62180a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f621c02c8e0 tx=0x7f621c005fd0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 -- 172.21.15.37:0/1507215504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c101480 msgr2=0x7f622c19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c101480 0x7f622c19c300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f621400c920 tx=0x7f621400cde0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 -- 172.21.15.37:0/1507215504 shutdown_connections 2024-03-26T06:22:18.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f62180a2800 0x7f62180a4cc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f622c101e40 0x7f622c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c101480 0x7f622c19c300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 --2- 172.21.15.37:0/1507215504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f622c100290 0x7f622c06a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 -- 172.21.15.37:0/1507215504 >> 172.21.15.37:0/1507215504 conn(0x7f622c0fb820 msgr2=0x7f622c104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:18.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 -- 172.21.15.37:0/1507215504 shutdown_connections 2024-03-26T06:22:18.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.750+0000 7f6231353640 1 -- 172.21.15.37:0/1507215504 wait complete. 2024-03-26T06:22:18.762 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:22:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.864+0000 7fd8c6659640 1 -- 172.21.15.37:0/81710014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102500 msgr2=0x7fd8c0102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.864+0000 7fd8c6659640 1 --2- 172.21.15.37:0/81710014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102500 0x7fd8c0102980 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009900 tx=0x7fd8b402bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.864+0000 7fd8c6659640 1 -- 172.21.15.37:0/81710014 shutdown_connections 2024-03-26T06:22:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.864+0000 7fd8c6659640 1 --2- 172.21.15.37:0/81710014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8c0102ec0 0x7fd8c0109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.864+0000 7fd8c6659640 1 --2- 172.21.15.37:0/81710014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102500 0x7fd8c0102980 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.864+0000 7fd8c6659640 1 --2- 172.21.15.37:0/81710014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8c0101310 0x7fd8c0101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.864+0000 7fd8c6659640 1 -- 172.21.15.37:0/81710014 >> 172.21.15.37:0/81710014 conn(0x7fd8c00fc8a0 msgr2=0x7fd8c00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.864+0000 7fd8c6659640 1 -- 172.21.15.37:0/81710014 shutdown_connections 2024-03-26T06:22:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.864+0000 7fd8c6659640 1 -- 172.21.15.37:0/81710014 wait complete. 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8c6659640 1 Processor -- start 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8c6659640 1 -- start start 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8c6659640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8c0101310 0x7fd8c0197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8c6659640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102500 0x7fd8c0197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8c6659640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8c0102ec0 0x7fd8c019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8c6659640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c010cfe0 con 0x7fd8c0102500 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8c6659640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c010ce60 con 0x7fd8c0101310 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8c6659640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c010d160 con 0x7fd8c0102ec0 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8bf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102500 0x7fd8c0197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8bf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102500 0x7fd8c0197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59450/0 (socket says 172.21.15.37:59450) 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8bf7fe640 1 -- 172.21.15.37:0/1673027473 learned_addr learned my addr 172.21.15.37:0/1673027473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8bffff640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8c0101310 0x7fd8c0197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8c4bf6640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8c0102ec0 0x7fd8c019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8bf7fe640 1 -- 172.21.15.37:0/1673027473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8c0102ec0 msgr2=0x7fd8c019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8bf7fe640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8c0102ec0 0x7fd8c019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8bf7fe640 1 -- 172.21.15.37:0/1673027473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8c0101310 msgr2=0x7fd8c0197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8bf7fe640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8c0101310 0x7fd8c0197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8bf7fe640 1 -- 172.21.15.37:0/1673027473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c019f2b0 con 0x7fd8c0102500 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8c4bf6640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8c0102ec0 0x7fd8c019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.865+0000 7fd8bffff640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8c0101310 0x7fd8c0197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.866+0000 7fd8bf7fe640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102500 0x7fd8c0197bd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009230 tx=0x7fd8b4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.866+0000 7fd8bd7fa640 1 -- 172.21.15.37:0/1673027473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4041070 con 0x7fd8c0102500 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.866+0000 7fd8c6659640 1 -- 172.21.15.37:0/1673027473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c019f540 con 0x7fd8c0102500 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.866+0000 7fd8bd7fa640 1 -- 172.21.15.37:0/1673027473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8b40054e0 con 0x7fd8c0102500 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.866+0000 7fd8c6659640 1 -- 172.21.15.37:0/1673027473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c019fa80 con 0x7fd8c0102500 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.866+0000 7fd8bd7fa640 1 -- 172.21.15.37:0/1673027473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4034440 con 0x7fd8c0102500 2024-03-26T06:22:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.867+0000 7fd8c6659640 1 -- 172.21.15.37:0/1673027473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c0108630 con 0x7fd8c0102500 2024-03-26T06:22:18.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.868+0000 7fd8bd7fa640 1 -- 172.21.15.37:0/1673027473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd8b4044070 con 0x7fd8c0102500 2024-03-26T06:22:18.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.868+0000 7fd8bd7fa640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd8a80a2800 0x7fd8a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:18.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.869+0000 7fd8bd7fa640 1 -- 172.21.15.37:0/1673027473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd8b40f0d20 con 0x7fd8c0102500 2024-03-26T06:22:18.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.869+0000 7fd8bffff640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd8a80a2800 0x7fd8a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:18.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.869+0000 7fd8bffff640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd8a80a2800 0x7fd8a80a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00058c0 tx=0x7fd8b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:18.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:18.872+0000 7fd8bd7fa640 1 -- 172.21.15.37:0/1673027473 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd8b40bdd70 con 0x7fd8c0102500 2024-03-26T06:22:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.054+0000 7fd8c6659640 1 -- 172.21.15.37:0/1673027473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8c002cff0 con 0x7fd8c0102500 2024-03-26T06:22:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.056+0000 7fd8bd7fa640 1 -- 172.21.15.37:0/1673027473 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22580 (secure 0 0 0) 0x7fd8b40bd670 con 0x7fd8c0102500 2024-03-26T06:22:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:19.058 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":87,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:21:48.711822+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:22:19.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 -- 172.21.15.37:0/1673027473 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd8a80a2800 msgr2=0x7fd8a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd8a80a2800 0x7fd8a80a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00058c0 tx=0x7fd8b0005850 comp rx=0 tx=0).stop 2024-03-26T06:22:19.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 -- 172.21.15.37:0/1673027473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102500 msgr2=0x7fd8c0197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102500 0x7fd8c0197bd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009230 tx=0x7fd8b4005dd0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 -- 172.21.15.37:0/1673027473 shutdown_connections 2024-03-26T06:22:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd8a80a2800 0x7fd8a80a4cc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8c0102ec0 0x7fd8c019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102500 0x7fd8c0197bd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 --2- 172.21.15.37:0/1673027473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8c0101310 0x7fd8c0197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 -- 172.21.15.37:0/1673027473 >> 172.21.15.37:0/1673027473 conn(0x7fd8c00fc8a0 msgr2=0x7fd8c0105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 -- 172.21.15.37:0/1673027473 shutdown_connections 2024-03-26T06:22:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.058+0000 7fd8c6659640 1 -- 172.21.15.37:0/1673027473 wait complete. 2024-03-26T06:22:19.071 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:22:19.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.173+0000 7f279c959640 1 -- 172.21.15.37:0/3634637182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2798101dc0 msgr2=0x7f2798108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.173+0000 7f279c959640 1 --2- 172.21.15.37:0/3634637182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2798101dc0 0x7f2798108650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2788009ca0 tx=0x7f278802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.173+0000 7f279c959640 1 -- 172.21.15.37:0/3634637182 shutdown_connections 2024-03-26T06:22:19.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.173+0000 7f279c959640 1 --2- 172.21.15.37:0/3634637182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2798101dc0 0x7f2798108650 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.173+0000 7f279c959640 1 --2- 172.21.15.37:0/3634637182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2798101400 0x7f2798101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.173+0000 7f279c959640 1 --2- 172.21.15.37:0/3634637182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2798100210 0x7f2798100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279c959640 1 -- 172.21.15.37:0/3634637182 >> 172.21.15.37:0/3634637182 conn(0x7f27980fb780 msgr2=0x7f27980fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279c959640 1 -- 172.21.15.37:0/3634637182 shutdown_connections 2024-03-26T06:22:19.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279c959640 1 -- 172.21.15.37:0/3634637182 wait complete. 2024-03-26T06:22:19.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279c959640 1 Processor -- start 2024-03-26T06:22:19.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279c959640 1 -- start start 2024-03-26T06:22:19.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279c959640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2798100210 0x7f27981041a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279c959640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2798101400 0x7f27981027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279c959640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2798101dc0 0x7f2798102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279c959640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279810be70 con 0x7f2798100210 2024-03-26T06:22:19.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279c959640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279810bcf0 con 0x7f2798101dc0 2024-03-26T06:22:19.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279c959640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279810bff0 con 0x7f2798101400 2024-03-26T06:22:19.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2798100210 0x7f27981041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.174+0000 7f279659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2798100210 0x7f27981041a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59464/0 (socket says 172.21.15.37:59464) 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f279659c640 1 -- 172.21.15.37:0/1987315539 learned_addr learned my addr 172.21.15.37:0/1987315539 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f2795d9b640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2798101400 0x7f27981027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f2796d9d640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2798101dc0 0x7f2798102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f279659c640 1 -- 172.21.15.37:0/1987315539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2798101400 msgr2=0x7f27981027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f279659c640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2798101400 0x7f27981027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f279659c640 1 -- 172.21.15.37:0/1987315539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2798101dc0 msgr2=0x7f2798102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f279659c640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2798101dc0 0x7f2798102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f279659c640 1 -- 172.21.15.37:0/1987315539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27981035b0 con 0x7f2798100210 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f2796d9d640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2798101dc0 0x7f2798102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f279659c640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2798100210 0x7f27981041a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f279000d950 tx=0x7f279000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f278f7fe640 1 -- 172.21.15.37:0/1987315539 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790017020 con 0x7f2798100210 2024-03-26T06:22:19.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f279c959640 1 -- 172.21.15.37:0/1987315539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27981a16f0 con 0x7f2798100210 2024-03-26T06:22:19.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f278f7fe640 1 -- 172.21.15.37:0/1987315539 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2790009c30 con 0x7f2798100210 2024-03-26T06:22:19.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f278f7fe640 1 -- 172.21.15.37:0/1987315539 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000f740 con 0x7f2798100210 2024-03-26T06:22:19.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.175+0000 7f279c959640 1 -- 172.21.15.37:0/1987315539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27981a1c90 con 0x7f2798100210 2024-03-26T06:22:19.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.176+0000 7f279c959640 1 -- 172.21.15.37:0/1987315539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27981a1fb0 con 0x7f2798100210 2024-03-26T06:22:19.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.178+0000 7f278f7fe640 1 -- 172.21.15.37:0/1987315539 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f279001a020 con 0x7f2798100210 2024-03-26T06:22:19.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.178+0000 7f278f7fe640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f27840a2800 0x7f27840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.179+0000 7f2795d9b640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f27840a2800 0x7f27840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.179+0000 7f278f7fe640 1 -- 172.21.15.37:0/1987315539 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f27900ce8c0 con 0x7f2798100210 2024-03-26T06:22:19.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.179+0000 7f2795d9b640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f27840a2800 0x7f27840a4cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2798103f10 tx=0x7f278000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.182+0000 7f278f7fe640 1 -- 172.21.15.37:0/1987315539 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2790021070 con 0x7f2798100210 2024-03-26T06:22:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.403+0000 7f279c959640 1 -- 172.21.15.37:0/1987315539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27981a2180 con 0x7f2798100210 2024-03-26T06:22:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.404+0000 7f278f7fe640 1 -- 172.21.15.37:0/1987315539 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11385 (secure 0 0 0) 0x7f2790009de0 con 0x7f2798100210 2024-03-26T06:22:19.405 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:19.405 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T06:20:53.631953+0000","modified":"2024-03-26T06:21:00.587013+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24494,"mds_2":24470,"mds_3":24469,"mds_4":14682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14670":{"gid":14670,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/1944002831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1944002831},{"type":"v1","addr":"172.21.15.37:6839","nonce":1944002831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14682":{"gid":14682,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6841/933565564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":933565564},{"type":"v1","addr":"172.21.15.37:6841","nonce":933565564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24394":{"gid":24394,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/1420321367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1420321367},{"type":"v1","addr":"172.21.15.66:6839","nonce":1420321367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-26T06:22:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-03-26T06:22:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 -- 172.21.15.37:0/1987315539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f27840a2800 msgr2=0x7f27840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f27840a2800 0x7f27840a4cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2798103f10 tx=0x7f278000b410 comp rx=0 tx=0).stop 2024-03-26T06:22:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 -- 172.21.15.37:0/1987315539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2798100210 msgr2=0x7f27981041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2798100210 0x7f27981041a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f279000d950 tx=0x7f279000de10 comp rx=0 tx=0).stop 2024-03-26T06:22:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 -- 172.21.15.37:0/1987315539 shutdown_connections 2024-03-26T06:22:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f27840a2800 0x7f27840a4cc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2798101dc0 0x7f2798102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2798101400 0x7f27981027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 --2- 172.21.15.37:0/1987315539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2798100210 0x7f27981041a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 -- 172.21.15.37:0/1987315539 >> 172.21.15.37:0/1987315539 conn(0x7f27980fb780 msgr2=0x7f27980fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 -- 172.21.15.37:0/1987315539 shutdown_connections 2024-03-26T06:22:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.406+0000 7f279c959640 1 -- 172.21.15.37:0/1987315539 wait complete. 2024-03-26T06:22:19.417 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:19 smithi037 ceph-mon[31974]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T06:22:19.417 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:19 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3520630371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:19.417 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:19 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1507215504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:19.418 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:19 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1673027473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:19.419 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T06:20:53.631953+0000', 'modified': '2024-03-26T06:21:00.587013+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24545, 'mds_1': 24494, 'mds_2': 24470, 'mds_3': 24469, 'mds_4': 14682}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14670': {'gid': 14670, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.37:6839/1944002831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6838', 'nonce': 1944002831}, {'type': 'v1', 'addr': '172.21.15.37:6839', 'nonce': 1944002831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14682': {'gid': 14682, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.37:6841/933565564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6840', 'nonce': 933565564}, {'type': 'v1', 'addr': '172.21.15.37:6841', 'nonce': 933565564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24394': {'gid': 24394, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6833/2169208399', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 2169208399}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 2169208399}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6833/2811553067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 2811553067}, {'type': 'v1', 'addr': '172.21.15.66:6833', 'nonce': 2811553067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/1494796094', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 1494796094}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 1494796094}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/801999981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 801999981}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 801999981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.149:6839/1810727373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 1810727373}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 1810727373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24470': {'gid': 24470, 'name': 'e', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.66:6835/4186424754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 4186424754}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 4186424754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.66:6837/3870845253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 3870845253}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 3870845253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6839/1420321367', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 1420321367}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 1420321367}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-26T06:22:19.419 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-26T06:22:19.420 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-26T06:22:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.522+0000 7f7991073640 1 -- 172.21.15.37:0/416247214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c102510 msgr2=0x7f798c102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.522+0000 7f7991073640 1 --2- 172.21.15.37:0/416247214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c102510 0x7f798c102990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7980009900 tx=0x7f798002bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.522+0000 7f7991073640 1 -- 172.21.15.37:0/416247214 shutdown_connections 2024-03-26T06:22:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.522+0000 7f7991073640 1 --2- 172.21.15.37:0/416247214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f798c102ed0 0x7f798c109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.522+0000 7f7991073640 1 --2- 172.21.15.37:0/416247214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c102510 0x7f798c102990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.522+0000 7f7991073640 1 --2- 172.21.15.37:0/416247214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f798c101320 0x7f798c101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.522+0000 7f7991073640 1 -- 172.21.15.37:0/416247214 >> 172.21.15.37:0/416247214 conn(0x7f798c0fc890 msgr2=0x7f798c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.522+0000 7f7991073640 1 -- 172.21.15.37:0/416247214 shutdown_connections 2024-03-26T06:22:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.522+0000 7f7991073640 1 -- 172.21.15.37:0/416247214 wait complete. 2024-03-26T06:22:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.523+0000 7f7991073640 1 Processor -- start 2024-03-26T06:22:19.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.523+0000 7f7991073640 1 -- start start 2024-03-26T06:22:19.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.523+0000 7f7991073640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c101320 0x7f798c0750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.523+0000 7f7991073640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f798c102510 0x7f798c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.523+0000 7f7991073640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f798c102ed0 0x7f798c073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.523+0000 7f7991073640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798c10d000 con 0x7f798c101320 2024-03-26T06:22:19.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.523+0000 7f7991073640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798c10ce80 con 0x7f798c102510 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.523+0000 7f7991073640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798c10d180 con 0x7f798c102ed0 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.523+0000 7f798ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c101320 0x7f798c0750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.523+0000 7f798ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c101320 0x7f798c0750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59486/0 (socket says 172.21.15.37:59486) 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f798ad9d640 1 -- 172.21.15.37:0/3728126753 learned_addr learned my addr 172.21.15.37:0/3728126753 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f798b59e640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f798c102ed0 0x7f798c073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f798a59c640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f798c102510 0x7f798c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f798ad9d640 1 -- 172.21.15.37:0/3728126753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f798c102ed0 msgr2=0x7f798c073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f798ad9d640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f798c102ed0 0x7f798c073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f798ad9d640 1 -- 172.21.15.37:0/3728126753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f798c102510 msgr2=0x7f798c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f798ad9d640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f798c102510 0x7f798c0736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f798ad9d640 1 -- 172.21.15.37:0/3728126753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c0744b0 con 0x7f798c101320 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f798ad9d640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c101320 0x7f798c0750a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f797c00c970 tx=0x7f797c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f7973fff640 1 -- 172.21.15.37:0/3728126753 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c007c10 con 0x7f798c101320 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f7991073640 1 -- 172.21.15.37:0/3728126753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798c1a1700 con 0x7f798c101320 2024-03-26T06:22:19.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f7973fff640 1 -- 172.21.15.37:0/3728126753 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f797c007db0 con 0x7f798c101320 2024-03-26T06:22:19.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f7973fff640 1 -- 172.21.15.37:0/3728126753 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c00f8a0 con 0x7f798c101320 2024-03-26T06:22:19.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.524+0000 7f7991073640 1 -- 172.21.15.37:0/3728126753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798c1a1ca0 con 0x7f798c101320 2024-03-26T06:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.526+0000 7f7973fff640 1 -- 172.21.15.37:0/3728126753 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f797c00fa60 con 0x7f798c101320 2024-03-26T06:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f7993c52740 1 Processor -- start 2024-03-26T06:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f7993c52740 1 -- start start 2024-03-26T06:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f7973fff640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f79740a2800 0x7f79740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f7993c52740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148d0e0 0x55a71148d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f7993c52740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148dad0 0x55a7114925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f7993c52740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f7993c52740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7113f27c0 con 0x55a711492c40 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f7993c52740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71139bb50 con 0x55a71148dad0 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f7993c52740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7113ee860 con 0x55a71148d0e0 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798b59e640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f79740a2800 0x7f79740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f7973fff640 1 -- 172.21.15.37:0/3728126753 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f797c0cee70 con 0x7f798c101320 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114950d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59494/0 (socket says 172.21.15.37:59494) 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798b59e640 1 -- 172.21.15.37:0/1478031385 learned_addr learned my addr 172.21.15.37:0/1478031385 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798a59c640 1 --2- 172.21.15.37:0/1478031385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148dad0 0x55a7114925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798ad9d640 1 --2- 172.21.15.37:0/1478031385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148d0e0 0x55a71148d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798b59e640 1 -- 172.21.15.37:0/1478031385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148d0e0 msgr2=0x55a71148d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798b59e640 1 --2- 172.21.15.37:0/1478031385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148d0e0 0x55a71148d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798b59e640 1 -- 172.21.15.37:0/1478031385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148dad0 msgr2=0x55a7114925d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798b59e640 1 --2- 172.21.15.37:0/1478031385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148dad0 0x55a7114925d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798b59e640 1 -- 172.21.15.37:0/1478031385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7113af9a0 con 0x55a711492c40 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.527+0000 7f798a59c640 1 --2- 172.21.15.37:0/1478031385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148dad0 0x55a7114925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.528+0000 7f798b59e640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f79740a2800 0x7f79740a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7984014c60 tx=0x7f798400ae60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.528+0000 7f798b59e640 1 --2- 172.21.15.37:0/1478031385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114950d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7984011c40 tx=0x7f798401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eabd9a34e0bf4e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.528+0000 7f79717fa640 1 -- 172.21.15.37:0/1478031385 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984022070 con 0x55a711492c40 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.528+0000 7f79717fa640 1 -- 172.21.15.37:0/1478031385 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7984003e50 con 0x55a711492c40 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.528+0000 7f79717fa640 1 -- 172.21.15.37:0/1478031385 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79840044e0 con 0x55a711492c40 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.528+0000 7f7993c52740 1 -- 172.21.15.37:0/1478031385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 msgr2=0x55a7114950d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.528+0000 7f7993c52740 1 --2- 172.21.15.37:0/1478031385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114950d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7984011c40 tx=0x7f798401b000 comp rx=0 tx=0).stop 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 -- 172.21.15.37:0/1478031385 shutdown_connections 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 --2- 172.21.15.37:0/1478031385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114950d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 --2- 172.21.15.37:0/1478031385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148dad0 0x55a7114925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 --2- 172.21.15.37:0/1478031385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148d0e0 0x55a71148d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 -- 172.21.15.37:0/1478031385 >> 172.21.15.37:0/1478031385 conn(0x55a711455400 msgr2=0x55a711488a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 -- 172.21.15.37:0/1478031385 shutdown_connections 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 -- 172.21.15.37:0/1478031385 wait complete. 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 -- start start 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 Processor -- start 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148d0e0 0x55a7114975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148dad0 0x55a711497af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a711495b90 con 0x55a711492c40 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7113aef90 con 0x55a71148d0e0 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a711495d10 con 0x55a71148dad0 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f7993c52740 10 client.0 _ll_register_callbacks cb 0x55a7113ea400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f798b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f798b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59508/0 (socket says 172.21.15.37:59508) 2024-03-26T06:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f798b59e640 1 -- 172.21.15.37:0/1292067383 learned_addr learned my addr 172.21.15.37:0/1292067383 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f798ad9d640 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148d0e0 0x55a7114975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.529+0000 7f798a59c640 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148dad0 0x55a711497af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f798b59e640 1 -- 172.21.15.37:0/1292067383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148dad0 msgr2=0x55a711497af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f798b59e640 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148dad0 0x55a711497af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f798b59e640 1 -- 172.21.15.37:0/1292067383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148d0e0 msgr2=0x55a7114975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f798b59e640 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148d0e0 0x55a7114975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f798b59e640 1 -- 172.21.15.37:0/1292067383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7111072d0 con 0x55a711492c40 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f798a59c640 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148dad0 0x55a711497af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f798b59e640 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149eb30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7984011d10 tx=0x7f7984011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55af0cc7d825684d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f796eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f796eff5640 1 -- 172.21.15.37:0/1292067383 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984022040 con 0x55a711492c40 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f796eff5640 1 -- 172.21.15.37:0/1292067383 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7984004c00 con 0x55a711492c40 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f7993c52740 1 -- 172.21.15.37:0/1292067383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7984003e50 con 0x55a711492c40 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f796eff5640 1 -- 172.21.15.37:0/1292067383 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7984004260 con 0x55a711492c40 2024-03-26T06:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f7993c52740 10 client.15099 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f7993c52740 1 -- 172.21.15.37:0/1292067383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a7111d16b0 con 0x55a711492c40 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f796eff5640 1 -- 172.21.15.37:0/1292067383 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7984003e50 con 0x55a711492c40 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f796eff5640 1 -- 172.21.15.37:0/1292067383 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f798402d020 con 0x55a711492c40 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f7993c52740 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f7993c52740 10 client.15099 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f7993c52740 20 client.15099 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f7993c52740 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55a7114a3e20 0x55a7114a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f7993c52740 4 client.15099 mds_command: new command op to 14682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.530+0000 7f7993c52740 1 -- 172.21.15.37:0/1292067383 --> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7113ffd00 con 0x55a7114a3e20 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.531+0000 7f798ad9d640 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55a7114a3e20 0x55a7114a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.531+0000 7f796eff5640 10 client.15099 ms_handle_connect on v2:172.21.15.37:6840/933565564 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.531+0000 7f798ad9d640 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55a7114a3e20 0x55a7114a30e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aae9e39817154e71 server_cookie=3757874149845980 in_seq=0 out_seq=0 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.531+0000 7f796eff5640 1 -- 172.21.15.37:0/1292067383 <== mds.4 v2:172.21.15.37:6840/933565564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a7113ffd00 con 0x55a7114a3e20 2024-03-26T06:22:19.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.531+0000 7f796eff5640 10 client.15099 handle_command_reply: tid=0 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.531+0000 7f7993c52740 1 client.15099 shutdown 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.531+0000 7f7993c52740 1 -- 172.21.15.37:0/1292067383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 msgr2=0x55a71149eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.531+0000 7f7993c52740 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149eb30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7984011d10 tx=0x7f7984011d40 comp rx=0 tx=0).stop 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.532+0000 7f7993c52740 1 -- 172.21.15.37:0/1292067383 shutdown_connections 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.532+0000 7f7993c52740 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55a7114a3e20 0x55a7114a30e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.532+0000 7f7993c52740 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149eb30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.532+0000 7f7993c52740 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148dad0 0x55a711497af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.532+0000 7f796eff5640 0 client.15099 ms_handle_reset on v2:172.21.15.37:6840/933565564 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.532+0000 7f7993c52740 1 --2- 172.21.15.37:0/1292067383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148d0e0 0x55a7114975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.532+0000 7f7993c52740 1 -- 172.21.15.37:0/1292067383 >> 172.21.15.37:0/1292067383 conn(0x55a711455400 msgr2=0x55a711487c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.532+0000 7f7993c52740 1 -- 172.21.15.37:0/1292067383 shutdown_connections 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.532+0000 7f7993c52740 1 -- 172.21.15.37:0/1292067383 wait complete. 2024-03-26T06:22:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.532+0000 7f7993c52740 20 client.15099 trim_cache size 0 max 16384 2024-03-26T06:22:19.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.547+0000 7f7993c52740 1 Processor -- start 2024-03-26T06:22:19.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.547+0000 7f7993c52740 1 -- start start 2024-03-26T06:22:19.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.547+0000 7f7993c52740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148d0e0 0x55a7114a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.547+0000 7f7993c52740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148dad0 0x55a71149eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.547+0000 7f7993c52740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.547+0000 7f7993c52740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7114a3640 con 0x55a711492c40 2024-03-26T06:22:19.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.547+0000 7f7993c52740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7114a3ca0 con 0x55a71148d0e0 2024-03-26T06:22:19.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.547+0000 7f7993c52740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7114a4300 con 0x55a71148dad0 2024-03-26T06:22:19.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.547+0000 7f798b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.547+0000 7f798b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59520/0 (socket says 172.21.15.37:59520) 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.547+0000 7f798b59e640 1 -- 172.21.15.37:0/969335019 learned_addr learned my addr 172.21.15.37:0/969335019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f798a59c640 1 --2- 172.21.15.37:0/969335019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148dad0 0x55a71149eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f798b59e640 1 -- 172.21.15.37:0/969335019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148dad0 msgr2=0x55a71149eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f798b59e640 1 --2- 172.21.15.37:0/969335019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148dad0 0x55a71149eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f798b59e640 1 -- 172.21.15.37:0/969335019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148d0e0 msgr2=0x55a7114a30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f798b59e640 1 --2- 172.21.15.37:0/969335019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148d0e0 0x55a7114a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f798b59e640 1 -- 172.21.15.37:0/969335019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a71148deb0 con 0x55a711492c40 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f798b59e640 1 --2- 172.21.15.37:0/969335019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114975b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7984003d00 tx=0x7f798400a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f924e33d64f5f9f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f7970ff9640 1 -- 172.21.15.37:0/969335019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984022070 con 0x55a711492c40 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f7970ff9640 1 -- 172.21.15.37:0/969335019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7984003e30 con 0x55a711492c40 2024-03-26T06:22:19.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f7970ff9640 1 -- 172.21.15.37:0/969335019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984004c20 con 0x55a711492c40 2024-03-26T06:22:19.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f7993c52740 1 -- 172.21.15.37:0/969335019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 msgr2=0x55a7114975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f7993c52740 1 --2- 172.21.15.37:0/969335019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114975b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7984003d00 tx=0x7f798400a550 comp rx=0 tx=0).stop 2024-03-26T06:22:19.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f7993c52740 1 -- 172.21.15.37:0/969335019 shutdown_connections 2024-03-26T06:22:19.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f7993c52740 1 --2- 172.21.15.37:0/969335019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a7114975b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f7993c52740 1 --2- 172.21.15.37:0/969335019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148dad0 0x55a71149eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.548+0000 7f7993c52740 1 --2- 172.21.15.37:0/969335019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148d0e0 0x55a7114a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 1 -- 172.21.15.37:0/969335019 >> 172.21.15.37:0/969335019 conn(0x55a711455400 msgr2=0x55a711496100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 1 -- 172.21.15.37:0/969335019 shutdown_connections 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 1 -- 172.21.15.37:0/969335019 wait complete. 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 1 -- start start 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 1 Processor -- start 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148d0e0 0x55a71149e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148dad0 0x55a71149b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7114a4fc0 con 0x55a711492c40 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7114a4960 con 0x55a71148dad0 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7114a5620 con 0x55a71148d0e0 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f7993c52740 10 client.0 _ll_register_callbacks cb 0x7f79840147b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f798b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f798b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59530/0 (socket says 172.21.15.37:59530) 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f798b59e640 1 -- 172.21.15.37:0/3796641892 learned_addr learned my addr 172.21.15.37:0/3796641892 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f798a59c640 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148dad0 0x55a71149b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f798ad9d640 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148d0e0 0x55a71149e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f798b59e640 1 -- 172.21.15.37:0/3796641892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148d0e0 msgr2=0x55a71149e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f798b59e640 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148d0e0 0x55a71149e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f798b59e640 1 -- 172.21.15.37:0/3796641892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148dad0 msgr2=0x55a71149b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f798b59e640 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148dad0 0x55a71149b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.549+0000 7f798b59e640 1 -- 172.21.15.37:0/3796641892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a71149bd90 con 0x55a711492c40 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f798b59e640 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149b7d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7984000f80 tx=0x7f7984011fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a870f20dfc85dc04 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f796eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f796eff5640 1 -- 172.21.15.37:0/3796641892 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984046070 con 0x55a711492c40 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f796eff5640 1 -- 172.21.15.37:0/3796641892 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7984004c00 con 0x55a711492c40 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f7993c52740 1 -- 172.21.15.37:0/3796641892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7984003e30 con 0x55a711492c40 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f796eff5640 1 -- 172.21.15.37:0/3796641892 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7984052050 con 0x55a711492c40 2024-03-26T06:22:19.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f7993c52740 10 client.15105 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f7993c52740 1 -- 172.21.15.37:0/3796641892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a7114faa60 con 0x55a711492c40 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f796eff5640 1 -- 172.21.15.37:0/3796641892 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7984003e30 con 0x55a711492c40 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f796eff5640 1 -- 172.21.15.37:0/3796641892 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f798402d050 con 0x55a711492c40 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f7993c52740 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f7993c52740 10 client.15105 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f7993c52740 20 client.15105 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f7993c52740 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55a7114fecb0 0x55a7114fac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f7993c52740 4 client.15105 mds_command: new command op to 14682 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f7993c52740 1 -- 172.21.15.37:0/3796641892 --> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a7113fa050 con 0x55a7114fecb0 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.550+0000 7f798ad9d640 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55a7114fecb0 0x55a7114fac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f798ad9d640 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55a7114fecb0 0x55a7114fac90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=43a3497c1bcefdd2 server_cookie=c26ce4a96c7febc5 in_seq=0 out_seq=0 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f796eff5640 10 client.15105 ms_handle_connect on v2:172.21.15.37:6840/933565564 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f796eff5640 1 -- 172.21.15.37:0/3796641892 <== mds.4 v2:172.21.15.37:6840/933565564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a7113fa050 con 0x55a7114fecb0 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f796eff5640 10 client.15105 handle_command_reply: tid=0 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 1 client.15105 shutdown 2024-03-26T06:22:19.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 1 -- 172.21.15.37:0/3796641892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 msgr2=0x55a71149b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149b7d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7984000f80 tx=0x7f7984011fb0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 1 -- 172.21.15.37:0/3796641892 shutdown_connections 2024-03-26T06:22:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55a7114fecb0 0x55a7114fac90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a711492c40 0x55a71149b7d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a71148dad0 0x55a71149b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f796eff5640 0 client.15105 ms_handle_reset on v2:172.21.15.37:6840/933565564 2024-03-26T06:22:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 1 --2- 172.21.15.37:0/3796641892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a71148d0e0 0x55a71149e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 1 -- 172.21.15.37:0/3796641892 >> 172.21.15.37:0/3796641892 conn(0x55a711455400 msgr2=0x55a71148ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 1 -- 172.21.15.37:0/3796641892 shutdown_connections 2024-03-26T06:22:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 1 -- 172.21.15.37:0/3796641892 wait complete. 2024-03-26T06:22:19.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.551+0000 7f7993c52740 20 client.15105 trim_cache size 0 max 16384 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 4, 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 14682, 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 79.968786277000007, 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 87, 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 93.269725964000003 2024-03-26T06:22:19.559 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:22:19.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 -- 172.21.15.37:0/3728126753 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f79740a2800 msgr2=0x7f79740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f79740a2800 0x7f79740a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7984014c60 tx=0x7f798400ae60 comp rx=0 tx=0).stop 2024-03-26T06:22:19.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 -- 172.21.15.37:0/3728126753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c101320 msgr2=0x7f798c0750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c101320 0x7f798c0750a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f797c00c970 tx=0x7f797c00ce30 comp rx=0 tx=0).stop 2024-03-26T06:22:19.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 -- 172.21.15.37:0/3728126753 shutdown_connections 2024-03-26T06:22:19.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f79740a2800 0x7f79740a4cc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f798c102ed0 0x7f798c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f798c102510 0x7f798c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 --2- 172.21.15.37:0/3728126753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c101320 0x7f798c0750a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 -- 172.21.15.37:0/3728126753 >> 172.21.15.37:0/3728126753 conn(0x7f798c0fc890 msgr2=0x7f798c105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 -- 172.21.15.37:0/3728126753 shutdown_connections 2024-03-26T06:22:19.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.552+0000 7f7991073640 1 -- 172.21.15.37:0/3728126753 wait complete. 2024-03-26T06:22:19.564 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-26T06:22:19.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:19 smithi066 ceph-mon[36504]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T06:22:19.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:19 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3520630371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:19.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:19 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1507215504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:19.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:19 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1673027473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:19.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:19 smithi149 ceph-mon[36681]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T06:22:19.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:19 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3520630371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:19.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:19 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1507215504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:19.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:19 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1673027473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:19.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.667+0000 7fa803897640 1 -- 172.21.15.37:0/2222322001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc107090 msgr2=0x7fa7fc109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.667+0000 7fa803897640 1 --2- 172.21.15.37:0/2222322001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc107090 0x7fa7fc109480 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009ca0 tx=0x7fa7f402c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.667+0000 7fa803897640 1 -- 172.21.15.37:0/2222322001 shutdown_connections 2024-03-26T06:22:19.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.667+0000 7fa803897640 1 --2- 172.21.15.37:0/2222322001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc107090 0x7fa7fc109480 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.667+0000 7fa803897640 1 --2- 172.21.15.37:0/2222322001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7fc104760 0x7fa7fc106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.667+0000 7fa803897640 1 --2- 172.21.15.37:0/2222322001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc101e30 0x7fa7fc104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.667+0000 7fa803897640 1 -- 172.21.15.37:0/2222322001 >> 172.21.15.37:0/2222322001 conn(0x7fa7fc0fb780 msgr2=0x7fa7fc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.667+0000 7fa803897640 1 -- 172.21.15.37:0/2222322001 shutdown_connections 2024-03-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.667+0000 7fa803897640 1 -- 172.21.15.37:0/2222322001 wait complete. 2024-03-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.667+0000 7fa803897640 1 Processor -- start 2024-03-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.667+0000 7fa803897640 1 -- start start 2024-03-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa803897640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7fc101e30 0x7fa7fc1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa803897640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc104760 0x7fa7fc199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa803897640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc107090 0x7fa7fc1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa803897640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc10cf90 con 0x7fa7fc107090 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa803897640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc10ce10 con 0x7fa7fc104760 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa803897640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc10d110 con 0x7fa7fc101e30 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa801e34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc107090 0x7fa7fc1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa801e34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc107090 0x7fa7fc1a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59552/0 (socket says 172.21.15.37:59552) 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa801e34640 1 -- 172.21.15.37:0/2690015596 learned_addr learned my addr 172.21.15.37:0/2690015596 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa801633640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7fc101e30 0x7fa7fc1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa800e32640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc104760 0x7fa7fc199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa801e34640 1 -- 172.21.15.37:0/2690015596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7fc101e30 msgr2=0x7fa7fc1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa801e34640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7fc101e30 0x7fa7fc1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa801e34640 1 -- 172.21.15.37:0/2690015596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc104760 msgr2=0x7fa7fc199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa801e34640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc104760 0x7fa7fc199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa801e34640 1 -- 172.21.15.37:0/2690015596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc1a13c0 con 0x7fa7fc107090 2024-03-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa800e32640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc104760 0x7fa7fc199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa801e34640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc107090 0x7fa7fc1a0d50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4037040 tx=0x7fa7f4032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa7f27fc640 1 -- 172.21.15.37:0/2690015596 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4039040 con 0x7fa7fc107090 2024-03-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa7f27fc640 1 -- 172.21.15.37:0/2690015596 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7f4044070 con 0x7fa7fc107090 2024-03-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa803897640 1 -- 172.21.15.37:0/2690015596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7fc1a1650 con 0x7fa7fc107090 2024-03-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa7f27fc640 1 -- 172.21.15.37:0/2690015596 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4004330 con 0x7fa7fc107090 2024-03-26T06:22:19.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.668+0000 7fa803897640 1 -- 172.21.15.37:0/2690015596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7fc1a1b40 con 0x7fa7fc107090 2024-03-26T06:22:19.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.670+0000 7fa7f27fc640 1 -- 172.21.15.37:0/2690015596 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa7f4003a00 con 0x7fa7fc107090 2024-03-26T06:22:19.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa80647b740 1 Processor -- start 2024-03-26T06:22:19.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa80647b740 1 -- start start 2024-03-26T06:22:19.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa7f27fc640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7e80a2800 0x7fa7e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa80647b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb360e0 0x55bb8fb36500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa80647b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb3b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa80647b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb3e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa80647b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fa9b7c0 con 0x55bb8fb36ad0 2024-03-26T06:22:19.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa80647b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fa44b50 con 0x55bb8fb360e0 2024-03-26T06:22:19.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa80647b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fa97860 con 0x55bb8fb3bc40 2024-03-26T06:22:19.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa7f27fc640 1 -- 172.21.15.37:0/2690015596 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa7f40f12e0 con 0x7fa7fc107090 2024-03-26T06:22:19.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa800e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb3b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa800e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb3b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59558/0 (socket says 172.21.15.37:59558) 2024-03-26T06:22:19.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa800e32640 1 -- 172.21.15.37:0/1290606537 learned_addr learned my addr 172.21.15.37:0/1290606537 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa801633640 1 --2- 172.21.15.37:0/1290606537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb360e0 0x55bb8fb36500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa800e32640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7e80a2800 0x7fa7e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.671+0000 7fa801e34640 1 --2- 172.21.15.37:0/1290606537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb3e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.672+0000 7fa800e32640 1 -- 172.21.15.37:0/1290606537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb3bc40 msgr2=0x55bb8fb3e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.672+0000 7fa800e32640 1 --2- 172.21.15.37:0/1290606537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb3e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.672+0000 7fa800e32640 1 -- 172.21.15.37:0/1290606537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb360e0 msgr2=0x55bb8fb36500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.672+0000 7fa800e32640 1 --2- 172.21.15.37:0/1290606537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb360e0 0x55bb8fb36500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.672+0000 7fa800e32640 1 -- 172.21.15.37:0/1290606537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb8fa589a0 con 0x55bb8fb36ad0 2024-03-26T06:22:19.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.672+0000 7fa801633640 1 --2- 172.21.15.37:0/1290606537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb360e0 0x55bb8fb36500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:22:19.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.672+0000 7fa800e32640 1 --2- 172.21.15.37:0/1290606537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb3b5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec012ac0 tx=0x7fa7ec012ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e962ccb8055516b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.672+0000 7fa800e32640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7e80a2800 0x7fa7e80a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00f730 tx=0x7fa7ec009000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.672+0000 7fa7e77fe640 1 -- 172.21.15.37:0/1290606537 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec022070 con 0x55bb8fb36ad0 2024-03-26T06:22:19.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.672+0000 7fa7e77fe640 1 -- 172.21.15.37:0/1290606537 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7ec00de50 con 0x55bb8fb36ad0 2024-03-26T06:22:19.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.672+0000 7fa7e77fe640 1 -- 172.21.15.37:0/1290606537 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec01eba0 con 0x55bb8fb36ad0 2024-03-26T06:22:19.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 -- 172.21.15.37:0/1290606537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 msgr2=0x55bb8fb3b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 --2- 172.21.15.37:0/1290606537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb3b5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec012ac0 tx=0x7fa7ec012ec0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 -- 172.21.15.37:0/1290606537 shutdown_connections 2024-03-26T06:22:19.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 --2- 172.21.15.37:0/1290606537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb3e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 --2- 172.21.15.37:0/1290606537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb3b5d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 --2- 172.21.15.37:0/1290606537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb360e0 0x55bb8fb36500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 -- 172.21.15.37:0/1290606537 >> 172.21.15.37:0/1290606537 conn(0x55bb8fafe400 msgr2=0x55bb8fb31a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 -- 172.21.15.37:0/1290606537 shutdown_connections 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 -- 172.21.15.37:0/1290606537 wait complete. 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 -- start start 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 Processor -- start 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb360e0 0x55bb8fb40650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb40b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.673+0000 7fa80647b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb47bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa80647b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fb3eb90 con 0x55bb8fb36ad0 2024-03-26T06:22:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa80647b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fa57f90 con 0x55bb8fb3bc40 2024-03-26T06:22:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa80647b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fb3ed10 con 0x55bb8fb360e0 2024-03-26T06:22:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa80647b740 10 client.0 _ll_register_callbacks cb 0x55bb8fa93400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa800e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb40b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa800e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb40b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59572/0 (socket says 172.21.15.37:59572) 2024-03-26T06:22:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa800e32640 1 -- 172.21.15.37:0/3380581438 learned_addr learned my addr 172.21.15.37:0/3380581438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa801633640 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb360e0 0x55bb8fb40650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa800e32640 1 -- 172.21.15.37:0/3380581438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb360e0 msgr2=0x55bb8fb40650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa800e32640 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb360e0 0x55bb8fb40650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa800e32640 1 -- 172.21.15.37:0/3380581438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb3bc40 msgr2=0x55bb8fb47bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa801e34640 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb47bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa800e32640 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb47bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa800e32640 1 -- 172.21.15.37:0/3380581438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb8f7b02d0 con 0x55bb8fb36ad0 2024-03-26T06:22:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa800e32640 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb40b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0058c0 tx=0x7fa7ec008b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f573c48732651f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa7e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/3380581438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec02b030 con 0x55bb8fb36ad0 2024-03-26T06:22:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/3380581438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa7ec01e8a0 con 0x55bb8fb36ad0 2024-03-26T06:22:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa80647b740 1 -- 172.21.15.37:0/3380581438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7ec00de50 con 0x55bb8fb36ad0 2024-03-26T06:22:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/3380581438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7ec022040 con 0x55bb8fb36ad0 2024-03-26T06:22:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa80647b740 10 client.15117 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.674+0000 7fa80647b740 1 -- 172.21.15.37:0/3380581438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bb8f87a6b0 con 0x55bb8fb36ad0 2024-03-26T06:22:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.675+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/3380581438 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7ec00de50 con 0x55bb8fb36ad0 2024-03-26T06:22:19.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.675+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/3380581438 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa7ec027890 con 0x55bb8fb36ad0 2024-03-26T06:22:19.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.675+0000 7fa80647b740 10 client.15117 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:19.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.675+0000 7fa80647b740 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T06:22:19.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.675+0000 7fa80647b740 20 client.15117 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:19.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.675+0000 7fa80647b740 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55bb8fb4c180 0x55bb8fb51220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.675+0000 7fa80647b740 4 client.15117 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:22:19.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.675+0000 7fa80647b740 1 -- 172.21.15.37:0/3380581438 --> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb8faa8d00 con 0x55bb8fb4c180 2024-03-26T06:22:19.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.675+0000 7fa801633640 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55bb8fb4c180 0x55bb8fb51220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa801633640 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55bb8fb4c180 0x55bb8fb51220 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd7033f9ad0bca5e server_cookie=7b4c9255d3061594 in_seq=0 out_seq=0 2024-03-26T06:22:19.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa7e4ff9640 10 client.15117 ms_handle_connect on v2:172.21.15.149:6838/1810727373 2024-03-26T06:22:19.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/3380581438 <== mds.3 v2:172.21.15.149:6838/1810727373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bb8faa8d00 con 0x55bb8fb4c180 2024-03-26T06:22:19.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa7e4ff9640 10 client.15117 handle_command_reply: tid=0 2024-03-26T06:22:19.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa80647b740 1 client.15117 shutdown 2024-03-26T06:22:19.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa80647b740 1 -- 172.21.15.37:0/3380581438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 msgr2=0x55bb8fb40b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa80647b740 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb40b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0058c0 tx=0x7fa7ec008b50 comp rx=0 tx=0).stop 2024-03-26T06:22:19.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa80647b740 1 -- 172.21.15.37:0/3380581438 shutdown_connections 2024-03-26T06:22:19.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa80647b740 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55bb8fb4c180 0x55bb8fb51220 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa80647b740 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb47bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa7e4ff9640 0 client.15117 ms_handle_reset on v2:172.21.15.149:6838/1810727373 2024-03-26T06:22:19.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa80647b740 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb40b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa80647b740 1 --2- 172.21.15.37:0/3380581438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb360e0 0x55bb8fb40650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.676+0000 7fa80647b740 1 -- 172.21.15.37:0/3380581438 >> 172.21.15.37:0/3380581438 conn(0x55bb8fafe400 msgr2=0x55bb8fb30c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.677+0000 7fa80647b740 1 -- 172.21.15.37:0/3380581438 shutdown_connections 2024-03-26T06:22:19.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.677+0000 7fa80647b740 1 -- 172.21.15.37:0/3380581438 wait complete. 2024-03-26T06:22:19.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.677+0000 7fa80647b740 20 client.15117 trim_cache size 0 max 16384 2024-03-26T06:22:19.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.692+0000 7fa80647b740 1 Processor -- start 2024-03-26T06:22:19.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.692+0000 7fa80647b740 1 -- start start 2024-03-26T06:22:19.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.692+0000 7fa80647b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb360e0 0x55bb8fb31410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.692+0000 7fa80647b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb47b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.692+0000 7fa80647b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb31950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa80647b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fb4c5c0 con 0x55bb8fb3bc40 2024-03-26T06:22:19.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa80647b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fb4cc20 con 0x55bb8fb360e0 2024-03-26T06:22:19.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa80647b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fb4d280 con 0x55bb8fb36ad0 2024-03-26T06:22:19.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa801e34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb31950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa801e34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb31950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59576/0 (socket says 172.21.15.37:59576) 2024-03-26T06:22:19.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa801e34640 1 -- 172.21.15.37:0/2245929939 learned_addr learned my addr 172.21.15.37:0/2245929939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa801633640 1 --2- 172.21.15.37:0/2245929939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb360e0 0x55bb8fb31410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa800e32640 1 --2- 172.21.15.37:0/2245929939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb47b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa801e34640 1 -- 172.21.15.37:0/2245929939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb36ad0 msgr2=0x55bb8fb47b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa801e34640 1 --2- 172.21.15.37:0/2245929939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb47b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa801e34640 1 -- 172.21.15.37:0/2245929939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb360e0 msgr2=0x55bb8fb31410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa801e34640 1 --2- 172.21.15.37:0/2245929939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb360e0 0x55bb8fb31410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa801e34640 1 -- 172.21.15.37:0/2245929939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb8fb327a0 con 0x55bb8fb3bc40 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa801e34640 1 --2- 172.21.15.37:0/2245929939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb31950 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40bd990 tx=0x7fa7f4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8c408ba593c269 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa7e6ffd640 1 -- 172.21.15.37:0/2245929939 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40bdc90 con 0x55bb8fb3bc40 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa7e6ffd640 1 -- 172.21.15.37:0/2245929939 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7f40c1de0 con 0x55bb8fb3bc40 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.693+0000 7fa7e6ffd640 1 -- 172.21.15.37:0/2245929939 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40bdef0 con 0x55bb8fb3bc40 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.694+0000 7fa80647b740 1 -- 172.21.15.37:0/2245929939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb3bc40 msgr2=0x55bb8fb31950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.694+0000 7fa80647b740 1 --2- 172.21.15.37:0/2245929939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb31950 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40bd990 tx=0x7fa7f4003970 comp rx=0 tx=0).stop 2024-03-26T06:22:19.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.694+0000 7fa80647b740 1 -- 172.21.15.37:0/2245929939 shutdown_connections 2024-03-26T06:22:19.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.694+0000 7fa80647b740 1 --2- 172.21.15.37:0/2245929939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb31950 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.694+0000 7fa80647b740 1 --2- 172.21.15.37:0/2245929939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb47b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.694+0000 7fa80647b740 1 --2- 172.21.15.37:0/2245929939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb360e0 0x55bb8fb31410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.694+0000 7fa80647b740 1 -- 172.21.15.37:0/2245929939 >> 172.21.15.37:0/2245929939 conn(0x55bb8fafe400 msgr2=0x55bb8fb3f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.694+0000 7fa80647b740 1 -- 172.21.15.37:0/2245929939 shutdown_connections 2024-03-26T06:22:19.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.694+0000 7fa80647b740 1 -- 172.21.15.37:0/2245929939 wait complete. 2024-03-26T06:22:19.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.694+0000 7fa80647b740 1 -- start start 2024-03-26T06:22:19.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.694+0000 7fa80647b740 1 Processor -- start 2024-03-26T06:22:19.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa80647b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb360e0 0x55bb8fb477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa80647b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb44250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa80647b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb44790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa80647b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fb4df40 con 0x55bb8fb360e0 2024-03-26T06:22:19.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa80647b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fb4d8e0 con 0x55bb8fb3bc40 2024-03-26T06:22:19.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa80647b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8fb4e5a0 con 0x55bb8fb36ad0 2024-03-26T06:22:19.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa80647b740 10 client.0 _ll_register_callbacks cb 0x7fa7ec00db10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:19.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa801633640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb360e0 0x55bb8fb477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa801633640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb360e0 0x55bb8fb477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59590/0 (socket says 172.21.15.37:59590) 2024-03-26T06:22:19.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa801633640 1 -- 172.21.15.37:0/1924852019 learned_addr learned my addr 172.21.15.37:0/1924852019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa801e34640 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb44790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa801633640 1 -- 172.21.15.37:0/1924852019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb36ad0 msgr2=0x55bb8fb44250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa801633640 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb44250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa801633640 1 -- 172.21.15.37:0/1924852019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb3bc40 msgr2=0x55bb8fb44790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa801633640 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb44790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa801633640 1 -- 172.21.15.37:0/1924852019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb8fb47de0 con 0x55bb8fb360e0 2024-03-26T06:22:19.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa801e34640 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb44790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:19.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa801633640 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb360e0 0x55bb8fb477b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55bb8fb4e790 tx=0x7fa7f800c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6b506c8b2d76c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa7e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/1924852019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f801b510 con 0x55bb8fb360e0 2024-03-26T06:22:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/1924852019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa7f801b6b0 con 0x55bb8fb360e0 2024-03-26T06:22:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.695+0000 7fa80647b740 1 -- 172.21.15.37:0/1924852019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7f40c1de0 con 0x55bb8fb360e0 2024-03-26T06:22:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/1924852019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7f801b8b0 con 0x55bb8fb360e0 2024-03-26T06:22:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa80647b740 10 client.15123 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa80647b740 1 -- 172.21.15.37:0/1924852019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bb8fba3ed0 con 0x55bb8fb360e0 2024-03-26T06:22:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/1924852019 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7f40c1de0 con 0x55bb8fb360e0 2024-03-26T06:22:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/1924852019 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa7f80234a0 con 0x55bb8fb360e0 2024-03-26T06:22:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa80647b740 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa80647b740 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T06:22:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa80647b740 20 client.15123 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa80647b740 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55bb8fb4c180 0x55bb8fba41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa80647b740 4 client.15123 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T06:22:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa80647b740 1 -- 172.21.15.37:0/1924852019 --> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bb8faa3050 con 0x55bb8fb4c180 2024-03-26T06:22:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.696+0000 7fa800e32640 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55bb8fb4c180 0x55bb8fba41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa7e4ff9640 10 client.15123 ms_handle_connect on v2:172.21.15.149:6838/1810727373 2024-03-26T06:22:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa800e32640 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55bb8fb4c180 0x55bb8fba41c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ae344fc51ff39dd3 server_cookie=a0a95ebb9c7206b0 in_seq=0 out_seq=0 2024-03-26T06:22:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa7e4ff9640 1 -- 172.21.15.37:0/1924852019 <== mds.3 v2:172.21.15.149:6838/1810727373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55bb8faa3050 con 0x55bb8fb4c180 2024-03-26T06:22:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa7e4ff9640 10 client.15123 handle_command_reply: tid=0 2024-03-26T06:22:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa80647b740 1 client.15123 shutdown 2024-03-26T06:22:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa80647b740 1 -- 172.21.15.37:0/1924852019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb360e0 msgr2=0x55bb8fb477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa80647b740 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb360e0 0x55bb8fb477b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55bb8fb4e790 tx=0x7fa7f800c360 comp rx=0 tx=0).stop 2024-03-26T06:22:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa80647b740 1 -- 172.21.15.37:0/1924852019 shutdown_connections 2024-03-26T06:22:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa80647b740 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55bb8fb4c180 0x55bb8fba41c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa80647b740 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bb8fb3bc40 0x55bb8fb44790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa80647b740 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb8fb36ad0 0x55bb8fb44250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa7e4ff9640 0 client.15123 ms_handle_reset on v2:172.21.15.149:6838/1810727373 2024-03-26T06:22:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa80647b740 1 --2- 172.21.15.37:0/1924852019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8fb360e0 0x55bb8fb477b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa80647b740 1 -- 172.21.15.37:0/1924852019 >> 172.21.15.37:0/1924852019 conn(0x55bb8fafe400 msgr2=0x55bb8fb34c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa80647b740 1 -- 172.21.15.37:0/1924852019 shutdown_connections 2024-03-26T06:22:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.697+0000 7fa80647b740 1 -- 172.21.15.37:0/1924852019 wait complete. 2024-03-26T06:22:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa80647b740 20 client.15123 trim_cache size 0 max 16384 2024-03-26T06:22:19.709 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:22:19.709 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:22:19.709 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 3, 2024-03-26T06:22:19.709 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24469, 2024-03-26T06:22:19.709 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-26T06:22:19.709 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-26T06:22:19.709 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-26T06:22:19.709 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 81.085055105999999, 2024-03-26T06:22:19.710 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-03-26T06:22:19.710 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 87, 2024-03-26T06:22:19.710 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T06:22:19.710 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 98.895936461999995 2024-03-26T06:22:19.710 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:22:19.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 -- 172.21.15.37:0/2690015596 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7e80a2800 msgr2=0x7fa7e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7e80a2800 0x7fa7e80a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00f730 tx=0x7fa7ec009000 comp rx=0 tx=0).stop 2024-03-26T06:22:19.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 -- 172.21.15.37:0/2690015596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc107090 msgr2=0x7fa7fc1a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc107090 0x7fa7fc1a0d50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4037040 tx=0x7fa7f4032f90 comp rx=0 tx=0).stop 2024-03-26T06:22:19.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 -- 172.21.15.37:0/2690015596 shutdown_connections 2024-03-26T06:22:19.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7e80a2800 0x7fa7e80a4cc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc107090 0x7fa7fc1a0d50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7fc104760 0x7fa7fc199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 --2- 172.21.15.37:0/2690015596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7fc101e30 0x7fa7fc1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 -- 172.21.15.37:0/2690015596 >> 172.21.15.37:0/2690015596 conn(0x7fa7fc0fb780 msgr2=0x7fa7fc101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 -- 172.21.15.37:0/2690015596 shutdown_connections 2024-03-26T06:22:19.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.698+0000 7fa803897640 1 -- 172.21.15.37:0/2690015596 wait complete. 2024-03-26T06:22:19.713 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-26T06:22:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.816+0000 7f1e53d9e640 1 -- 172.21.15.37:0/3667219348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e4c0ff370 msgr2=0x7f1e4c0ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.816+0000 7f1e53d9e640 1 --2- 172.21.15.37:0/3667219348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e4c0ff370 0x7f1e4c0ff7f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009900 tx=0x7f1e3c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.816+0000 7f1e53d9e640 1 -- 172.21.15.37:0/3667219348 shutdown_connections 2024-03-26T06:22:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.816+0000 7f1e53d9e640 1 --2- 172.21.15.37:0/3667219348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e4c0ffd30 0x7f1e4c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.816+0000 7f1e53d9e640 1 --2- 172.21.15.37:0/3667219348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e4c0ff370 0x7f1e4c0ff7f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.816+0000 7f1e53d9e640 1 --2- 172.21.15.37:0/3667219348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4c102900 0x7f1e4c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.816+0000 7f1e53d9e640 1 -- 172.21.15.37:0/3667219348 >> 172.21.15.37:0/3667219348 conn(0x7f1e4c0faef0 msgr2=0x7f1e4c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.816+0000 7f1e53d9e640 1 -- 172.21.15.37:0/3667219348 shutdown_connections 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.816+0000 7f1e53d9e640 1 -- 172.21.15.37:0/3667219348 wait complete. 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e53d9e640 1 Processor -- start 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e53d9e640 1 -- start start 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e53d9e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e4c0ff370 0x7f1e4c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e53d9e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e4c0ffd30 0x7f1e4c199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e53d9e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4c102900 0x7f1e4c1a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e53d9e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c10d020 con 0x7f1e4c0ffd30 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e53d9e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c10cea0 con 0x7f1e4c0ff370 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e53d9e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c10d1a0 con 0x7f1e4c102900 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e51339640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e4c0ffd30 0x7f1e4c199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e51339640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e4c0ffd30 0x7f1e4c199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59606/0 (socket says 172.21.15.37:59606) 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e51339640 1 -- 172.21.15.37:0/1201893123 learned_addr learned my addr 172.21.15.37:0/1201893123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e51b3a640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e4c0ff370 0x7f1e4c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e5233b640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4c102900 0x7f1e4c1a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e51339640 1 -- 172.21.15.37:0/1201893123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4c102900 msgr2=0x7f1e4c1a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e51339640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4c102900 0x7f1e4c1a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e51339640 1 -- 172.21.15.37:0/1201893123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e4c0ff370 msgr2=0x7f1e4c1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e51339640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e4c0ff370 0x7f1e4c1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e51339640 1 -- 172.21.15.37:0/1201893123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c1a14f0 con 0x7f1e4c0ffd30 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e51339640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e4c0ffd30 0x7f1e4c199e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c02efd0 tx=0x7f1e3c02cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.817+0000 7f1e42ffd640 1 -- 172.21.15.37:0/1201893123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c041070 con 0x7f1e4c0ffd30 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.818+0000 7f1e42ffd640 1 -- 172.21.15.37:0/1201893123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c0054e0 con 0x7f1e4c0ffd30 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.818+0000 7f1e53d9e640 1 -- 172.21.15.37:0/1201893123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c1a1780 con 0x7f1e4c0ffd30 2024-03-26T06:22:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.818+0000 7f1e42ffd640 1 -- 172.21.15.37:0/1201893123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c034440 con 0x7f1e4c0ffd30 2024-03-26T06:22:19.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.818+0000 7f1e53d9e640 1 -- 172.21.15.37:0/1201893123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4c1a1c70 con 0x7f1e4c0ffd30 2024-03-26T06:22:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.820+0000 7f1e42ffd640 1 -- 172.21.15.37:0/1201893123 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1e3c044070 con 0x7f1e4c0ffd30 2024-03-26T06:22:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.820+0000 7f1e5697e740 1 Processor -- start 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5697e740 1 -- start start 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e42ffd640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1e380a2800 0x7f1e380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce60e0 0x55f22dce6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce6ad0 0x55f22dceb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dcee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5697e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dc4b7c0 con 0x55f22dcebc40 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5697e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dbf4b50 con 0x55f22dce60e0 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5697e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dc47860 con 0x55f22dce6ad0 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5233b640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1e380a2800 0x7f1e380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e42ffd640 1 -- 172.21.15.37:0/1201893123 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1e3c0f0710 con 0x7f1e4c0ffd30 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5233b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dcee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5233b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dcee0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59608/0 (socket says 172.21.15.37:59608) 2024-03-26T06:22:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5233b640 1 -- 172.21.15.37:0/3330298631 learned_addr learned my addr 172.21.15.37:0/3330298631 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e51339640 1 --2- 172.21.15.37:0/3330298631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce6ad0 0x55f22dceb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e51b3a640 1 --2- 172.21.15.37:0/3330298631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce60e0 0x55f22dce6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5233b640 1 -- 172.21.15.37:0/3330298631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce6ad0 msgr2=0x55f22dceb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5233b640 1 --2- 172.21.15.37:0/3330298631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce6ad0 0x55f22dceb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5233b640 1 -- 172.21.15.37:0/3330298631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce60e0 msgr2=0x55f22dce6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5233b640 1 --2- 172.21.15.37:0/3330298631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce60e0 0x55f22dce6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5233b640 1 -- 172.21.15.37:0/3330298631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f22dc089a0 con 0x55f22dcebc40 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5233b640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1e380a2800 0x7f1e380a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e44012be0 tx=0x7f1e44009db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e5233b640 1 --2- 172.21.15.37:0/3330298631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dcee0d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1e44010800 tx=0x7f1e4401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf30479b98b1f35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.821+0000 7f1e37fff640 1 -- 172.21.15.37:0/3330298631 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44004b20 con 0x55f22dcebc40 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.822+0000 7f1e37fff640 1 -- 172.21.15.37:0/3330298631 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e44004cc0 con 0x55f22dcebc40 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.822+0000 7f1e37fff640 1 -- 172.21.15.37:0/3330298631 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e440051e0 con 0x55f22dcebc40 2024-03-26T06:22:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.822+0000 7f1e5697e740 1 -- 172.21.15.37:0/3330298631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 msgr2=0x55f22dcee0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.822+0000 7f1e5697e740 1 --2- 172.21.15.37:0/3330298631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dcee0d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1e44010800 tx=0x7f1e4401c000 comp rx=0 tx=0).stop 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.822+0000 7f1e5697e740 1 -- 172.21.15.37:0/3330298631 shutdown_connections 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.822+0000 7f1e5697e740 1 --2- 172.21.15.37:0/3330298631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dcee0d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.822+0000 7f1e5697e740 1 --2- 172.21.15.37:0/3330298631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce6ad0 0x55f22dceb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.822+0000 7f1e5697e740 1 --2- 172.21.15.37:0/3330298631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce60e0 0x55f22dce6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.822+0000 7f1e5697e740 1 -- 172.21.15.37:0/3330298631 >> 172.21.15.37:0/3330298631 conn(0x55f22dcae400 msgr2=0x55f22dce1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.822+0000 7f1e5697e740 1 -- 172.21.15.37:0/3330298631 shutdown_connections 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.822+0000 7f1e5697e740 1 -- 172.21.15.37:0/3330298631 wait complete. 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e5697e740 1 -- start start 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e5697e740 1 Processor -- start 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce60e0 0x55f22dcf0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dce6ad0 0x55f22dcf0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dcebc40 0x55f22dcf7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e5697e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dceeb90 con 0x55f22dce6ad0 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e5697e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dc07f90 con 0x55f22dcebc40 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e5697e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dceed10 con 0x55f22dce60e0 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e5697e740 10 client.0 _ll_register_callbacks cb 0x55f22dc43400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e51339640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dce6ad0 0x55f22dcf0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e51339640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dce6ad0 0x55f22dcf0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59624/0 (socket says 172.21.15.37:59624) 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e51339640 1 -- 172.21.15.37:0/1703375058 learned_addr learned my addr 172.21.15.37:0/1703375058 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e51b3a640 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce60e0 0x55f22dcf0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e5233b640 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dcebc40 0x55f22dcf7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e51339640 1 -- 172.21.15.37:0/1703375058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce60e0 msgr2=0x55f22dcf0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e51339640 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce60e0 0x55f22dcf0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e51339640 1 -- 172.21.15.37:0/1703375058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dcebc40 msgr2=0x55f22dcf7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e51339640 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dcebc40 0x55f22dcf7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e51339640 1 -- 172.21.15.37:0/1703375058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f22d9602d0 con 0x55f22dce6ad0 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.823+0000 7f1e5233b640 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dcebc40 0x55f22dcf7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e51339640 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dce6ad0 0x55f22dcf0ac0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c000c00 tx=0x7f1e3c0bcc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e9363d342d3da3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e357fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e357fa640 1 -- 172.21.15.37:0/1703375058 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c04b040 con 0x55f22dce6ad0 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e357fa640 1 -- 172.21.15.37:0/1703375058 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1e3c0bb3c0 con 0x55f22dce6ad0 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e5697e740 1 -- 172.21.15.37:0/1703375058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e44004cc0 con 0x55f22dce6ad0 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e357fa640 1 -- 172.21.15.37:0/1703375058 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e3c0dd070 con 0x55f22dce6ad0 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e5697e740 10 client.15135 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e5697e740 1 -- 172.21.15.37:0/1703375058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f22da2a6b0 con 0x55f22dce6ad0 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e357fa640 1 -- 172.21.15.37:0/1703375058 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e44004cc0 con 0x55f22dce6ad0 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e357fa640 1 -- 172.21.15.37:0/1703375058 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1e3c0bd320 con 0x55f22dce6ad0 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e5697e740 10 client.15135 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e5697e740 10 client.15135 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T06:22:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e5697e740 20 client.15135 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e5697e740 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x55f22dcfd180 0x55f22dcfbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e5697e740 4 client.15135 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.824+0000 7f1e5697e740 1 -- 172.21.15.37:0/1703375058 --> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f22dc58d00 con 0x55f22dcfd180 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.825+0000 7f1e51b3a640 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x55f22dcfd180 0x55f22dcfbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.825+0000 7f1e51b3a640 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x55f22dcfd180 0x55f22dcfbff0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26facd249b79a555 server_cookie=fe311fd29d8002de in_seq=0 out_seq=0 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.825+0000 7f1e357fa640 10 client.15135 ms_handle_connect on v2:172.21.15.66:6834/4186424754 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e357fa640 1 -- 172.21.15.37:0/1703375058 <== mds.2 v2:172.21.15.66:6834/4186424754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f22dc58d00 con 0x55f22dcfd180 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e357fa640 10 client.15135 handle_command_reply: tid=0 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 1 client.15135 shutdown 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 1 -- 172.21.15.37:0/1703375058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dce6ad0 msgr2=0x55f22dcf0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dce6ad0 0x55f22dcf0ac0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c000c00 tx=0x7f1e3c0bcc00 comp rx=0 tx=0).stop 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 1 -- 172.21.15.37:0/1703375058 shutdown_connections 2024-03-26T06:22:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x55f22dcfd180 0x55f22dcfbff0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dcebc40 0x55f22dcf7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dce6ad0 0x55f22dcf0ac0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e357fa640 0 client.15135 ms_handle_reset on v2:172.21.15.66:6834/4186424754 2024-03-26T06:22:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 1 --2- 172.21.15.37:0/1703375058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce60e0 0x55f22dcf0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 1 -- 172.21.15.37:0/1703375058 >> 172.21.15.37:0/1703375058 conn(0x55f22dcae400 msgr2=0x55f22dce0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 1 -- 172.21.15.37:0/1703375058 shutdown_connections 2024-03-26T06:22:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 1 -- 172.21.15.37:0/1703375058 wait complete. 2024-03-26T06:22:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.826+0000 7f1e5697e740 20 client.15135 trim_cache size 0 max 16384 2024-03-26T06:22:19.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5697e740 1 Processor -- start 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5697e740 1 -- start start 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce60e0 0x55f22dce1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce6ad0 0x55f22dcf7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd05620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5697e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dcfc9a0 con 0x55f22dcebc40 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5697e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dcfd000 con 0x55f22dce60e0 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5697e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dcfd660 con 0x55f22dce6ad0 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5233b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd05620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5233b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd05620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59634/0 (socket says 172.21.15.37:59634) 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5233b640 1 -- 172.21.15.37:0/4167404415 learned_addr learned my addr 172.21.15.37:0/4167404415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5233b640 1 -- 172.21.15.37:0/4167404415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce6ad0 msgr2=0x55f22dcf7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5233b640 1 --2- 172.21.15.37:0/4167404415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce6ad0 0x55f22dcf7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5233b640 1 -- 172.21.15.37:0/4167404415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce60e0 msgr2=0x55f22dce1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5233b640 1 --2- 172.21.15.37:0/4167404415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce60e0 0x55f22dce1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5233b640 1 -- 172.21.15.37:0/4167404415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f22dce6eb0 con 0x55f22dcebc40 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e5233b640 1 --2- 172.21.15.37:0/4167404415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd05620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1e440095a0 tx=0x7f1e44004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceef3c183ceb0bb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e377fe640 1 -- 172.21.15.37:0/4167404415 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4402d020 con 0x55f22dcebc40 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e377fe640 1 -- 172.21.15.37:0/4167404415 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e44004e10 con 0x55f22dcebc40 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.842+0000 7f1e377fe640 1 -- 172.21.15.37:0/4167404415 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44005330 con 0x55f22dcebc40 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.843+0000 7f1e5697e740 1 -- 172.21.15.37:0/4167404415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 msgr2=0x55f22dd05620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.843+0000 7f1e5697e740 1 --2- 172.21.15.37:0/4167404415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd05620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1e440095a0 tx=0x7f1e44004ca0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.843+0000 7f1e5697e740 1 -- 172.21.15.37:0/4167404415 shutdown_connections 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.843+0000 7f1e5697e740 1 --2- 172.21.15.37:0/4167404415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd05620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.843+0000 7f1e5697e740 1 --2- 172.21.15.37:0/4167404415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce6ad0 0x55f22dcf7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.843+0000 7f1e5697e740 1 --2- 172.21.15.37:0/4167404415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce60e0 0x55f22dce1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.843+0000 7f1e5697e740 1 -- 172.21.15.37:0/4167404415 >> 172.21.15.37:0/4167404415 conn(0x55f22dcae400 msgr2=0x55f22dcfbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.843+0000 7f1e5697e740 1 -- 172.21.15.37:0/4167404415 shutdown_connections 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.843+0000 7f1e5697e740 1 -- 172.21.15.37:0/4167404415 wait complete. 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.843+0000 7f1e5697e740 1 -- start start 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.843+0000 7f1e5697e740 1 Processor -- start 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce60e0 0x55f22dcf4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce6ad0 0x55f22dcf4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5697e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd52d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5697e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dcfe320 con 0x55f22dcebc40 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5697e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dcfdcc0 con 0x55f22dce6ad0 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5697e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22dcfe980 con 0x55f22dce60e0 2024-03-26T06:22:19.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5697e740 10 client.0 _ll_register_callbacks cb 0x7f1e440126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5233b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd52d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5233b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd52d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59642/0 (socket says 172.21.15.37:59642) 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5233b640 1 -- 172.21.15.37:0/3534361018 learned_addr learned my addr 172.21.15.37:0/3534361018 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e51339640 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce6ad0 0x55f22dcf4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e51b3a640 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce60e0 0x55f22dcf4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5233b640 1 -- 172.21.15.37:0/3534361018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce60e0 msgr2=0x55f22dcf4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5233b640 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce60e0 0x55f22dcf4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5233b640 1 -- 172.21.15.37:0/3534361018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce6ad0 msgr2=0x55f22dcf4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5233b640 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce6ad0 0x55f22dcf4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5233b640 1 -- 172.21.15.37:0/3534361018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f22dcf7d70 con 0x55f22dcebc40 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e51339640 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce6ad0 0x55f22dcf4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5233b640 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd52d80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1e44000f80 tx=0x7f1e4400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae6ae4d31e329df server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e357fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e357fa640 1 -- 172.21.15.37:0/3534361018 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44038070 con 0x55f22dcebc40 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e357fa640 1 -- 172.21.15.37:0/3534361018 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1e440050b0 con 0x55f22dcebc40 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e5697e740 1 -- 172.21.15.37:0/3534361018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e44004e10 con 0x55f22dcebc40 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.844+0000 7f1e357fa640 1 -- 172.21.15.37:0/3534361018 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e4402d040 con 0x55f22dcebc40 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.845+0000 7f1e5697e740 10 client.15141 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.845+0000 7f1e5697e740 1 -- 172.21.15.37:0/3534361018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f22dd54290 con 0x55f22dcebc40 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.845+0000 7f1e357fa640 1 -- 172.21.15.37:0/3534361018 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e44004e10 con 0x55f22dcebc40 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.845+0000 7f1e357fa640 1 -- 172.21.15.37:0/3534361018 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1e4401f070 con 0x55f22dcebc40 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.845+0000 7f1e5697e740 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.845+0000 7f1e5697e740 10 client.15141 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.845+0000 7f1e5697e740 20 client.15141 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.845+0000 7f1e5697e740 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x55f22dd58270 0x55f22dd54460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.845+0000 7f1e5697e740 4 client.15141 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.845+0000 7f1e5697e740 1 -- 172.21.15.37:0/3534361018 --> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f22dc53050 con 0x55f22dd58270 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.845+0000 7f1e51b3a640 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x55f22dd58270 0x55f22dd54460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e357fa640 10 client.15141 ms_handle_connect on v2:172.21.15.66:6834/4186424754 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e51b3a640 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x55f22dd58270 0x55f22dd54460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9faf9fd50790e38f server_cookie=14934396ce282568 in_seq=0 out_seq=0 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e357fa640 1 -- 172.21.15.37:0/3534361018 <== mds.2 v2:172.21.15.66:6834/4186424754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55f22dc53050 con 0x55f22dd58270 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e357fa640 10 client.15141 handle_command_reply: tid=0 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 1 client.15141 shutdown 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 1 -- 172.21.15.37:0/3534361018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 msgr2=0x55f22dd52d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd52d80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1e44000f80 tx=0x7f1e4400c8e0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 1 -- 172.21.15.37:0/3534361018 shutdown_connections 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x55f22dd58270 0x55f22dd54460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f22dcebc40 0x55f22dd52d80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f22dce6ad0 0x55f22dcf4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 1 --2- 172.21.15.37:0/3534361018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f22dce60e0 0x55f22dcf4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e357fa640 0 client.15141 ms_handle_reset on v2:172.21.15.66:6834/4186424754 2024-03-26T06:22:19.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 1 -- 172.21.15.37:0/3534361018 >> 172.21.15.37:0/3534361018 conn(0x55f22dcae400 msgr2=0x55f22dce1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 1 -- 172.21.15.37:0/3534361018 shutdown_connections 2024-03-26T06:22:19.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 1 -- 172.21.15.37:0/3534361018 wait complete. 2024-03-26T06:22:19.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.846+0000 7f1e5697e740 20 client.15141 trim_cache size 0 max 16384 2024-03-26T06:22:19.855 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:22:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:22:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-03-26T06:22:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24470, 2024-03-26T06:22:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-26T06:22:19.856 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-26T06:22:19.856 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-26T06:22:19.856 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 82.240521146999995, 2024-03-26T06:22:19.856 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-03-26T06:22:19.856 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 87, 2024-03-26T06:22:19.856 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T06:22:19.856 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 109.697752663 2024-03-26T06:22:19.856 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:22:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 -- 172.21.15.37:0/1201893123 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1e380a2800 msgr2=0x7f1e380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1e380a2800 0x7f1e380a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e44012be0 tx=0x7f1e44009db0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 -- 172.21.15.37:0/1201893123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e4c0ffd30 msgr2=0x7f1e4c199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e4c0ffd30 0x7f1e4c199e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c02efd0 tx=0x7f1e3c02cdc0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 -- 172.21.15.37:0/1201893123 shutdown_connections 2024-03-26T06:22:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f1e380a2800 0x7f1e380a4cc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4c102900 0x7f1e4c1a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e4c0ffd30 0x7f1e4c199e10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 --2- 172.21.15.37:0/1201893123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e4c0ff370 0x7f1e4c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 -- 172.21.15.37:0/1201893123 >> 172.21.15.37:0/1201893123 conn(0x7f1e4c0faef0 msgr2=0x7f1e4c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 -- 172.21.15.37:0/1201893123 shutdown_connections 2024-03-26T06:22:19.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.847+0000 7f1e53d9e640 1 -- 172.21.15.37:0/1201893123 wait complete. 2024-03-26T06:22:19.859 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-26T06:22:19.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.961+0000 7f9062b74640 1 -- 172.21.15.37:0/1321856881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f905c1025a0 msgr2=0x7f905c102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.961+0000 7f9062b74640 1 --2- 172.21.15.37:0/1321856881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f905c1025a0 0x7f905c102a20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9050009930 tx=0x7f905002bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:19.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.961+0000 7f9062b74640 1 -- 172.21.15.37:0/1321856881 shutdown_connections 2024-03-26T06:22:19.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.961+0000 7f9062b74640 1 --2- 172.21.15.37:0/1321856881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c102f60 0x7f905c1097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.961+0000 7f9062b74640 1 --2- 172.21.15.37:0/1321856881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f905c1025a0 0x7f905c102a20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.961+0000 7f9062b74640 1 --2- 172.21.15.37:0/1321856881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f905c0ff5d0 0x7f905c0ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.961+0000 7f9062b74640 1 -- 172.21.15.37:0/1321856881 >> 172.21.15.37:0/1321856881 conn(0x7f905c0faee0 msgr2=0x7f905c0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.961+0000 7f9062b74640 1 -- 172.21.15.37:0/1321856881 shutdown_connections 2024-03-26T06:22:19.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.961+0000 7f9062b74640 1 -- 172.21.15.37:0/1321856881 wait complete. 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9062b74640 1 Processor -- start 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9062b74640 1 -- start start 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9062b74640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f905c0ff5d0 0x7f905c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9062b74640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f905c1025a0 0x7f905c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9062b74640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c102f60 0x7f905c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9062b74640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c10cfc0 con 0x7f905c0ff5d0 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9062b74640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c10ce40 con 0x7f905c102f60 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9062b74640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c10d140 con 0x7f905c1025a0 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9060910640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f905c0ff5d0 0x7f905c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9060910640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f905c0ff5d0 0x7f905c197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59646/0 (socket says 172.21.15.37:59646) 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9060910640 1 -- 172.21.15.37:0/4096697600 learned_addr learned my addr 172.21.15.37:0/4096697600 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f905bfff640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f905c1025a0 0x7f905c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9061111640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c102f60 0x7f905c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9060910640 1 -- 172.21.15.37:0/4096697600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f905c1025a0 msgr2=0x7f905c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9060910640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f905c1025a0 0x7f905c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9060910640 1 -- 172.21.15.37:0/4096697600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c102f60 msgr2=0x7f905c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9060910640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c102f60 0x7f905c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.962+0000 7f9060910640 1 -- 172.21.15.37:0/4096697600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c19f2a0 con 0x7f905c0ff5d0 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.963+0000 7f9061111640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c102f60 0x7f905c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.963+0000 7f9060910640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f905c0ff5d0 0x7f905c197680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f905400ccb0 tx=0x7f905400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.963+0000 7f9059ffb640 1 -- 172.21.15.37:0/4096697600 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054003990 con 0x7f905c0ff5d0 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.963+0000 7f9059ffb640 1 -- 172.21.15.37:0/4096697600 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9054003b30 con 0x7f905c0ff5d0 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.963+0000 7f9062b74640 1 -- 172.21.15.37:0/4096697600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c19f590 con 0x7f905c0ff5d0 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.963+0000 7f9059ffb640 1 -- 172.21.15.37:0/4096697600 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905400f7b0 con 0x7f905c0ff5d0 2024-03-26T06:22:19.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.963+0000 7f9062b74640 1 -- 172.21.15.37:0/4096697600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c19fab0 con 0x7f905c0ff5d0 2024-03-26T06:22:19.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.965+0000 7f9059ffb640 1 -- 172.21.15.37:0/4096697600 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f905400f950 con 0x7f905c0ff5d0 2024-03-26T06:22:19.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f906582f740 1 Processor -- start 2024-03-26T06:22:19.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f906582f740 1 -- start start 2024-03-26T06:22:19.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f9059ffb640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f90440a2800 0x7f90440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f9061111640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f90440a2800 0x7f90440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f906582f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c872fe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f906582f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c872fead0 0x556c873035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f906582f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c87303c40 0x556c873060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f906582f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c872637c0 con 0x556c87303c40 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f906582f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8720cb50 con 0x556c872fead0 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f906582f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8725f860 con 0x556c872fe0e0 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f9059ffb640 1 -- 172.21.15.37:0/4096697600 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f90540cebf0 con 0x7f905c0ff5d0 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f9061111640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c87303c40 0x556c873060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f9061111640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c87303c40 0x556c873060d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59654/0 (socket says 172.21.15.37:59654) 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f9061111640 1 -- 172.21.15.37:0/4057940834 learned_addr learned my addr 172.21.15.37:0/4057940834 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f905bfff640 1 --2- 172.21.15.37:0/4057940834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c872fead0 0x556c873035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f9060910640 1 --2- 172.21.15.37:0/4057940834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c872fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f9061111640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f90440a2800 0x7f90440a4cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f904c009fd0 tx=0x7f904c014830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f905bfff640 1 -- 172.21.15.37:0/4057940834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 msgr2=0x556c872fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f905bfff640 1 --2- 172.21.15.37:0/4057940834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c872fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f905bfff640 1 -- 172.21.15.37:0/4057940834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c87303c40 msgr2=0x556c873060d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.966+0000 7f905bfff640 1 --2- 172.21.15.37:0/4057940834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c87303c40 0x556c873060d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.967+0000 7f905bfff640 1 -- 172.21.15.37:0/4057940834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c872209a0 con 0x556c872fead0 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.967+0000 7f9061111640 1 --2- 172.21.15.37:0/4057940834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c87303c40 0x556c873060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.967+0000 7f905bfff640 1 --2- 172.21.15.37:0/4057940834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c872fead0 0x556c873035d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f905002c170 tx=0x7f905002c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62ecf10c57d87f2f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.967+0000 7f9042ffd640 1 -- 172.21.15.37:0/4057940834 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905002cb80 con 0x556c872fead0 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.967+0000 7f9042ffd640 1 -- 172.21.15.37:0/4057940834 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9050005040 con 0x556c872fead0 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.967+0000 7f9042ffd640 1 -- 172.21.15.37:0/4057940834 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90500049b0 con 0x556c872fead0 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 -- 172.21.15.37:0/4057940834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c872fead0 msgr2=0x556c873035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 --2- 172.21.15.37:0/4057940834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c872fead0 0x556c873035d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f905002c170 tx=0x7f905002c6d0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 -- 172.21.15.37:0/4057940834 shutdown_connections 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 --2- 172.21.15.37:0/4057940834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c87303c40 0x556c873060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 --2- 172.21.15.37:0/4057940834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c872fead0 0x556c873035d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 --2- 172.21.15.37:0/4057940834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c872fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 -- 172.21.15.37:0/4057940834 >> 172.21.15.37:0/4057940834 conn(0x556c872c6400 msgr2=0x556c872f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 -- 172.21.15.37:0/4057940834 shutdown_connections 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 -- 172.21.15.37:0/4057940834 wait complete. 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 -- start start 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 Processor -- start 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c87308590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c87308ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c8730fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c87306b90 con 0x556c872fead0 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8721ff90 con 0x556c87303c40 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.968+0000 7f906582f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c87306d10 con 0x556c872fe0e0 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f906582f740 10 client.0 _ll_register_callbacks cb 0x556c8725b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f905bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c87308ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f905bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c87308ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59670/0 (socket says 172.21.15.37:59670) 2024-03-26T06:22:19.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f905bfff640 1 -- 172.21.15.37:0/2744718761 learned_addr learned my addr 172.21.15.37:0/2744718761 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f9061111640 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c8730fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f9060910640 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c87308590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f905bfff640 1 -- 172.21.15.37:0/2744718761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 msgr2=0x556c87308590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f905bfff640 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c87308590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f905bfff640 1 -- 172.21.15.37:0/2744718761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 msgr2=0x556c8730fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f905bfff640 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c8730fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f905bfff640 1 -- 172.21.15.37:0/2744718761 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c86f782d0 con 0x556c872fead0 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f905bfff640 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c87308ad0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9050006010 tx=0x7f9050004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6062cd3deb977763 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f903bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f903bfff640 1 -- 172.21.15.37:0/2744718761 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050005d50 con 0x556c872fead0 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f906582f740 1 -- 172.21.15.37:0/2744718761 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9050005040 con 0x556c872fead0 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f903bfff640 1 -- 172.21.15.37:0/2744718761 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f90500053f0 con 0x556c872fead0 2024-03-26T06:22:19.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f903bfff640 1 -- 172.21.15.37:0/2744718761 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90500055f0 con 0x556c872fead0 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f906582f740 10 client.15153 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f906582f740 1 -- 172.21.15.37:0/2744718761 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556c870426b0 con 0x556c872fead0 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.969+0000 7f903bfff640 1 -- 172.21.15.37:0/2744718761 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9050005040 con 0x556c872fead0 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.970+0000 7f903bfff640 1 -- 172.21.15.37:0/2744718761 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9050003960 con 0x556c872fead0 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.970+0000 7f906582f740 10 client.15153 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.970+0000 7f906582f740 10 client.15153 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.970+0000 7f906582f740 20 client.15153 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.970+0000 7f906582f740 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x556c87314000 0x556c87319070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.970+0000 7f906582f740 4 client.15153 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.970+0000 7f906582f740 1 -- 172.21.15.37:0/2744718761 --> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c87270d00 con 0x556c87314000 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.970+0000 7f9060910640 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x556c87314000 0x556c87319070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.970+0000 7f9060910640 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x556c87314000 0x556c87319070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76ef0f76b6061fa4 server_cookie=1064021669fa7078 in_seq=0 out_seq=0 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.970+0000 7f903bfff640 10 client.15153 ms_handle_connect on v2:172.21.15.66:6836/3870845253 2024-03-26T06:22:19.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f903bfff640 1 -- 172.21.15.37:0/2744718761 <== mds.1 v2:172.21.15.66:6836/3870845253 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c87270d00 con 0x556c87314000 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f903bfff640 10 client.15153 handle_command_reply: tid=0 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f906582f740 1 client.15153 shutdown 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f906582f740 1 -- 172.21.15.37:0/2744718761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 msgr2=0x556c87308ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f906582f740 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c87308ad0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9050006010 tx=0x7f9050004250 comp rx=0 tx=0).stop 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f906582f740 1 -- 172.21.15.37:0/2744718761 shutdown_connections 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f906582f740 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c8730fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f906582f740 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c87308ad0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f906582f740 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x556c87314000 0x556c87319070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f906582f740 1 --2- 172.21.15.37:0/2744718761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c87308590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f903bfff640 0 client.15153 ms_handle_reset on v2:172.21.15.66:6836/3870845253 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.971+0000 7f906582f740 1 -- 172.21.15.37:0/2744718761 >> 172.21.15.37:0/2744718761 conn(0x556c872c6400 msgr2=0x556c872f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.972+0000 7f906582f740 1 -- 172.21.15.37:0/2744718761 shutdown_connections 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.972+0000 7f906582f740 1 -- 172.21.15.37:0/2744718761 wait complete. 2024-03-26T06:22:19.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.972+0000 7f906582f740 20 client.15153 trim_cache size 0 max 16384 2024-03-26T06:22:19.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.987+0000 7f906582f740 1 Processor -- start 2024-03-26T06:22:19.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.987+0000 7f906582f740 1 -- start start 2024-03-26T06:22:19.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.987+0000 7f906582f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fe0e0 0x556c8731d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.987+0000 7f906582f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fead0 0x556c8730fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.987+0000 7f906582f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c872fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.987+0000 7f906582f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c87314410 con 0x556c872fe0e0 2024-03-26T06:22:19.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.987+0000 7f906582f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c87314a70 con 0x556c87303c40 2024-03-26T06:22:19.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.987+0000 7f906582f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c873150d0 con 0x556c872fead0 2024-03-26T06:22:19.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.987+0000 7f9060910640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fe0e0 0x556c8731d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.987+0000 7f9060910640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fe0e0 0x556c8731d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59682/0 (socket says 172.21.15.37:59682) 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.987+0000 7f9060910640 1 -- 172.21.15.37:0/920194288 learned_addr learned my addr 172.21.15.37:0/920194288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f9061111640 1 --2- 172.21.15.37:0/920194288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c872fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f9060910640 1 -- 172.21.15.37:0/920194288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fead0 msgr2=0x556c8730fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f9060910640 1 --2- 172.21.15.37:0/920194288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fead0 0x556c8730fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f9060910640 1 -- 172.21.15.37:0/920194288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 msgr2=0x556c872fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f9060910640 1 --2- 172.21.15.37:0/920194288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c872fcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f9060910640 1 -- 172.21.15.37:0/920194288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c872f9ff0 con 0x556c872fe0e0 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f9061111640 1 --2- 172.21.15.37:0/920194288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c872fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f9060910640 1 --2- 172.21.15.37:0/920194288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fe0e0 0x556c8731d7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f905409b530 tx=0x7f90540076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2298f68a468a13c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f90427fc640 1 -- 172.21.15.37:0/920194288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905401d040 con 0x556c872fe0e0 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f90427fc640 1 -- 172.21.15.37:0/920194288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90540998a0 con 0x556c872fe0e0 2024-03-26T06:22:19.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f90427fc640 1 -- 172.21.15.37:0/920194288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540a38b0 con 0x556c872fe0e0 2024-03-26T06:22:19.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f906582f740 1 -- 172.21.15.37:0/920194288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fe0e0 msgr2=0x556c8731d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:19.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f906582f740 1 --2- 172.21.15.37:0/920194288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fe0e0 0x556c8731d7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f905409b530 tx=0x7f90540076b0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f906582f740 1 -- 172.21.15.37:0/920194288 shutdown_connections 2024-03-26T06:22:19.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f906582f740 1 --2- 172.21.15.37:0/920194288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c872fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f906582f740 1 --2- 172.21.15.37:0/920194288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fead0 0x556c8730fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f906582f740 1 --2- 172.21.15.37:0/920194288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fe0e0 0x556c8731d7a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:19.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f906582f740 1 -- 172.21.15.37:0/920194288 >> 172.21.15.37:0/920194288 conn(0x556c872c6400 msgr2=0x556c8731d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:19.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f906582f740 1 -- 172.21.15.37:0/920194288 shutdown_connections 2024-03-26T06:22:19.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.988+0000 7f906582f740 1 -- 172.21.15.37:0/920194288 wait complete. 2024-03-26T06:22:19.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f906582f740 1 -- start start 2024-03-26T06:22:19.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f906582f740 1 Processor -- start 2024-03-26T06:22:19.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f906582f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c8730f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:19.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f906582f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c8730fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f906582f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c8730c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f906582f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c87315d90 con 0x556c872fead0 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f906582f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c87315730 con 0x556c87303c40 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f906582f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c873163f0 con 0x556c872fe0e0 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f906582f740 10 client.0 _ll_register_callbacks cb 0x7f904c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f905bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c8730fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f905bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c8730fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59686/0 (socket says 172.21.15.37:59686) 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f905bfff640 1 -- 172.21.15.37:0/218863607 learned_addr learned my addr 172.21.15.37:0/218863607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f9061111640 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c8730c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f9060910640 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c8730f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f905bfff640 1 -- 172.21.15.37:0/218863607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 msgr2=0x556c8730f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f905bfff640 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c8730f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f905bfff640 1 -- 172.21.15.37:0/218863607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 msgr2=0x556c8730c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f905bfff640 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c8730c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f905bfff640 1 -- 172.21.15.37:0/218863607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c8730c8d0 con 0x556c872fead0 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.989+0000 7f9061111640 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c8730c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f905bfff640 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c8730fa90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f905002c320 tx=0x7f90500095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00418df59ed0561 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f903bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f903bfff640 1 -- 172.21.15.37:0/218863607 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050000e60 con 0x556c872fead0 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f903bfff640 1 -- 172.21.15.37:0/218863607 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f90500035f0 con 0x556c872fead0 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f906582f740 1 -- 172.21.15.37:0/218863607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90540998a0 con 0x556c872fead0 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f903bfff640 1 -- 172.21.15.37:0/218863607 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9050003810 con 0x556c872fead0 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f906582f740 10 client.15159 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f906582f740 1 -- 172.21.15.37:0/218863607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556c8736ff00 con 0x556c872fead0 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f903bfff640 1 -- 172.21.15.37:0/218863607 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90540998a0 con 0x556c872fead0 2024-03-26T06:22:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f903bfff640 1 -- 172.21.15.37:0/218863607 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9050060030 con 0x556c872fead0 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f906582f740 10 client.15159 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f906582f740 10 client.15159 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f906582f740 20 client.15159 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f906582f740 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x556c87374550 0x556c873701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f906582f740 4 client.15159 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.990+0000 7f906582f740 1 -- 172.21.15.37:0/218863607 --> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556c8726b050 con 0x556c87374550 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f9060910640 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x556c87374550 0x556c873701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f903bfff640 10 client.15159 ms_handle_connect on v2:172.21.15.66:6836/3870845253 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f9060910640 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x556c87374550 0x556c873701f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52aed6edde874635 server_cookie=100c02d2c8ed6b0 in_seq=0 out_seq=0 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f903bfff640 1 -- 172.21.15.37:0/218863607 <== mds.1 v2:172.21.15.66:6836/3870845253 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556c8726b050 con 0x556c87374550 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f903bfff640 10 client.15159 handle_command_reply: tid=0 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f906582f740 1 client.15159 shutdown 2024-03-26T06:22:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f906582f740 1 -- 172.21.15.37:0/218863607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 msgr2=0x556c8730fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f906582f740 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c8730fa90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f905002c320 tx=0x7f90500095e0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f906582f740 1 -- 172.21.15.37:0/218863607 shutdown_connections 2024-03-26T06:22:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f906582f740 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c87303c40 0x556c8730c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f906582f740 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556c872fead0 0x556c8730fa90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f906582f740 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x556c87374550 0x556c873701f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f906582f740 1 --2- 172.21.15.37:0/218863607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c872fe0e0 0x556c8730f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f903bfff640 0 client.15159 ms_handle_reset on v2:172.21.15.66:6836/3870845253 2024-03-26T06:22:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f906582f740 1 -- 172.21.15.37:0/218863607 >> 172.21.15.37:0/218863607 conn(0x556c872c6400 msgr2=0x556c8731d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.991+0000 7f906582f740 1 -- 172.21.15.37:0/218863607 shutdown_connections 2024-03-26T06:22:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f906582f740 1 -- 172.21.15.37:0/218863607 wait complete. 2024-03-26T06:22:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f906582f740 20 client.15159 trim_cache size 0 max 16384 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24494, 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 83.398609137999998, 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 87, 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 101.82266322300001 2024-03-26T06:22:20.004 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:22:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 -- 172.21.15.37:0/4096697600 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f90440a2800 msgr2=0x7f90440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f90440a2800 0x7f90440a4cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f904c009fd0 tx=0x7f904c014830 comp rx=0 tx=0).stop 2024-03-26T06:22:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 -- 172.21.15.37:0/4096697600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f905c0ff5d0 msgr2=0x7f905c197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f905c0ff5d0 0x7f905c197680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f905400ccb0 tx=0x7f905400a620 comp rx=0 tx=0).stop 2024-03-26T06:22:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 -- 172.21.15.37:0/4096697600 shutdown_connections 2024-03-26T06:22:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f90440a2800 0x7f90440a4cc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f905c102f60 0x7f905c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f905c1025a0 0x7f905c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 --2- 172.21.15.37:0/4096697600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f905c0ff5d0 0x7f905c197680 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 -- 172.21.15.37:0/4096697600 >> 172.21.15.37:0/4096697600 conn(0x7f905c0faee0 msgr2=0x7f905c105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 -- 172.21.15.37:0/4096697600 shutdown_connections 2024-03-26T06:22:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:19.992+0000 7f9062b74640 1 -- 172.21.15.37:0/4096697600 wait complete. 2024-03-26T06:22:20.007 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T06:22:20.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 -- 172.21.15.37:0/300688837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34b81046c0 msgr2=0x7f34b8106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 --2- 172.21.15.37:0/300688837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34b81046c0 0x7f34b8106ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009900 tx=0x7f34a802bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:20.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 -- 172.21.15.37:0/300688837 shutdown_connections 2024-03-26T06:22:20.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 --2- 172.21.15.37:0/300688837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b8106ff0 0x7f34b8109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 --2- 172.21.15.37:0/300688837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34b81046c0 0x7f34b8106ab0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 --2- 172.21.15.37:0/300688837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34b8101d90 0x7f34b8104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 -- 172.21.15.37:0/300688837 >> 172.21.15.37:0/300688837 conn(0x7f34b80fb7a0 msgr2=0x7f34b80fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:20.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 -- 172.21.15.37:0/300688837 shutdown_connections 2024-03-26T06:22:20.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 -- 172.21.15.37:0/300688837 wait complete. 2024-03-26T06:22:20.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 Processor -- start 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 -- start start 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34b8101d90 0x7f34b8197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34b81046c0 0x7f34b8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b8106ff0 0x7f34b819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b810cfa0 con 0x7f34b81046c0 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b810ce20 con 0x7f34b8101d90 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.111+0000 7f34c0268640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b810d120 con 0x7f34b8106ff0 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34bd803640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34b81046c0 0x7f34b8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34bd803640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34b81046c0 0x7f34b8197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59700/0 (socket says 172.21.15.37:59700) 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34bd803640 1 -- 172.21.15.37:0/2159899567 learned_addr learned my addr 172.21.15.37:0/2159899567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34be004640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34b8101d90 0x7f34b8197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34bd803640 1 -- 172.21.15.37:0/2159899567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b8106ff0 msgr2=0x7f34b819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34bd803640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b8106ff0 0x7f34b819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34bd803640 1 -- 172.21.15.37:0/2159899567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34b8101d90 msgr2=0x7f34b8197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34bd803640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34b8101d90 0x7f34b8197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34bd803640 1 -- 172.21.15.37:0/2159899567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b819f250 con 0x7f34b81046c0 2024-03-26T06:22:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34be004640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34b8101d90 0x7f34b8197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34bd803640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34b81046c0 0x7f34b8197b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f34a8000c00 tx=0x7f34a8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34af7fe640 1 -- 172.21.15.37:0/2159899567 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8041070 con 0x7f34b81046c0 2024-03-26T06:22:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34af7fe640 1 -- 172.21.15.37:0/2159899567 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34a80044d0 con 0x7f34b81046c0 2024-03-26T06:22:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34c0268640 1 -- 172.21.15.37:0/2159899567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b819f4e0 con 0x7f34b81046c0 2024-03-26T06:22:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34af7fe640 1 -- 172.21.15.37:0/2159899567 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8034440 con 0x7f34b81046c0 2024-03-26T06:22:20.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.112+0000 7f34c0268640 1 -- 172.21.15.37:0/2159899567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b819f9d0 con 0x7f34b81046c0 2024-03-26T06:22:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.114+0000 7f34af7fe640 1 -- 172.21.15.37:0/2159899567 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f34a8044070 con 0x7f34b81046c0 2024-03-26T06:22:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.115+0000 7f34c2e47740 1 Processor -- start 2024-03-26T06:22:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.115+0000 7f34c2e47740 1 -- start start 2024-03-26T06:22:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.115+0000 7f34af7fe640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f34a40a2800 0x7f34a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.115+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f43614f0 0x55f8f4361910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.115+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4361ee0 0x55f8f43669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f4367050 0x55f8f43694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34c2e47740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f42c27c0 con 0x55f8f43614f0 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34c2e47740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f426bb50 con 0x55f8f4367050 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34c2e47740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f42be860 con 0x55f8f4361ee0 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34be004640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f43614f0 0x55f8f4361910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34af7fe640 1 -- 172.21.15.37:0/2159899567 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f34a80f0700 con 0x7f34b81046c0 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34be805640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f34a40a2800 0x7f34a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34be004640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f43614f0 0x55f8f4361910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59702/0 (socket says 172.21.15.37:59702) 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34be004640 1 -- 172.21.15.37:0/2243134665 learned_addr learned my addr 172.21.15.37:0/2243134665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34bd803640 1 --2- 172.21.15.37:0/2243134665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4361ee0 0x55f8f43669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34be004640 1 -- 172.21.15.37:0/2243134665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4361ee0 msgr2=0x55f8f43669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34be004640 1 --2- 172.21.15.37:0/2243134665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4361ee0 0x55f8f43669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34be004640 1 -- 172.21.15.37:0/2243134665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f4367050 msgr2=0x55f8f43694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34be004640 1 --2- 172.21.15.37:0/2243134665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f4367050 0x55f8f43694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34be004640 1 -- 172.21.15.37:0/2243134665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8f427f9a0 con 0x55f8f43614f0 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34be004640 1 --2- 172.21.15.37:0/2243134665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f43614f0 0x55f8f4361910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009c30 tx=0x7f34b4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad99f1ac752de0a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34be805640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f34a40a2800 0x7f34a40a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009270 tx=0x7f34b0009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34acbfe640 1 -- 172.21.15.37:0/2243134665 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4004020 con 0x55f8f43614f0 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34acbfe640 1 -- 172.21.15.37:0/2243134665 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34b40041c0 con 0x55f8f43614f0 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.116+0000 7f34acbfe640 1 -- 172.21.15.37:0/2243134665 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4012650 con 0x55f8f43614f0 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.117+0000 7f34c2e47740 1 -- 172.21.15.37:0/2243134665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f43614f0 msgr2=0x55f8f4361910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.117+0000 7f34c2e47740 1 --2- 172.21.15.37:0/2243134665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f43614f0 0x55f8f4361910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009c30 tx=0x7f34b4010740 comp rx=0 tx=0).stop 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.117+0000 7f34c2e47740 1 -- 172.21.15.37:0/2243134665 shutdown_connections 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.117+0000 7f34c2e47740 1 --2- 172.21.15.37:0/2243134665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f4367050 0x55f8f43694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.117+0000 7f34c2e47740 1 --2- 172.21.15.37:0/2243134665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4361ee0 0x55f8f43669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.117+0000 7f34c2e47740 1 --2- 172.21.15.37:0/2243134665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f43614f0 0x55f8f4361910 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.117+0000 7f34c2e47740 1 -- 172.21.15.37:0/2243134665 >> 172.21.15.37:0/2243134665 conn(0x55f8f4325400 msgr2=0x55f8f4358a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.117+0000 7f34c2e47740 1 -- 172.21.15.37:0/2243134665 shutdown_connections 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.117+0000 7f34c2e47740 1 -- 172.21.15.37:0/2243134665 wait complete. 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.117+0000 7f34c2e47740 1 -- start start 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.117+0000 7f34c2e47740 1 Processor -- start 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f43614f0 0x55f8f436b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4361ee0 0x55f8f436beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f4372ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34c2e47740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f436a000 con 0x55f8f4367050 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34c2e47740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f427ef90 con 0x55f8f43614f0 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34c2e47740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f436a180 con 0x55f8f4361ee0 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34c2e47740 10 client.0 _ll_register_callbacks cb 0x55f8f42ba400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34be805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f4372ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34be805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f4372ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59718/0 (socket says 172.21.15.37:59718) 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34be805640 1 -- 172.21.15.37:0/2115133242 learned_addr learned my addr 172.21.15.37:0/2115133242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34bd803640 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4361ee0 0x55f8f436beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34be004640 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f43614f0 0x55f8f436b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34be805640 1 -- 172.21.15.37:0/2115133242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4361ee0 msgr2=0x55f8f436beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34be805640 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4361ee0 0x55f8f436beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34be805640 1 -- 172.21.15.37:0/2115133242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f43614f0 msgr2=0x55f8f436b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34be805640 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f43614f0 0x55f8f436b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34be805640 1 -- 172.21.15.37:0/2115133242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8f3fd72d0 con 0x55f8f4367050 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34be805640 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f4372ef0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f34b00096a0 tx=0x7f34b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a713a1b957841d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34a1ffb640 1 -- 172.21.15.37:0/2115133242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0023020 con 0x55f8f4367050 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34a1ffb640 1 -- 172.21.15.37:0/2115133242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f34b001f860 con 0x55f8f4367050 2024-03-26T06:22:20.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.118+0000 7f34c2e47740 1 -- 172.21.15.37:0/2115133242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34b40041c0 con 0x55f8f4367050 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34a1ffb640 1 -- 172.21.15.37:0/2115133242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34b001fa60 con 0x55f8f4367050 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34c2e47740 10 client.15171 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34c2e47740 1 -- 172.21.15.37:0/2115133242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f8f40a16b0 con 0x55f8f4367050 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34a1ffb640 1 -- 172.21.15.37:0/2115133242 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34b40041c0 con 0x55f8f4367050 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34a1ffb640 1 -- 172.21.15.37:0/2115133242 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f34b001e3b0 con 0x55f8f4367050 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34c2e47740 10 client.15171 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34c2e47740 10 client.15171 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34c2e47740 20 client.15171 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34c2e47740 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f8f43785c0 0x55f8f43775d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34c2e47740 4 client.15171 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34c2e47740 1 -- 172.21.15.37:0/2115133242 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8f42cfd00 con 0x55f8f43785c0 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.119+0000 7f34be004640 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f8f43785c0 0x55f8f43775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.120+0000 7f34a1ffb640 10 client.15171 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:22:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.120+0000 7f34be004640 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f8f43785c0 0x55f8f43775d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf992f94c5a86f8 server_cookie=820ab0c10f8df75 in_seq=0 out_seq=0 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.120+0000 7f34a1ffb640 1 -- 172.21.15.37:0/2115133242 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f8f42cfd00 con 0x55f8f43785c0 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.120+0000 7f34a1ffb640 10 client.15171 handle_command_reply: tid=0 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.120+0000 7f34c2e47740 1 client.15171 shutdown 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.120+0000 7f34c2e47740 1 -- 172.21.15.37:0/2115133242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 msgr2=0x55f8f4372ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.120+0000 7f34c2e47740 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f4372ef0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f34b00096a0 tx=0x7f34b0004a40 comp rx=0 tx=0).stop 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.121+0000 7f34c2e47740 1 -- 172.21.15.37:0/2115133242 shutdown_connections 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.121+0000 7f34c2e47740 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f8f43785c0 0x55f8f43775d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.121+0000 7f34c2e47740 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f4372ef0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.121+0000 7f34c2e47740 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4361ee0 0x55f8f436beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.121+0000 7f34c2e47740 1 --2- 172.21.15.37:0/2115133242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f43614f0 0x55f8f436b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.121+0000 7f34c2e47740 1 -- 172.21.15.37:0/2115133242 >> 172.21.15.37:0/2115133242 conn(0x55f8f4325400 msgr2=0x55f8f4357c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.121+0000 7f34a1ffb640 0 client.15171 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.121+0000 7f34c2e47740 1 -- 172.21.15.37:0/2115133242 shutdown_connections 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.121+0000 7f34c2e47740 1 -- 172.21.15.37:0/2115133242 wait complete. 2024-03-26T06:22:20.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.121+0000 7f34c2e47740 20 client.15171 trim_cache size 0 max 16384 2024-03-26T06:22:20.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.136+0000 7f34c2e47740 1 Processor -- start 2024-03-26T06:22:20.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.136+0000 7f34c2e47740 1 -- start start 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.136+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f43614f0 0x55f8f4380a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.136+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4361ee0 0x55f8f4372e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.136+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4367050 0x55f8f436b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.136+0000 7f34c2e47740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f4377de0 con 0x55f8f4361ee0 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.136+0000 7f34c2e47740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f4378440 con 0x55f8f43614f0 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.136+0000 7f34c2e47740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f4378aa0 con 0x55f8f4367050 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34bd803640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4361ee0 0x55f8f4372e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34bd803640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4361ee0 0x55f8f4372e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59734/0 (socket says 172.21.15.37:59734) 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34bd803640 1 -- 172.21.15.37:0/1075209560 learned_addr learned my addr 172.21.15.37:0/1075209560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34bd803640 1 -- 172.21.15.37:0/1075209560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4367050 msgr2=0x55f8f436b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34bd803640 1 --2- 172.21.15.37:0/1075209560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4367050 0x55f8f436b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34bd803640 1 -- 172.21.15.37:0/1075209560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f43614f0 msgr2=0x55f8f4380a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34be004640 1 --2- 172.21.15.37:0/1075209560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f43614f0 0x55f8f4380a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34bd803640 1 --2- 172.21.15.37:0/1075209560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f43614f0 0x55f8f4380a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34bd803640 1 -- 172.21.15.37:0/1075209560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8f43622c0 con 0x55f8f4361ee0 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34be004640 1 --2- 172.21.15.37:0/1075209560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f43614f0 0x55f8f4380a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34bd803640 1 --2- 172.21.15.37:0/1075209560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4361ee0 0x55f8f4372e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f34a80ee1b0 tx=0x7f34a8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cd383793195aba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34a3fff640 1 -- 172.21.15.37:0/1075209560 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a804b040 con 0x55f8f4361ee0 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34a3fff640 1 -- 172.21.15.37:0/1075209560 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34a80bd750 con 0x55f8f4361ee0 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34a3fff640 1 -- 172.21.15.37:0/1075209560 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8041d30 con 0x55f8f4361ee0 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34c2e47740 1 -- 172.21.15.37:0/1075209560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4361ee0 msgr2=0x55f8f4372e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.137+0000 7f34c2e47740 1 --2- 172.21.15.37:0/1075209560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4361ee0 0x55f8f4372e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f34a80ee1b0 tx=0x7f34a8009680 comp rx=0 tx=0).stop 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 -- 172.21.15.37:0/1075209560 shutdown_connections 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 --2- 172.21.15.37:0/1075209560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f4367050 0x55f8f436b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 --2- 172.21.15.37:0/1075209560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4361ee0 0x55f8f4372e10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 --2- 172.21.15.37:0/1075209560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f43614f0 0x55f8f4380a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 -- 172.21.15.37:0/1075209560 >> 172.21.15.37:0/1075209560 conn(0x55f8f4325400 msgr2=0x55f8f436a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 -- 172.21.15.37:0/1075209560 shutdown_connections 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 -- 172.21.15.37:0/1075209560 wait complete. 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 -- start start 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 Processor -- start 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f43614f0 0x55f8f436fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f4361ee0 0x55f8f43703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f43ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f4379760 con 0x55f8f4367050 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f4379100 con 0x55f8f4361ee0 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8f4379dc0 con 0x55f8f43614f0 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34c2e47740 10 client.0 _ll_register_callbacks cb 0x7f34b40095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34be805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f43ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34be805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f43ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59736/0 (socket says 172.21.15.37:59736) 2024-03-26T06:22:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34be805640 1 -- 172.21.15.37:0/3138789336 learned_addr learned my addr 172.21.15.37:0/3138789336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34be004640 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f43614f0 0x55f8f436fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34bd803640 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f4361ee0 0x55f8f43703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34be805640 1 -- 172.21.15.37:0/3138789336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f43614f0 msgr2=0x55f8f436fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34be805640 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f43614f0 0x55f8f436fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34be805640 1 -- 172.21.15.37:0/3138789336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f4361ee0 msgr2=0x55f8f43703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34be805640 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f4361ee0 0x55f8f43703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.138+0000 7f34be805640 1 -- 172.21.15.37:0/3138789336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8f43ce700 con 0x55f8f4367050 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34be004640 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f43614f0 0x55f8f436fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34bd803640 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f4361ee0 0x55f8f43703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34be805640 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f43ce1c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f34b002c040 tx=0x7f34b0009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7122a5f16fb955 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34a1ffb640 1 -- 172.21.15.37:0/3138789336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0023040 con 0x55f8f4367050 2024-03-26T06:22:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34a1ffb640 1 -- 172.21.15.37:0/3138789336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f34b001f660 con 0x55f8f4367050 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34c2e47740 1 -- 172.21.15.37:0/3138789336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34a80bd750 con 0x55f8f4367050 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34a1ffb640 1 -- 172.21.15.37:0/3138789336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34b001f880 con 0x55f8f4367050 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34c2e47740 10 client.15177 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34c2e47740 1 -- 172.21.15.37:0/3138789336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f8f43cf970 con 0x55f8f4367050 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34a1ffb640 1 -- 172.21.15.37:0/3138789336 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34a80bd750 con 0x55f8f4367050 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34a1ffb640 1 -- 172.21.15.37:0/3138789336 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f34b003c030 con 0x55f8f4367050 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34c2e47740 10 client.15177 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34c2e47740 10 client.15177 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34c2e47740 20 client.15177 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34c2e47740 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f8f43d4090 0x55f8f43cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34c2e47740 4 client.15177 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.139+0000 7f34c2e47740 1 -- 172.21.15.37:0/3138789336 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f8f42ca050 con 0x55f8f43d4090 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34be004640 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f8f43d4090 0x55f8f43cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34be004640 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f8f43d4090 0x55f8f43cfc60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b26e0b81e3d7777 server_cookie=52b10d8d811eaed2 in_seq=0 out_seq=0 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34a1ffb640 10 client.15177 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34a1ffb640 1 -- 172.21.15.37:0/3138789336 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f8f42ca050 con 0x55f8f43d4090 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34a1ffb640 10 client.15177 handle_command_reply: tid=0 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 1 client.15177 shutdown 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 1 -- 172.21.15.37:0/3138789336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 msgr2=0x55f8f43ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f43ce1c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f34b002c040 tx=0x7f34b0009c50 comp rx=0 tx=0).stop 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 1 -- 172.21.15.37:0/3138789336 shutdown_connections 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f8f43d4090 0x55f8f43cfc60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8f4367050 0x55f8f43ce1c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f8f4361ee0 0x55f8f43703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34a1ffb640 0 client.15177 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 1 --2- 172.21.15.37:0/3138789336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f43614f0 0x55f8f436fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 1 -- 172.21.15.37:0/3138789336 >> 172.21.15.37:0/3138789336 conn(0x55f8f4325400 msgr2=0x55f8f4380a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 1 -- 172.21.15.37:0/3138789336 shutdown_connections 2024-03-26T06:22:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 1 -- 172.21.15.37:0/3138789336 wait complete. 2024-03-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.140+0000 7f34c2e47740 20 client.15177 trim_cache size 0 max 16384 2024-03-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-03-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24545, 2024-03-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 85.869839733999996, 2024-03-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-03-26T06:22:20.149 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 87, 2024-03-26T06:22:20.150 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T06:22:20.150 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 91.010112981000006 2024-03-26T06:22:20.150 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:22:20.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 -- 172.21.15.37:0/2159899567 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f34a40a2800 msgr2=0x7f34a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f34a40a2800 0x7f34a40a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009270 tx=0x7f34b0009760 comp rx=0 tx=0).stop 2024-03-26T06:22:20.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 -- 172.21.15.37:0/2159899567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34b81046c0 msgr2=0x7f34b8197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:20.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34b81046c0 0x7f34b8197b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f34a8000c00 tx=0x7f34a8005d50 comp rx=0 tx=0).stop 2024-03-26T06:22:20.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 -- 172.21.15.37:0/2159899567 shutdown_connections 2024-03-26T06:22:20.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f34a40a2800 0x7f34a40a4cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b8106ff0 0x7f34b819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34b81046c0 0x7f34b8197b70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 --2- 172.21.15.37:0/2159899567 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34b8101d90 0x7f34b8197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:20.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 -- 172.21.15.37:0/2159899567 >> 172.21.15.37:0/2159899567 conn(0x7f34b80fb7a0 msgr2=0x7f34b8102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:20.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 -- 172.21.15.37:0/2159899567 shutdown_connections 2024-03-26T06:22:20.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:20.141+0000 7f34c0268640 1 -- 172.21.15.37:0/2159899567 wait complete. 2024-03-26T06:22:20.153 INFO:tasks.cephadm:Setup complete, yielding 2024-03-26T06:22:20.153 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T06:22:20.163 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-26T06:22:20.163 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-26T06:22:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:22:20.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:20 smithi037 ceph-mon[31974]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-26T06:22:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:20 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1987315539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:20.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:20 smithi066 ceph-mon[36504]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-26T06:22:20.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:20 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1987315539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:20.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:20 smithi149 ceph-mon[36681]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-26T06:22:20.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:20 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1987315539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:22.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.052+0000 7f65726d7700 1 -- 172.21.15.37:0/125588683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c100d10 msgr2=0x7f656c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:22.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.052+0000 7f65726d7700 1 --2- 172.21.15.37:0/125588683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c100d10 0x7f656c101130 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f65540099c0 tx=0x7f6554009cd0 comp rx=0 tx=0).stop 2024-03-26T06:22:22.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.052+0000 7f65726d7700 1 -- 172.21.15.37:0/125588683 shutdown_connections 2024-03-26T06:22:22.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.052+0000 7f65726d7700 1 --2- 172.21.15.37:0/125588683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c100d10 0x7f656c101130 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.052+0000 7f65726d7700 1 --2- 172.21.15.37:0/125588683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f656c1028b0 0x7f656c109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.052+0000 7f65726d7700 1 --2- 172.21.15.37:0/125588683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656c101f10 0x7f656c102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.052+0000 7f65726d7700 1 -- 172.21.15.37:0/125588683 >> 172.21.15.37:0/125588683 conn(0x7f656c0fc2d0 msgr2=0x7f656c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.052+0000 7f65726d7700 1 -- 172.21.15.37:0/125588683 shutdown_connections 2024-03-26T06:22:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.052+0000 7f65726d7700 1 -- 172.21.15.37:0/125588683 wait complete. 2024-03-26T06:22:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f65726d7700 1 Processor -- start 2024-03-26T06:22:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f65726d7700 1 -- start start 2024-03-26T06:22:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f65726d7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f656c100d10 0x7f656c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f65726d7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656c101f10 0x7f656c1974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f65726d7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c1028b0 0x7f656c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f65726d7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c10c950 con 0x7f656c1028b0 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f65726d7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c10c7d0 con 0x7f656c100d10 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f65726d7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c10cad0 con 0x7f656c101f10 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f6570c74700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c1028b0 0x7f656c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f6570c74700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c1028b0 0x7f656c19e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59750/0 (socket says 172.21.15.37:59750) 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f6570c74700 1 -- 172.21.15.37:0/1082843464 learned_addr learned my addr 172.21.15.37:0/1082843464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f656bfff700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f656c100d10 0x7f656c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f656b7fe700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656c101f10 0x7f656c1974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f6570c74700 1 -- 172.21.15.37:0/1082843464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656c101f10 msgr2=0x7f656c1974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f6570c74700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656c101f10 0x7f656c1974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f6570c74700 1 -- 172.21.15.37:0/1082843464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f656c100d10 msgr2=0x7f656c196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f6570c74700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f656c100d10 0x7f656c196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.053+0000 7f6570c74700 1 -- 172.21.15.37:0/1082843464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c19ebf0 con 0x7f656c1028b0 2024-03-26T06:22:22.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.054+0000 7f656bfff700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f656c100d10 0x7f656c196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:22.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.054+0000 7f656b7fe700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656c101f10 0x7f656c1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:22.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.054+0000 7f6570c74700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c1028b0 0x7f656c19e550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f656400b9a0 tx=0x7f656400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:22.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.054+0000 7f65697fa700 1 -- 172.21.15.37:0/1082843464 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564019070 con 0x7f656c1028b0 2024-03-26T06:22:22.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.054+0000 7f65697fa700 1 -- 172.21.15.37:0/1082843464 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6564014440 con 0x7f656c1028b0 2024-03-26T06:22:22.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.054+0000 7f65726d7700 1 -- 172.21.15.37:0/1082843464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c19eee0 con 0x7f656c1028b0 2024-03-26T06:22:22.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.054+0000 7f65697fa700 1 -- 172.21.15.37:0/1082843464 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65640075d0 con 0x7f656c1028b0 2024-03-26T06:22:22.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.054+0000 7f65726d7700 1 -- 172.21.15.37:0/1082843464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c19f3c0 con 0x7f656c1028b0 2024-03-26T06:22:22.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.055+0000 7f65726d7700 1 -- 172.21.15.37:0/1082843464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c068aa0 con 0x7f656c1028b0 2024-03-26T06:22:22.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.056+0000 7f65697fa700 1 -- 172.21.15.37:0/1082843464 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f656401c070 con 0x7f656c1028b0 2024-03-26T06:22:22.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.057+0000 7f65697fa700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f65580a2890 0x7f65580a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:22.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.057+0000 7f656bfff700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f65580a2890 0x7f65580a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:22.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.057+0000 7f65697fa700 1 -- 172.21.15.37:0/1082843464 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f65640cde80 con 0x7f656c1028b0 2024-03-26T06:22:22.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.057+0000 7f656bfff700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f65580a2890 0x7f65580a4d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6554006010 tx=0x7f6554005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.061+0000 7f65697fa700 1 -- 172.21.15.37:0/1082843464 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f656409afc0 con 0x7f656c1028b0 2024-03-26T06:22:22.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.286+0000 7f65726d7700 1 -- 172.21.15.37:0/1082843464 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f656c062220 con 0x7f65580a2890 2024-03-26T06:22:22.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.287+0000 7f65697fa700 1 -- 172.21.15.37:0/1082843464 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f656c062220 con 0x7f65580a2890 2024-03-26T06:22:22.291 INFO:teuthology.orchestra.run.smithi037.stdout:Backend: cephadm 2024-03-26T06:22:22.291 INFO:teuthology.orchestra.run.smithi037.stdout:Available: Yes 2024-03-26T06:22:22.291 INFO:teuthology.orchestra.run.smithi037.stdout:Paused: No 2024-03-26T06:22:22.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.289+0000 7f65726d7700 1 -- 172.21.15.37:0/1082843464 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f65580a2890 msgr2=0x7f65580a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:22.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.289+0000 7f65726d7700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f65580a2890 0x7f65580a4d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6554006010 tx=0x7f6554005c20 comp rx=0 tx=0).stop 2024-03-26T06:22:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.289+0000 7f65726d7700 1 -- 172.21.15.37:0/1082843464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c1028b0 msgr2=0x7f656c19e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.289+0000 7f65726d7700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c1028b0 0x7f656c19e550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f656400b9a0 tx=0x7f656400bd60 comp rx=0 tx=0).stop 2024-03-26T06:22:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.290+0000 7f65726d7700 1 -- 172.21.15.37:0/1082843464 shutdown_connections 2024-03-26T06:22:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.290+0000 7f65726d7700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c1028b0 0x7f656c19e550 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.290+0000 7f65726d7700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f65580a2890 0x7f65580a4d50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.290+0000 7f65726d7700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f656c100d10 0x7f656c196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.290+0000 7f65726d7700 1 --2- 172.21.15.37:0/1082843464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656c101f10 0x7f656c1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.290+0000 7f65726d7700 1 -- 172.21.15.37:0/1082843464 >> 172.21.15.37:0/1082843464 conn(0x7f656c0fc2d0 msgr2=0x7f656c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.290+0000 7f65726d7700 1 -- 172.21.15.37:0/1082843464 shutdown_connections 2024-03-26T06:22:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:22.290+0000 7f65726d7700 1 -- 172.21.15.37:0/1082843464 wait complete. 2024-03-26T06:22:22.477 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:22 smithi037 ceph-mon[31974]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-26T06:22:22.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:22 smithi066 ceph-mon[36504]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-26T06:22:22.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:22 smithi149 ceph-mon[36681]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-26T06:22:22.969 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-26T06:22:23.220 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:22:23.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:23 smithi066 ceph-mon[36504]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:23.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:23 smithi037 ceph-mon[31974]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:23.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:23 smithi149 ceph-mon[36681]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:24.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:24 smithi037 ceph-mon[31974]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T06:22:24.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:24 smithi066 ceph-mon[36504]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T06:22:24.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:24 smithi149 ceph-mon[36681]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T06:22:24.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.838+0000 7f73ccbef700 1 -- 172.21.15.37:0/2980219382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73c81000d0 msgr2=0x7f73c81004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:24.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.838+0000 7f73ccbef700 1 --2- 172.21.15.37:0/2980219382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73c81000d0 0x7f73c81004f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009990 tx=0x7f73b8009ca0 comp rx=0 tx=0).stop 2024-03-26T06:22:24.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.838+0000 7f73ccbef700 1 -- 172.21.15.37:0/2980219382 shutdown_connections 2024-03-26T06:22:24.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.838+0000 7f73ccbef700 1 --2- 172.21.15.37:0/2980219382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73c81000d0 0x7f73c81004f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:24.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.838+0000 7f73ccbef700 1 --2- 172.21.15.37:0/2980219382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c8103a50 0x7f73c810a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:24.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.838+0000 7f73ccbef700 1 --2- 172.21.15.37:0/2980219382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c81030b0 0x7f73c8103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:24.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.838+0000 7f73ccbef700 1 -- 172.21.15.37:0/2980219382 >> 172.21.15.37:0/2980219382 conn(0x7f73c80fb9f0 msgr2=0x7f73c80fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:24.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.839+0000 7f73ccbef700 1 -- 172.21.15.37:0/2980219382 shutdown_connections 2024-03-26T06:22:24.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.839+0000 7f73ccbef700 1 -- 172.21.15.37:0/2980219382 wait complete. 2024-03-26T06:22:24.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.839+0000 7f73ccbef700 1 Processor -- start 2024-03-26T06:22:24.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.839+0000 7f73ccbef700 1 -- start start 2024-03-26T06:22:24.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.839+0000 7f73ccbef700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c81000d0 0x7f73c8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:24.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.839+0000 7f73ccbef700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73c81030b0 0x7f73c8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:24.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73ccbef700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c8103a50 0x7f73c819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73ccbef700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c810da40 con 0x7f73c81030b0 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73ccbef700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c810d8c0 con 0x7f73c81000d0 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73ccbef700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c810dbc0 con 0x7f73c8103a50 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73c81030b0 0x7f73c8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73c81030b0 0x7f73c8198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59754/0 (socket says 172.21.15.37:59754) 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c5d9b700 1 -- 172.21.15.37:0/3315010913 learned_addr learned my addr 172.21.15.37:0/3315010913 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c659c700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c81000d0 0x7f73c8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c6d9d700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c8103a50 0x7f73c819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c5d9b700 1 -- 172.21.15.37:0/3315010913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c8103a50 msgr2=0x7f73c819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c5d9b700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c8103a50 0x7f73c819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c5d9b700 1 -- 172.21.15.37:0/3315010913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c81000d0 msgr2=0x7f73c8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c5d9b700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c81000d0 0x7f73c8198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c5d9b700 1 -- 172.21.15.37:0/3315010913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c819fc80 con 0x7f73c81030b0 2024-03-26T06:22:24.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c6d9d700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c8103a50 0x7f73c819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:24.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73c5d9b700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73c81030b0 0x7f73c8198560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00c9e0 tx=0x7f73bc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:24.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73b77fe700 1 -- 172.21.15.37:0/3315010913 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc007ac0 con 0x7f73c81030b0 2024-03-26T06:22:24.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73b77fe700 1 -- 172.21.15.37:0/3315010913 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73bc007c60 con 0x7f73c81030b0 2024-03-26T06:22:24.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73ccbef700 1 -- 172.21.15.37:0/3315010913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c819ff70 con 0x7f73c81030b0 2024-03-26T06:22:24.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.840+0000 7f73b77fe700 1 -- 172.21.15.37:0/3315010913 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc018730 con 0x7f73c81030b0 2024-03-26T06:22:24.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.841+0000 7f73ccbef700 1 -- 172.21.15.37:0/3315010913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c81a04a0 con 0x7f73c81030b0 2024-03-26T06:22:24.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.842+0000 7f73ccbef700 1 -- 172.21.15.37:0/3315010913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c8068aa0 con 0x7f73c81030b0 2024-03-26T06:22:24.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.843+0000 7f73b77fe700 1 -- 172.21.15.37:0/3315010913 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f73bc0188d0 con 0x7f73c81030b0 2024-03-26T06:22:24.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.843+0000 7f73b77fe700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73b00a2890 0x7f73b00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:24.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.843+0000 7f73c659c700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73b00a2890 0x7f73b00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:24.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.843+0000 7f73b77fe700 1 -- 172.21.15.37:0/3315010913 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f73bc0cea10 con 0x7f73c81030b0 2024-03-26T06:22:24.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.844+0000 7f73c659c700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73b00a2890 0x7f73b00a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73b8003890 tx=0x7f73b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:24.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:24.848+0000 7f73b77fe700 1 -- 172.21.15.37:0/3315010913 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f73bc014030 con 0x7f73c81030b0 2024-03-26T06:22:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.061+0000 7f73ccbef700 1 -- 172.21.15.37:0/3315010913 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f73c8062220 con 0x7f73b00a2890 2024-03-26T06:22:25.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.075+0000 7f73b77fe700 1 -- 172.21.15.37:0/3315010913 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f73c8062220 con 0x7f73b00a2890 2024-03-26T06:22:25.077 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-26T06:22:25.077 INFO:teuthology.orchestra.run.smithi037.stdout:mds.a smithi037 running (99s) 15s ago 98s 17.6M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 9e8f8e1ed77c 2024-03-26T06:22:25.077 INFO:teuthology.orchestra.run.smithi037.stdout:mds.b smithi066 running (96s) 30s ago 95s 25.7M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 442f7c8a9335 2024-03-26T06:22:25.077 INFO:teuthology.orchestra.run.smithi037.stdout:mds.c smithi149 running (110s) 30s ago 109s 17.0M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 3f6908e4e203 2024-03-26T06:22:25.077 INFO:teuthology.orchestra.run.smithi037.stdout:mds.d smithi037 running (118s) 15s ago 117s 15.3M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 8b6b55fa840c 2024-03-26T06:22:25.077 INFO:teuthology.orchestra.run.smithi037.stdout:mds.e smithi066 running (115s) 30s ago 114s 17.5M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 cda5fce98f2f 2024-03-26T06:22:25.077 INFO:teuthology.orchestra.run.smithi037.stdout:mds.f smithi149 running (112s) 30s ago 112s 16.0M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 ad3a23bcf9d8 2024-03-26T06:22:25.077 INFO:teuthology.orchestra.run.smithi037.stdout:mds.g smithi037 running (101s) 15s ago 101s 19.7M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 3eccb2b6cbd6 2024-03-26T06:22:25.077 INFO:teuthology.orchestra.run.smithi037.stdout:mds.h smithi066 running (107s) 30s ago 106s 16.5M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 47ef9a70fe8e 2024-03-26T06:22:25.077 INFO:teuthology.orchestra.run.smithi037.stdout:mds.i smithi149 running (104s) 30s ago 104s 19.4M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 4f27664c4382 2024-03-26T06:22:25.077 INFO:teuthology.orchestra.run.smithi037.stdout:mds.j smithi037 running (2m) 15s ago 2m 14.0M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 7cd9991f93c1 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:mds.k smithi066 running (2m) 30s ago 2m 16.1M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 5ed6198eac4d 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:mds.l smithi149 running (2m) 30s ago 2m 16.2M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 2379c8bfc0c2 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.x smithi037 *:9283,8765 running (9m) 15s ago 9m 515M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 e777f6a6a513 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.y smithi066 *:8443,8765 running (7m) 30s ago 7m 452M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 9354924ed7bb 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.z smithi149 *:8443,8765 running (7m) 30s ago 7m 450M - 19.0.0-1964-g7e98f5f6 10cdb6adb615 4847a6846fae 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:mon.a smithi037 running (9m) 15s ago 9m 52.5M 2048M 19.0.0-1964-g7e98f5f6 10cdb6adb615 7e242629978a 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:mon.b smithi066 running (7m) 30s ago 7m 39.3M 2048M 19.0.0-1964-g7e98f5f6 10cdb6adb615 c5ad1d6a1432 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:mon.c smithi149 running (7m) 30s ago 7m 39.1M 2048M 19.0.0-1964-g7e98f5f6 10cdb6adb615 db92c4504829 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (7m) 15s ago 7m 59.1M 4273M 19.0.0-1964-g7e98f5f6 10cdb6adb615 bd091674e0d1 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi066 running (6m) 30s ago 6m 48.9M 4270M 19.0.0-1964-g7e98f5f6 10cdb6adb615 bccd2b310f58 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi149 running (6m) 30s ago 6m 60.0M 1451M 19.0.0-1964-g7e98f5f6 10cdb6adb615 0cb87caeb09e 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi037 running (5m) 15s ago 5m 57.0M 4273M 19.0.0-1964-g7e98f5f6 10cdb6adb615 589358d39148 2024-03-26T06:22:25.078 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi066 running (5m) 30s ago 5m 45.2M 4270M 19.0.0-1964-g7e98f5f6 10cdb6adb615 b55f12906546 2024-03-26T06:22:25.079 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi149 running (5m) 30s ago 5m 47.2M 1451M 19.0.0-1964-g7e98f5f6 10cdb6adb615 bec29ba7d6e4 2024-03-26T06:22:25.079 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 smithi037 running (4m) 15s ago 4m 45.4M 4273M 19.0.0-1964-g7e98f5f6 10cdb6adb615 5a1928977410 2024-03-26T06:22:25.079 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 smithi066 running (4m) 30s ago 4m 57.3M 4270M 19.0.0-1964-g7e98f5f6 10cdb6adb615 23ffe3128da1 2024-03-26T06:22:25.079 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 smithi149 running (3m) 30s ago 3m 45.6M 1451M 19.0.0-1964-g7e98f5f6 10cdb6adb615 461ca5beb81e 2024-03-26T06:22:25.079 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 smithi037 running (3m) 15s ago 3m 52.0M 4273M 19.0.0-1964-g7e98f5f6 10cdb6adb615 2c3cdf08385a 2024-03-26T06:22:25.079 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 smithi066 running (2m) 30s ago 2m 57.3M 4270M 19.0.0-1964-g7e98f5f6 10cdb6adb615 f82df426b87d 2024-03-26T06:22:25.079 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 smithi149 running (2m) 30s ago 2m 43.0M 1451M 19.0.0-1964-g7e98f5f6 10cdb6adb615 682658acdb00 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.077+0000 7f73ccbef700 1 -- 172.21.15.37:0/3315010913 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73b00a2890 msgr2=0x7f73b00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.077+0000 7f73ccbef700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73b00a2890 0x7f73b00a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73b8003890 tx=0x7f73b8005fd0 comp rx=0 tx=0).stop 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.077+0000 7f73ccbef700 1 -- 172.21.15.37:0/3315010913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73c81030b0 msgr2=0x7f73c8198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.077+0000 7f73ccbef700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73c81030b0 0x7f73c8198560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00c9e0 tx=0x7f73bc00ccf0 comp rx=0 tx=0).stop 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.078+0000 7f73ccbef700 1 -- 172.21.15.37:0/3315010913 shutdown_connections 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.078+0000 7f73ccbef700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f73b00a2890 0x7f73b00a4d50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.078+0000 7f73ccbef700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73c8103a50 0x7f73c819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.078+0000 7f73ccbef700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73c81000d0 0x7f73c8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.078+0000 7f73ccbef700 1 --2- 172.21.15.37:0/3315010913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73c81030b0 0x7f73c8198560 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.078+0000 7f73ccbef700 1 -- 172.21.15.37:0/3315010913 >> 172.21.15.37:0/3315010913 conn(0x7f73c80fb9f0 msgr2=0x7f73c8106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.078+0000 7f73ccbef700 1 -- 172.21.15.37:0/3315010913 shutdown_connections 2024-03-26T06:22:25.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:25.078+0000 7f73ccbef700 1 -- 172.21.15.37:0/3315010913 wait complete. 2024-03-26T06:22:25.711 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-26T06:22:25.955 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:22:26.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:26 smithi037 ceph-mon[31974]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T06:22:26.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:26 smithi037 ceph-mon[31974]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:26.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:26 smithi066 ceph-mon[36504]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T06:22:26.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:26 smithi066 ceph-mon[36504]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:26.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:26 smithi149 ceph-mon[36681]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T06:22:26.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:26 smithi149 ceph-mon[36681]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:27.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.507+0000 7fb3b8f30700 1 -- 172.21.15.37:0/4134497062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102920 msgr2=0x7fb3b41091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.507+0000 7fb3b8f30700 1 --2- 172.21.15.37:0/4134497062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102920 0x7fb3b41091d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00b0d0 tx=0x7fb3ac00b3e0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.508+0000 7fb3b8f30700 1 -- 172.21.15.37:0/4134497062 shutdown_connections 2024-03-26T06:22:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.508+0000 7fb3b8f30700 1 --2- 172.21.15.37:0/4134497062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102920 0x7fb3b41091d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.508+0000 7fb3b8f30700 1 --2- 172.21.15.37:0/4134497062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3b4100d80 0x7fb3b41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.508+0000 7fb3b8f30700 1 --2- 172.21.15.37:0/4134497062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3b4101f80 0x7fb3b41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.508+0000 7fb3b8f30700 1 -- 172.21.15.37:0/4134497062 >> 172.21.15.37:0/4134497062 conn(0x7fb3b40fc340 msgr2=0x7fb3b40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.508+0000 7fb3b8f30700 1 -- 172.21.15.37:0/4134497062 shutdown_connections 2024-03-26T06:22:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.508+0000 7fb3b8f30700 1 -- 172.21.15.37:0/4134497062 wait complete. 2024-03-26T06:22:27.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.508+0000 7fb3b8f30700 1 Processor -- start 2024-03-26T06:22:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.508+0000 7fb3b8f30700 1 -- start start 2024-03-26T06:22:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b8f30700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3b4100d80 0x7fb3b4104d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b8f30700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3b4101f80 0x7fb3b4103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b8f30700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102920 0x7fb3b41038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b8f30700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b410c900 con 0x7fb3b4102920 2024-03-26T06:22:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b8f30700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b410c780 con 0x7fb3b4100d80 2024-03-26T06:22:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b8f30700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b410ca80 con 0x7fb3b4101f80 2024-03-26T06:22:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102920 0x7fb3b41038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102920 0x7fb3b41038a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59770/0 (socket says 172.21.15.37:59770) 2024-03-26T06:22:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b2d9d700 1 -- 172.21.15.37:0/2963768603 learned_addr learned my addr 172.21.15.37:0/2963768603 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b1d9b700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3b4101f80 0x7fb3b4103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b259c700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3b4100d80 0x7fb3b4104d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b2d9d700 1 -- 172.21.15.37:0/2963768603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3b4101f80 msgr2=0x7fb3b4103360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b2d9d700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3b4101f80 0x7fb3b4103360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b2d9d700 1 -- 172.21.15.37:0/2963768603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3b4100d80 msgr2=0x7fb3b4104d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b2d9d700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3b4100d80 0x7fb3b4104d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b2d9d700 1 -- 172.21.15.37:0/2963768603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b4104150 con 0x7fb3b4102920 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b259c700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3b4100d80 0x7fb3b4104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b2d9d700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102920 0x7fb3b41038a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac006070 tx=0x7fb3ac007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3a37fe700 1 -- 172.21.15.37:0/2963768603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac021020 con 0x7fb3b4102920 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3a37fe700 1 -- 172.21.15.37:0/2963768603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3ac01d950 con 0x7fb3b4102920 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3a37fe700 1 -- 172.21.15.37:0/2963768603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac005510 con 0x7fb3b4102920 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.509+0000 7fb3b8f30700 1 -- 172.21.15.37:0/2963768603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b41a20e0 con 0x7fb3b4102920 2024-03-26T06:22:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.510+0000 7fb3b8f30700 1 -- 172.21.15.37:0/2963768603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b41a25f0 con 0x7fb3b4102920 2024-03-26T06:22:27.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.510+0000 7fb3b8f30700 1 -- 172.21.15.37:0/2963768603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b4068aa0 con 0x7fb3b4102920 2024-03-26T06:22:27.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.511+0000 7fb3a37fe700 1 -- 172.21.15.37:0/2963768603 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb3ac024020 con 0x7fb3b4102920 2024-03-26T06:22:27.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.512+0000 7fb3a37fe700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb39c0a2890 0x7fb39c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:27.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.512+0000 7fb3b259c700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb39c0a2890 0x7fb39c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:27.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.512+0000 7fb3a37fe700 1 -- 172.21.15.37:0/2963768603 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb3ac00f020 con 0x7fb3b4102920 2024-03-26T06:22:27.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.512+0000 7fb3b259c700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb39c0a2890 0x7fb39c0a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8005fd0 tx=0x7fb3a8005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:27.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.516+0000 7fb3a37fe700 1 -- 172.21.15.37:0/2963768603 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb3ac09d5d0 con 0x7fb3b4102920 2024-03-26T06:22:27.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.731+0000 7fb3b8f30700 1 -- 172.21.15.37:0/2963768603 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3b41a2850 con 0x7fb39c0a2890 2024-03-26T06:22:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.737+0000 7fb3a37fe700 1 -- 172.21.15.37:0/2963768603 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fb3b41a2850 con 0x7fb39c0a2890 2024-03-26T06:22:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-26T06:22:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:mds.all 12/12 32s ago 2m smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-03-26T06:22:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:mgr 3/3 32s ago 7m smithi037=x;smithi066=y;smithi149=z;count:3 2024-03-26T06:22:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:mon 3/3 32s ago 7m smithi037:172.21.15.37=a;smithi066:172.21.15.66=b;smithi149:172.21.15.149=c;count:3 2024-03-26T06:22:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:osd 12 32s ago - 2024-03-26T06:22:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 -- 172.21.15.37:0/2963768603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb39c0a2890 msgr2=0x7fb39c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb39c0a2890 0x7fb39c0a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8005fd0 tx=0x7fb3a8005de0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 -- 172.21.15.37:0/2963768603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102920 msgr2=0x7fb3b41038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102920 0x7fb3b41038a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac006070 tx=0x7fb3ac007dc0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 -- 172.21.15.37:0/2963768603 shutdown_connections 2024-03-26T06:22:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102920 0x7fb3b41038a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb39c0a2890 0x7fb39c0a4d50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3b4100d80 0x7fb3b4104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 --2- 172.21.15.37:0/2963768603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3b4101f80 0x7fb3b4103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:27.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 -- 172.21.15.37:0/2963768603 >> 172.21.15.37:0/2963768603 conn(0x7fb3b40fc340 msgr2=0x7fb3b40fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:27.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 -- 172.21.15.37:0/2963768603 shutdown_connections 2024-03-26T06:22:27.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:27.740+0000 7fb3b8f30700 1 -- 172.21.15.37:0/2963768603 wait complete. 2024-03-26T06:22:28.355 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:28 smithi037 ceph-mon[31974]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T06:22:28.355 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:28 smithi037 ceph-mon[31974]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:28.383 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-26T06:22:28.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:28 smithi066 ceph-mon[36504]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T06:22:28.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:28 smithi066 ceph-mon[36504]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:28.628 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:22:28.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:28 smithi149 ceph-mon[36681]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T06:22:28.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:28 smithi149 ceph-mon[36681]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:30.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.417+0000 7fa7d041a700 1 -- 172.21.15.37:0/1710916661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c8102f70 msgr2=0x7fa7c81033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:30.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.417+0000 7fa7d041a700 1 --2- 172.21.15.37:0/1710916661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c8102f70 0x7fa7c81033f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009990 tx=0x7fa7b8009ca0 comp rx=0 tx=0).stop 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.417+0000 7fa7d041a700 1 -- 172.21.15.37:0/1710916661 shutdown_connections 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.417+0000 7fa7d041a700 1 --2- 172.21.15.37:0/1710916661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c8103930 0x7fa7c810a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.417+0000 7fa7d041a700 1 --2- 172.21.15.37:0/1710916661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c8101e10 0x7fa7c8102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.417+0000 7fa7d041a700 1 --2- 172.21.15.37:0/1710916661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c8102f70 0x7fa7c81033f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.417+0000 7fa7d041a700 1 -- 172.21.15.37:0/1710916661 >> 172.21.15.37:0/1710916661 conn(0x7fa7c80fd3f0 msgr2=0x7fa7c80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.417+0000 7fa7d041a700 1 -- 172.21.15.37:0/1710916661 shutdown_connections 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.417+0000 7fa7d041a700 1 -- 172.21.15.37:0/1710916661 wait complete. 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7d041a700 1 Processor -- start 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7d041a700 1 -- start start 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7d041a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c8101e10 0x7fa7c8075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7d041a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c8102f70 0x7fa7c80741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7d041a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c8103930 0x7fa7c8074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7d041a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c810da60 con 0x7fa7c8101e10 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7d041a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c810d8e0 con 0x7fa7c8102f70 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7d041a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c810dbe0 con 0x7fa7c8103930 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7ce1b6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c8101e10 0x7fa7c8075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7ce1b6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c8101e10 0x7fa7c8075b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51478/0 (socket says 172.21.15.37:51478) 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7ce1b6700 1 -- 172.21.15.37:0/1228568734 learned_addr learned my addr 172.21.15.37:0/1228568734 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7cd9b5700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c8102f70 0x7fa7c80741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7ce9b7700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c8103930 0x7fa7c8074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7ce1b6700 1 -- 172.21.15.37:0/1228568734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c8103930 msgr2=0x7fa7c8074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7ce1b6700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c8103930 0x7fa7c8074710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7ce1b6700 1 -- 172.21.15.37:0/1228568734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c8102f70 msgr2=0x7fa7c80741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.418+0000 7fa7ce1b6700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c8102f70 0x7fa7c80741d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.419+0000 7fa7ce1b6700 1 -- 172.21.15.37:0/1228568734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c8074fc0 con 0x7fa7c8101e10 2024-03-26T06:22:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.419+0000 7fa7ce1b6700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c8101e10 0x7fa7c8075b80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400d950 tx=0x7fa7c400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:30.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.419+0000 7fa7bf7fe700 1 -- 172.21.15.37:0/1228568734 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4016070 con 0x7fa7c8101e10 2024-03-26T06:22:30.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.419+0000 7fa7bf7fe700 1 -- 172.21.15.37:0/1228568734 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7c4009e20 con 0x7fa7c8101e10 2024-03-26T06:22:30.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.419+0000 7fa7d041a700 1 -- 172.21.15.37:0/1228568734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c81a21d0 con 0x7fa7c8101e10 2024-03-26T06:22:30.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.419+0000 7fa7bf7fe700 1 -- 172.21.15.37:0/1228568734 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c400f620 con 0x7fa7c8101e10 2024-03-26T06:22:30.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.419+0000 7fa7d041a700 1 -- 172.21.15.37:0/1228568734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c81a2760 con 0x7fa7c8101e10 2024-03-26T06:22:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.420+0000 7fa7d041a700 1 -- 172.21.15.37:0/1228568734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c8068aa0 con 0x7fa7c8101e10 2024-03-26T06:22:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.421+0000 7fa7bf7fe700 1 -- 172.21.15.37:0/1228568734 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa7c4019070 con 0x7fa7c8101e10 2024-03-26T06:22:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.422+0000 7fa7bf7fe700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7b40a2890 0x7fa7b40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.422+0000 7fa7cd9b5700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7b40a2890 0x7fa7b40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.422+0000 7fa7bf7fe700 1 -- 172.21.15.37:0/1228568734 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa7c40ce640 con 0x7fa7c8101e10 2024-03-26T06:22:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.422+0000 7fa7cd9b5700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7b40a2890 0x7fa7b40a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800b5c0 tx=0x7fa7b80038f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:30.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.426+0000 7fa7bf7fe700 1 -- 172.21.15.37:0/1228568734 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa7c409b6f0 con 0x7fa7c8101e10 2024-03-26T06:22:30.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:30 smithi037 ceph-mon[31974]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-26T06:22:30.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:30 smithi066 ceph-mon[36504]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-26T06:22:30.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.636+0000 7fa7d041a700 1 -- 172.21.15.37:0/1228568734 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7c8062220 con 0x7fa7b40a2890 2024-03-26T06:22:30.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.638+0000 7fa7bf7fe700 1 -- 172.21.15.37:0/1228568734 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fa7c8062220 con 0x7fa7b40a2890 2024-03-26T06:22:30.638 INFO:teuthology.orchestra.run.smithi037.stdout:HOST ADDR LABELS STATUS 2024-03-26T06:22:30.638 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 172.21.15.37 2024-03-26T06:22:30.638 INFO:teuthology.orchestra.run.smithi037.stdout:smithi066 172.21.15.66 2024-03-26T06:22:30.638 INFO:teuthology.orchestra.run.smithi037.stdout:smithi149 172.21.15.149 2024-03-26T06:22:30.638 INFO:teuthology.orchestra.run.smithi037.stdout:3 hosts in cluster 2024-03-26T06:22:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 -- 172.21.15.37:0/1228568734 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7b40a2890 msgr2=0x7fa7b40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7b40a2890 0x7fa7b40a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800b5c0 tx=0x7fa7b80038f0 comp rx=0 tx=0).stop 2024-03-26T06:22:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 -- 172.21.15.37:0/1228568734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c8101e10 msgr2=0x7fa7c8075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c8101e10 0x7fa7c8075b80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400d950 tx=0x7fa7c400dc60 comp rx=0 tx=0).stop 2024-03-26T06:22:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 -- 172.21.15.37:0/1228568734 shutdown_connections 2024-03-26T06:22:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c8101e10 0x7fa7c8075b80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa7b40a2890 0x7fa7b40a4d50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7c8103930 0x7fa7c8074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 --2- 172.21.15.37:0/1228568734 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7c8102f70 0x7fa7c80741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 -- 172.21.15.37:0/1228568734 >> 172.21.15.37:0/1228568734 conn(0x7fa7c80fd3f0 msgr2=0x7fa7c8106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 -- 172.21.15.37:0/1228568734 shutdown_connections 2024-03-26T06:22:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:30.640+0000 7fa7d041a700 1 -- 172.21.15.37:0/1228568734 wait complete. 2024-03-26T06:22:30.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:30 smithi149 ceph-mon[36681]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-26T06:22:31.436 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-26T06:22:31.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:31 smithi066 ceph-mon[36504]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:31.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:31 smithi149 ceph-mon[36681]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:31.683 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:22:31.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:31 smithi037 ceph-mon[31974]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:32.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:32 smithi037 ceph-mon[31974]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-26T06:22:32.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:32 smithi066 ceph-mon[36504]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-26T06:22:32.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:32 smithi149 ceph-mon[36681]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-26T06:22:33.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.404+0000 7fa2ddf93700 1 -- 172.21.15.37:0/737710760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8103930 msgr2=0x7fa2d810a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:33.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.404+0000 7fa2ddf93700 1 --2- 172.21.15.37:0/737710760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8103930 0x7fa2d810a250 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000b0a0 tx=0x7fa2d000b3b0 comp rx=0 tx=0).stop 2024-03-26T06:22:33.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.404+0000 7fa2ddf93700 1 -- 172.21.15.37:0/737710760 shutdown_connections 2024-03-26T06:22:33.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.404+0000 7fa2ddf93700 1 --2- 172.21.15.37:0/737710760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8103930 0x7fa2d810a250 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:33.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.404+0000 7fa2ddf93700 1 --2- 172.21.15.37:0/737710760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2d8101e10 0x7fa2d8102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:33.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.404+0000 7fa2ddf93700 1 --2- 172.21.15.37:0/737710760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d8102f70 0x7fa2d81033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:33.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.404+0000 7fa2ddf93700 1 -- 172.21.15.37:0/737710760 >> 172.21.15.37:0/737710760 conn(0x7fa2d80fd3f0 msgr2=0x7fa2d80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:33.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.404+0000 7fa2ddf93700 1 -- 172.21.15.37:0/737710760 shutdown_connections 2024-03-26T06:22:33.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.404+0000 7fa2ddf93700 1 -- 172.21.15.37:0/737710760 wait complete. 2024-03-26T06:22:33.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.405+0000 7fa2ddf93700 1 Processor -- start 2024-03-26T06:22:33.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.405+0000 7fa2ddf93700 1 -- start start 2024-03-26T06:22:33.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.405+0000 7fa2ddf93700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8101e10 0x7fa2d8198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:33.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.405+0000 7fa2ddf93700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d8102f70 0x7fa2d8198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.405+0000 7fa2ddf93700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2d8103930 0x7fa2d819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.405+0000 7fa2ddf93700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d810dae0 con 0x7fa2d8101e10 2024-03-26T06:22:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.405+0000 7fa2ddf93700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d810d960 con 0x7fa2d8103930 2024-03-26T06:22:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.405+0000 7fa2ddf93700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d810dc60 con 0x7fa2d8102f70 2024-03-26T06:22:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.405+0000 7fa2d77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8101e10 0x7fa2d8198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d6ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d8102f70 0x7fa2d8198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d6ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d8102f70 0x7fa2d8198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:49366/0 (socket says 172.21.15.37:49366) 2024-03-26T06:22:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d6ffd700 1 -- 172.21.15.37:0/2067835412 learned_addr learned my addr 172.21.15.37:0/2067835412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d7fff700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2d8103930 0x7fa2d819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d77fe700 1 -- 172.21.15.37:0/2067835412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d8102f70 msgr2=0x7fa2d8198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:33.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d77fe700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d8102f70 0x7fa2d8198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d77fe700 1 -- 172.21.15.37:0/2067835412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2d8103930 msgr2=0x7fa2d819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d77fe700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2d8103930 0x7fa2d819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d77fe700 1 -- 172.21.15.37:0/2067835412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d819fd80 con 0x7fa2d8101e10 2024-03-26T06:22:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d6ffd700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d8102f70 0x7fa2d8198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:22:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d77fe700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8101e10 0x7fa2d8198120 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800b480 tx=0x7fa2c800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d4ff9700 1 -- 172.21.15.37:0/2067835412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8015020 con 0x7fa2d8101e10 2024-03-26T06:22:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d4ff9700 1 -- 172.21.15.37:0/2067835412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2c800be50 con 0x7fa2d8101e10 2024-03-26T06:22:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2ddf93700 1 -- 172.21.15.37:0/2067835412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d8074070 con 0x7fa2d8101e10 2024-03-26T06:22:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2d4ff9700 1 -- 172.21.15.37:0/2067835412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8010ce0 con 0x7fa2d8101e10 2024-03-26T06:22:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.406+0000 7fa2ddf93700 1 -- 172.21.15.37:0/2067835412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d8074680 con 0x7fa2d8101e10 2024-03-26T06:22:33.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.407+0000 7fa2ddf93700 1 -- 172.21.15.37:0/2067835412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d8068aa0 con 0x7fa2d8101e10 2024-03-26T06:22:33.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.408+0000 7fa2d4ff9700 1 -- 172.21.15.37:0/2067835412 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa2c8018020 con 0x7fa2d8101e10 2024-03-26T06:22:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.409+0000 7fa2d4ff9700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2c00a2910 0x7fa2c00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.409+0000 7fa2d6ffd700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2c00a2910 0x7fa2c00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.409+0000 7fa2d4ff9700 1 -- 172.21.15.37:0/2067835412 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa2c80ce5b0 con 0x7fa2d8101e10 2024-03-26T06:22:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.409+0000 7fa2d6ffd700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2c00a2910 0x7fa2c00a4dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009ea0 tx=0x7fa2cc009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.413+0000 7fa2d4ff9700 1 -- 172.21.15.37:0/2067835412 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa2c801f020 con 0x7fa2d8101e10 2024-03-26T06:22:33.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.626+0000 7fa2ddf93700 1 -- 172.21.15.37:0/2067835412 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2d8108c10 con 0x7fa2c00a2910 2024-03-26T06:22:33.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.627+0000 7fa2d4ff9700 1 -- 172.21.15.37:0/2067835412 <== mgr.14152 v2:172.21.15.37:6800/641545212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fa2d8108c10 con 0x7fa2c00a2910 2024-03-26T06:22:33.629 INFO:teuthology.orchestra.run.smithi037.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-26T06:22:33.629 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005T400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-26T06:22:33.630 INFO:teuthology.orchestra.run.smithi037.stdout:smithi066 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53430091400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-26T06:22:33.630 INFO:teuthology.orchestra.run.smithi037.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-26T06:22:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.629+0000 7fa2ddf93700 1 -- 172.21.15.37:0/2067835412 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2c00a2910 msgr2=0x7fa2c00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.629+0000 7fa2ddf93700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2c00a2910 0x7fa2c00a4dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009ea0 tx=0x7fa2cc009290 comp rx=0 tx=0).stop 2024-03-26T06:22:33.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.629+0000 7fa2ddf93700 1 -- 172.21.15.37:0/2067835412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8101e10 msgr2=0x7fa2d8198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:33.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.629+0000 7fa2ddf93700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8101e10 0x7fa2d8198120 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800b480 tx=0x7fa2c800b790 comp rx=0 tx=0).stop 2024-03-26T06:22:33.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.629+0000 7fa2ddf93700 1 -- 172.21.15.37:0/2067835412 shutdown_connections 2024-03-26T06:22:33.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.629+0000 7fa2ddf93700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8101e10 0x7fa2d8198120 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:33.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.629+0000 7fa2ddf93700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa2c00a2910 0x7fa2c00a4dd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:33.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.629+0000 7fa2ddf93700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2d8103930 0x7fa2d819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:33.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.629+0000 7fa2ddf93700 1 --2- 172.21.15.37:0/2067835412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d8102f70 0x7fa2d8198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:33.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.629+0000 7fa2ddf93700 1 -- 172.21.15.37:0/2067835412 >> 172.21.15.37:0/2067835412 conn(0x7fa2d80fd3f0 msgr2=0x7fa2d8106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:33.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.630+0000 7fa2ddf93700 1 -- 172.21.15.37:0/2067835412 shutdown_connections 2024-03-26T06:22:33.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:33.630+0000 7fa2ddf93700 1 -- 172.21.15.37:0/2067835412 wait complete. 2024-03-26T06:22:34.417 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T06:22:34.433 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-26T06:22:34.434 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-26T06:22:34.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:34 smithi066 ceph-mon[36504]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-26T06:22:34.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:34 smithi066 ceph-mon[36504]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:34.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:34 smithi149 ceph-mon[36681]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-26T06:22:34.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:34 smithi149 ceph-mon[36681]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:34.682 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:22:34.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:34 smithi037 ceph-mon[31974]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-26T06:22:34.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:34 smithi037 ceph-mon[31974]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:22:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.390+0000 7f3937a4e700 1 -- 172.21.15.37:0/618625368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39301028c0 msgr2=0x7f3930109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.390+0000 7f3937a4e700 1 --2- 172.21.15.37:0/618625368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39301028c0 0x7f3930109170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f392800b0d0 tx=0x7f392800b3e0 comp rx=0 tx=0).stop 2024-03-26T06:22:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.390+0000 7f3937a4e700 1 -- 172.21.15.37:0/618625368 shutdown_connections 2024-03-26T06:22:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.390+0000 7f3937a4e700 1 --2- 172.21.15.37:0/618625368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39301028c0 0x7f3930109170 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.390+0000 7f3937a4e700 1 --2- 172.21.15.37:0/618625368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3930100d20 0x7f3930101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.390+0000 7f3937a4e700 1 --2- 172.21.15.37:0/618625368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3930101f20 0x7f3930102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.390+0000 7f3937a4e700 1 -- 172.21.15.37:0/618625368 >> 172.21.15.37:0/618625368 conn(0x7f39300fc2e0 msgr2=0x7f39300fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.390+0000 7f3937a4e700 1 -- 172.21.15.37:0/618625368 shutdown_connections 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.390+0000 7f3937a4e700 1 -- 172.21.15.37:0/618625368 wait complete. 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3937a4e700 1 Processor -- start 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3937a4e700 1 -- start start 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3937a4e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3930100d20 0x7f393006b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3937a4e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3930101f20 0x7f393019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3937a4e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39301028c0 0x7f393019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3937a4e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393010c8f0 con 0x7f39301028c0 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3937a4e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393010c770 con 0x7f3930100d20 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3937a4e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393010ca70 con 0x7f3930101f20 2024-03-26T06:22:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3935feb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39301028c0 0x7f393019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3935feb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39301028c0 0x7f393019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51500/0 (socket says 172.21.15.37:51500) 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3935feb700 1 -- 172.21.15.37:0/1514433611 learned_addr learned my addr 172.21.15.37:0/1514433611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3934fe9700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3930101f20 0x7f393019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f39357ea700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3930100d20 0x7f393006b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3935feb700 1 -- 172.21.15.37:0/1514433611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3930101f20 msgr2=0x7f393019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3935feb700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3930101f20 0x7f393019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3935feb700 1 -- 172.21.15.37:0/1514433611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3930100d20 msgr2=0x7f393006b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3935feb700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3930100d20 0x7f393006b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.391+0000 7f3935feb700 1 -- 172.21.15.37:0/1514433611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393019fc50 con 0x7f39301028c0 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.392+0000 7f39357ea700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3930100d20 0x7f393006b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.392+0000 7f3934fe9700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3930101f20 0x7f393019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.392+0000 7f3935feb700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39301028c0 0x7f393019f380 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f392800be70 tx=0x7f3928005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.392+0000 7f39267fc700 1 -- 172.21.15.37:0/1514433611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392801f070 con 0x7f39301028c0 2024-03-26T06:22:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.392+0000 7f39267fc700 1 -- 172.21.15.37:0/1514433611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3928005070 con 0x7f39301028c0 2024-03-26T06:22:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.392+0000 7f3937a4e700 1 -- 172.21.15.37:0/1514433611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393019fee0 con 0x7f39301028c0 2024-03-26T06:22:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.392+0000 7f39267fc700 1 -- 172.21.15.37:0/1514433611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392801b660 con 0x7f39301028c0 2024-03-26T06:22:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.392+0000 7f3937a4e700 1 -- 172.21.15.37:0/1514433611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39301a0390 con 0x7f39301028c0 2024-03-26T06:22:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.393+0000 7f3937a4e700 1 -- 172.21.15.37:0/1514433611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3930068aa0 con 0x7f39301028c0 2024-03-26T06:22:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.394+0000 7f39267fc700 1 -- 172.21.15.37:0/1514433611 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3928022070 con 0x7f39301028c0 2024-03-26T06:22:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.395+0000 7f39267fc700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f391c0a2890 0x7f391c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.395+0000 7f39357ea700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f391c0a2890 0x7f391c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.395+0000 7f39267fc700 1 -- 172.21.15.37:0/1514433611 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f392800f020 con 0x7f39301028c0 2024-03-26T06:22:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.395+0000 7f39357ea700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f391c0a2890 0x7f391c0a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f392c005950 tx=0x7f392c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:36.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.398+0000 7f39267fc700 1 -- 172.21.15.37:0/1514433611 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3928029020 con 0x7f39301028c0 2024-03-26T06:22:36.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:36 smithi066 ceph-mon[36504]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:22:36.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.651+0000 7f3937a4e700 1 -- 172.21.15.37:0/1514433611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f393002d990 con 0x7f39301028c0 2024-03-26T06:22:36.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:36 smithi037 ceph-mon[31974]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:22:36.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.651+0000 7f39267fc700 1 -- 172.21.15.37:0/1514433611 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2965 (secure 0 0 0) 0x7f393002d990 con 0x7f39301028c0 2024-03-26T06:22:36.653 INFO:teuthology.orchestra.run.smithi037.stdout:e26 2024-03-26T06:22:36.653 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T06:22:36.653 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T06:22:36.653 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-26T06:22:36.653 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:36.653 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 26 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-26T06:20:53.631953+0000 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-26T06:21:00.587013+0000 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 300 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-26T06:22:36.654 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 0 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 5 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1,2,3,4 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=24545,1=24494,2=24470,3=24469,4=14682} 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-26T06:22:36.655 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 0 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b{0:24545} state up:active seq 3 addr [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c{0:24451} state up:standby-replay seq 1 addr [v2:172.21.15.149:6836/801999981,v1:172.21.15.149:6837/801999981] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.h{1:24494} state up:active seq 6 addr [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.f{1:24442} state up:standby-replay seq 1 addr [v2:172.21.15.149:6834/1494796094,v1:172.21.15.149:6835/1494796094] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.e{2:24470} state up:active seq 8 addr [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.k{2:24437} state up:standby-replay seq 1 addr [v2:172.21.15.66:6832/2811553067,v1:172.21.15.66:6833/2811553067] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.i{3:24469} state up:active seq 6 addr [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.l{3:24394} state up:standby-replay seq 1 addr [v2:172.21.15.149:6832/2169208399,v1:172.21.15.149:6833/2169208399] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a{4:14682} state up:active seq 5 addr [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.g{4:14670} state up:standby-replay seq 1 addr [v2:172.21.15.37:6838/1944002831,v1:172.21.15.37:6839/1944002831] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-26T06:22:36.656 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:36.657 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.j{-1:14586} state up:standby seq 1 addr [v2:172.21.15.37:6834/1987413446,v1:172.21.15.37:6835/1987413446] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.657 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d{-1:14598} state up:standby seq 1 addr [v2:172.21.15.37:6836/3479806072,v1:172.21.15.37:6837/3479806072] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T06:22:36.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.653+0000 7f3937a4e700 1 -- 172.21.15.37:0/1514433611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f391c0a2890 msgr2=0x7f391c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:36.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.653+0000 7f3937a4e700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f391c0a2890 0x7f391c0a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f392c005950 tx=0x7f392c0058e0 comp rx=0 tx=0).stop 2024-03-26T06:22:36.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.653+0000 7f3937a4e700 1 -- 172.21.15.37:0/1514433611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39301028c0 msgr2=0x7f393019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:36.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.653+0000 7f3937a4e700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39301028c0 0x7f393019f380 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f392800be70 tx=0x7f3928005b40 comp rx=0 tx=0).stop 2024-03-26T06:22:36.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.654+0000 7f3937a4e700 1 -- 172.21.15.37:0/1514433611 shutdown_connections 2024-03-26T06:22:36.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.654+0000 7f3937a4e700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39301028c0 0x7f393019f380 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:36.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.654+0000 7f3937a4e700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f391c0a2890 0x7f391c0a4d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:36.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.654+0000 7f3937a4e700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3930100d20 0x7f393006b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:36.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.654+0000 7f3937a4e700 1 --2- 172.21.15.37:0/1514433611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3930101f20 0x7f393019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:36.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.654+0000 7f3937a4e700 1 -- 172.21.15.37:0/1514433611 >> 172.21.15.37:0/1514433611 conn(0x7f39300fc2e0 msgr2=0x7f3930105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:36.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.654+0000 7f3937a4e700 1 -- 172.21.15.37:0/1514433611 shutdown_connections 2024-03-26T06:22:36.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:36.654+0000 7f3937a4e700 1 -- 172.21.15.37:0/1514433611 wait complete. 2024-03-26T06:22:36.658 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-03-26T06:22:36.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:36 smithi149 ceph-mon[36681]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:22:37.332 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-26T06:22:37.577 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:22:37.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:37 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1514433611' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T06:22:37.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:37 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1514433611' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T06:22:37.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:37 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1514433611' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T06:22:38.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:38 smithi066 ceph-mon[36504]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:38.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:38 smithi149 ceph-mon[36681]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:38.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:38 smithi037 ceph-mon[31974]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.217+0000 7f8cc967e700 1 -- 172.21.15.37:0/2099947266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4102fa0 msgr2=0x7f8cc4103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.217+0000 7f8cc967e700 1 --2- 172.21.15.37:0/2099947266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4102fa0 0x7f8cc4103420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009990 tx=0x7f8cac009ca0 comp rx=0 tx=0).stop 2024-03-26T06:22:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.217+0000 7f8cc967e700 1 -- 172.21.15.37:0/2099947266 shutdown_connections 2024-03-26T06:22:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.217+0000 7f8cc967e700 1 --2- 172.21.15.37:0/2099947266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc4103960 0x7f8cc410a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.217+0000 7f8cc967e700 1 --2- 172.21.15.37:0/2099947266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cc4101e40 0x7f8cc4102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.217+0000 7f8cc967e700 1 --2- 172.21.15.37:0/2099947266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4102fa0 0x7f8cc4103420 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.217+0000 7f8cc967e700 1 -- 172.21.15.37:0/2099947266 >> 172.21.15.37:0/2099947266 conn(0x7f8cc40fd3e0 msgr2=0x7f8cc40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.217+0000 7f8cc967e700 1 -- 172.21.15.37:0/2099947266 shutdown_connections 2024-03-26T06:22:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.218+0000 7f8cc967e700 1 -- 172.21.15.37:0/2099947266 wait complete. 2024-03-26T06:22:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.218+0000 7f8cc967e700 1 Processor -- start 2024-03-26T06:22:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.218+0000 7f8cc967e700 1 -- start start 2024-03-26T06:22:39.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.218+0000 7f8cc967e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4101e40 0x7f8cc4075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.218+0000 7f8cc967e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc4102fa0 0x7f8cc40741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.218+0000 7f8cc967e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cc4103960 0x7f8cc4074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.218+0000 7f8cc967e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc410da90 con 0x7f8cc4101e40 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.218+0000 7f8cc967e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc410d910 con 0x7f8cc4102fa0 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.218+0000 7f8cc967e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc410dc10 con 0x7f8cc4103960 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4101e40 0x7f8cc4075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4101e40 0x7f8cc4075b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44524/0 (socket says 172.21.15.37:44524) 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc2ffd700 1 -- 172.21.15.37:0/1186534532 learned_addr learned my addr 172.21.15.37:0/1186534532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc37fe700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cc4103960 0x7f8cc4074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc27fc700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc4102fa0 0x7f8cc40741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc2ffd700 1 -- 172.21.15.37:0/1186534532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cc4103960 msgr2=0x7f8cc4074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc2ffd700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cc4103960 0x7f8cc4074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc2ffd700 1 -- 172.21.15.37:0/1186534532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc4102fa0 msgr2=0x7f8cc40741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:39.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc2ffd700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc4102fa0 0x7f8cc40741d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:39.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc2ffd700 1 -- 172.21.15.37:0/1186534532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4074fc0 con 0x7f8cc4101e40 2024-03-26T06:22:39.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc2ffd700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4101e40 0x7f8cc4075b80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400ba50 tx=0x7f8cb400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:39.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cbbfff700 1 -- 172.21.15.37:0/1186534532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4019070 con 0x7f8cc4101e40 2024-03-26T06:22:39.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cbbfff700 1 -- 172.21.15.37:0/1186534532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cb400ce10 con 0x7f8cc4101e40 2024-03-26T06:22:39.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cbbfff700 1 -- 172.21.15.37:0/1186534532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4007390 con 0x7f8cc4101e40 2024-03-26T06:22:39.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc967e700 1 -- 172.21.15.37:0/1186534532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc41a2140 con 0x7f8cc4101e40 2024-03-26T06:22:39.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.219+0000 7f8cc967e700 1 -- 172.21.15.37:0/1186534532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc41a26d0 con 0x7f8cc4101e40 2024-03-26T06:22:39.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.220+0000 7f8cc967e700 1 -- 172.21.15.37:0/1186534532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc4068aa0 con 0x7f8cc4101e40 2024-03-26T06:22:39.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.221+0000 7f8cbbfff700 1 -- 172.21.15.37:0/1186534532 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8cb401c070 con 0x7f8cc4101e40 2024-03-26T06:22:39.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.222+0000 7f8cbbfff700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8cb00a2910 0x7f8cb00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:39.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.222+0000 7f8cc27fc700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8cb00a2910 0x7f8cb00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:39.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.222+0000 7f8cbbfff700 1 -- 172.21.15.37:0/1186534532 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8cb40ce000 con 0x7f8cc4101e40 2024-03-26T06:22:39.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.222+0000 7f8cc27fc700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8cb00a2910 0x7f8cb00a4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00b5c0 tx=0x7f8cac0038f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:39.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.225+0000 7f8cbbfff700 1 -- 172.21.15.37:0/1186534532 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8cb409b050 con 0x7f8cc4101e40 2024-03-26T06:22:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.432+0000 7f8cc967e700 1 -- 172.21.15.37:0/1186534532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f8cc402d830 con 0x7f8cc4101e40 2024-03-26T06:22:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.433+0000 7f8cbbfff700 1 -- 172.21.15.37:0/1186534532 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5259 (secure 0 0 0) 0x7f8cb409a950 con 0x7f8cc4101e40 2024-03-26T06:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 87 2024-03-26T06:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout:fsid a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-26T06:12:42.888145+0000 2024-03-26T06:22:39.434 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-26T06:21:48.711822+0000 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:crush_version 26 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:full_ratio 0.95 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:backfillfull_ratio 0.9 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:nearfull_ratio 0.85 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:require_min_compat_client luminous 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:min_compat_client luminous 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:require_osd_release squid 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:stretch_mode_enabled false 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-26T06:22:39.435 INFO:teuthology.orchestra.run.smithi037.stdout:max_osd 12 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6802/362526230,v1:172.21.15.37:6803/362526230] [v2:172.21.15.37:6804/362526230,v1:172.21.15.37:6805/362526230] exists,up 16004b33-cf5b-43d7-8c6d-04223f061c9d 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6800/331563667,v1:172.21.15.66:6801/331563667] [v2:172.21.15.66:6802/331563667,v1:172.21.15.66:6803/331563667] exists,up 802bebb3-7af9-45af-bd5e-b5b7e1156403 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/611823377,v1:172.21.15.149:6801/611823377] [v2:172.21.15.149:6802/611823377,v1:172.21.15.149:6803/611823377] exists,up 47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 up in weight 1 up_from 25 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6810/2227972060,v1:172.21.15.37:6811/2227972060] [v2:172.21.15.37:6812/2227972060,v1:172.21.15.37:6813/2227972060] exists,up 68dcbfb5-853c-4843-b19d-5018e9c6e268 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6808/1994195514,v1:172.21.15.66:6809/1994195514] [v2:172.21.15.66:6810/1994195514,v1:172.21.15.66:6811/1994195514] exists,up 4ce571a6-aa1d-40e6-a6d3-df9122391314 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/3463585638,v1:172.21.15.149:6809/3463585638] [v2:172.21.15.149:6810/3463585638,v1:172.21.15.149:6811/3463585638] exists,up c2ade669-ee83-41ad-957b-ea5b88768477 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6818/1971653917,v1:172.21.15.37:6819/1971653917] [v2:172.21.15.37:6820/1971653917,v1:172.21.15.37:6821/1971653917] exists,up 5dd601fe-399f-4ba5-87ab-d602a0930403 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6816/1024919875,v1:172.21.15.66:6817/1024919875] [v2:172.21.15.66:6818/1024919875,v1:172.21.15.66:6819/1024919875] exists,up f84e7511-f0e6-4cd0-b78f-cee1696df1b5 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/192122283,v1:172.21.15.149:6817/192122283] [v2:172.21.15.149:6818/192122283,v1:172.21.15.149:6819/192122283] exists,up 388ebc46-fd6a-48e1-a245-a8fe4c8d937e 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6826/2324500917,v1:172.21.15.37:6827/2324500917] [v2:172.21.15.37:6828/2324500917,v1:172.21.15.37:6829/2324500917] exists,up c030ee66-7e52-4cf8-a8c9-3987bd5e7328 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6824/364342155,v1:172.21.15.66:6825/364342155] [v2:172.21.15.66:6826/364342155,v1:172.21.15.66:6827/364342155] exists,up e7ea7170-f2ed-4d72-afee-7b2828f3a774 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/2983393449,v1:172.21.15.149:6825/2983393449] [v2:172.21.15.149:6826/2983393449,v1:172.21.15.149:6827/2983393449] exists,up 780b3500-9d7a-48c1-8e39-ea0abb98dd5c 2024-03-26T06:22:39.436 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.8 [2,11] 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.16 [2,4] 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.1c [8,11] 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.18 [3,9] 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.1d [5,9] 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.21 [4,6] 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.27 [4,8] 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.2c [4,9] 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/2549717127 expires 2024-03-27T06:13:46.665797+0000 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/2065883347 expires 2024-03-27T06:13:46.665797+0000 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/401419890 expires 2024-03-27T06:13:46.665797+0000 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1548093856 expires 2024-03-27T06:13:18.570869+0000 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/401419890 expires 2024-03-27T06:13:46.665797+0000 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/718373544 expires 2024-03-27T06:13:18.570869+0000 2024-03-26T06:22:39.437 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3042978467 expires 2024-03-27T06:13:18.570869+0000 2024-03-26T06:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/718373544 expires 2024-03-27T06:13:18.570869+0000 2024-03-26T06:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3391028472 expires 2024-03-27T06:13:18.570869+0000 2024-03-26T06:22:39.438 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1382640415 expires 2024-03-27T06:13:46.665797+0000 2024-03-26T06:22:39.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.435+0000 7f8cc967e700 1 -- 172.21.15.37:0/1186534532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8cb00a2910 msgr2=0x7f8cb00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.435+0000 7f8cc967e700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8cb00a2910 0x7f8cb00a4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00b5c0 tx=0x7f8cac0038f0 comp rx=0 tx=0).stop 2024-03-26T06:22:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.435+0000 7f8cc967e700 1 -- 172.21.15.37:0/1186534532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4101e40 msgr2=0x7f8cc4075b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.435+0000 7f8cc967e700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4101e40 0x7f8cc4075b80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400ba50 tx=0x7f8cb400bd60 comp rx=0 tx=0).stop 2024-03-26T06:22:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.435+0000 7f8cc967e700 1 -- 172.21.15.37:0/1186534532 shutdown_connections 2024-03-26T06:22:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.435+0000 7f8cc967e700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4101e40 0x7f8cc4075b80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.435+0000 7f8cc967e700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f8cb00a2910 0x7f8cb00a4dd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.435+0000 7f8cc967e700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cc4103960 0x7f8cc4074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.435+0000 7f8cc967e700 1 --2- 172.21.15.37:0/1186534532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc4102fa0 0x7f8cc40741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.435+0000 7f8cc967e700 1 -- 172.21.15.37:0/1186534532 >> 172.21.15.37:0/1186534532 conn(0x7f8cc40fd3e0 msgr2=0x7f8cc4106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.436+0000 7f8cc967e700 1 -- 172.21.15.37:0/1186534532 shutdown_connections 2024-03-26T06:22:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:39.436+0000 7f8cc967e700 1 -- 172.21.15.37:0/1186534532 wait complete. 2024-03-26T06:22:40.122 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-26T06:22:40.145 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:22:40.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.253+0000 7fa8266c0640 1 -- 172.21.15.37:0/790303465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa820101dc0 msgr2=0x7fa820108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.253+0000 7fa8266c0640 1 --2- 172.21.15.37:0/790303465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa820101dc0 0x7fa820108650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa818009ca0 tx=0x7fa81802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.253+0000 7fa8266c0640 1 -- 172.21.15.37:0/790303465 shutdown_connections 2024-03-26T06:22:40.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.253+0000 7fa8266c0640 1 --2- 172.21.15.37:0/790303465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa820101dc0 0x7fa820108650 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.253+0000 7fa8266c0640 1 --2- 172.21.15.37:0/790303465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820101400 0x7fa820101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.253+0000 7fa8266c0640 1 --2- 172.21.15.37:0/790303465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa820100210 0x7fa820100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.253+0000 7fa8266c0640 1 -- 172.21.15.37:0/790303465 >> 172.21.15.37:0/790303465 conn(0x7fa8200fb780 msgr2=0x7fa8200fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.253+0000 7fa8266c0640 1 -- 172.21.15.37:0/790303465 shutdown_connections 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.253+0000 7fa8266c0640 1 -- 172.21.15.37:0/790303465 wait complete. 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.253+0000 7fa8266c0640 1 Processor -- start 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.253+0000 7fa8266c0640 1 -- start start 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa8266c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820100210 0x7fa820196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa8266c0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa820101400 0x7fa820196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa8266c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa820101dc0 0x7fa82019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa8266c0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82010be70 con 0x7fa820101dc0 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa8266c0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82010bcf0 con 0x7fa820100210 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa8266c0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82010bff0 con 0x7fa820101400 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa824c5d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa820101dc0 0x7fa82019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa824c5d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa820101dc0 0x7fa82019d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44556/0 (socket says 172.21.15.37:44556) 2024-03-26T06:22:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa824c5d640 1 -- 172.21.15.37:0/4276751532 learned_addr learned my addr 172.21.15.37:0/4276751532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa81ffff640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820100210 0x7fa820196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa824c5d640 1 -- 172.21.15.37:0/4276751532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa820101400 msgr2=0x7fa820196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa824c5d640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa820101400 0x7fa820196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa824c5d640 1 -- 172.21.15.37:0/4276751532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820100210 msgr2=0x7fa820196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa824c5d640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820100210 0x7fa820196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa824c5d640 1 -- 172.21.15.37:0/4276751532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82019e040 con 0x7fa820101dc0 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa81ffff640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820100210 0x7fa820196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.254+0000 7fa824c5d640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa820101dc0 0x7fa82019d9d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa818037040 tx=0x7fa818032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.255+0000 7fa81d7fa640 1 -- 172.21.15.37:0/4276751532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818039040 con 0x7fa820101dc0 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.255+0000 7fa81d7fa640 1 -- 172.21.15.37:0/4276751532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa818044070 con 0x7fa820101dc0 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.255+0000 7fa8266c0640 1 -- 172.21.15.37:0/4276751532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82019e2d0 con 0x7fa820101dc0 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.255+0000 7fa81d7fa640 1 -- 172.21.15.37:0/4276751532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818004330 con 0x7fa820101dc0 2024-03-26T06:22:40.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.255+0000 7fa8266c0640 1 -- 172.21.15.37:0/4276751532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82019e810 con 0x7fa820101dc0 2024-03-26T06:22:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.256+0000 7fa8266c0640 1 -- 172.21.15.37:0/4276751532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8201074d0 con 0x7fa820101dc0 2024-03-26T06:22:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.257+0000 7fa81d7fa640 1 -- 172.21.15.37:0/4276751532 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa818003a00 con 0x7fa820101dc0 2024-03-26T06:22:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.257+0000 7fa81d7fa640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa8080a2880 0x7fa8080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.257+0000 7fa81ffff640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa8080a2880 0x7fa8080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:40.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.257+0000 7fa81d7fa640 1 -- 172.21.15.37:0/4276751532 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa8180f1240 con 0x7fa820101dc0 2024-03-26T06:22:40.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.258+0000 7fa81ffff640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa8080a2880 0x7fa8080a4d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa820101260 tx=0x7fa810005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:40.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.262+0000 7fa81d7fa640 1 -- 172.21.15.37:0/4276751532 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa8180be2f0 con 0x7fa820101dc0 2024-03-26T06:22:40.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.447+0000 7fa8266c0640 1 -- 172.21.15.37:0/4276751532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa820061ef0 con 0x7fa820101dc0 2024-03-26T06:22:40.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:40 smithi037 ceph-mon[31974]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:22:40.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:40 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1186534532' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T06:22:40.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.448+0000 7fa81d7fa640 1 -- 172.21.15.37:0/4276751532 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22580 (secure 0 0 0) 0x7fa8180bdbf0 con 0x7fa820101dc0 2024-03-26T06:22:40.450 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:40.450 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":87,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:21:48.711822+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:22:40.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 -- 172.21.15.37:0/4276751532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa8080a2880 msgr2=0x7fa8080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa8080a2880 0x7fa8080a4d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa820101260 tx=0x7fa810005db0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 -- 172.21.15.37:0/4276751532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa820101dc0 msgr2=0x7fa82019d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa820101dc0 0x7fa82019d9d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa818037040 tx=0x7fa818032f90 comp rx=0 tx=0).stop 2024-03-26T06:22:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 -- 172.21.15.37:0/4276751532 shutdown_connections 2024-03-26T06:22:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa8080a2880 0x7fa8080a4d40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa820101dc0 0x7fa82019d9d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa820101400 0x7fa820196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 --2- 172.21.15.37:0/4276751532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa820100210 0x7fa820196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 -- 172.21.15.37:0/4276751532 >> 172.21.15.37:0/4276751532 conn(0x7fa8200fb780 msgr2=0x7fa8201046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 -- 172.21.15.37:0/4276751532 shutdown_connections 2024-03-26T06:22:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.450+0000 7fa8266c0640 1 -- 172.21.15.37:0/4276751532 wait complete. 2024-03-26T06:22:40.463 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T06:16:17.005303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T06:20:51.878592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T06:20:52.836733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T06:22:40.463 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T06:22:40.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 -- 172.21.15.37:0/2111298117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa040101dc0 msgr2=0x7fa040108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 --2- 172.21.15.37:0/2111298117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa040101dc0 0x7fa040108650 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa038009ca0 tx=0x7fa03802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 -- 172.21.15.37:0/2111298117 shutdown_connections 2024-03-26T06:22:40.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 --2- 172.21.15.37:0/2111298117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa040101dc0 0x7fa040108650 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 --2- 172.21.15.37:0/2111298117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa040101400 0x7fa040101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 --2- 172.21.15.37:0/2111298117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa040100210 0x7fa040100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 -- 172.21.15.37:0/2111298117 >> 172.21.15.37:0/2111298117 conn(0x7fa0400fb7a0 msgr2=0x7fa0400fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:40.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 -- 172.21.15.37:0/2111298117 shutdown_connections 2024-03-26T06:22:40.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 -- 172.21.15.37:0/2111298117 wait complete. 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 Processor -- start 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 -- start start 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa040100210 0x7fa040197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa040101400 0x7fa040197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa040101dc0 0x7fa04019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04010bf00 con 0x7fa040100210 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04010bd80 con 0x7fa040101dc0 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.569+0000 7fa046ed1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04010c080 con 0x7fa040101400 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa044c6d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa040100210 0x7fa040197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa044c6d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa040100210 0x7fa040197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44572/0 (socket says 172.21.15.37:44572) 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa044c6d640 1 -- 172.21.15.37:0/1313126407 learned_addr learned my addr 172.21.15.37:0/1313126407 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa04546e640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa040101dc0 0x7fa04019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa03ffff640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa040101400 0x7fa040197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa044c6d640 1 -- 172.21.15.37:0/1313126407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa040101400 msgr2=0x7fa040197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa044c6d640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa040101400 0x7fa040197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa044c6d640 1 -- 172.21.15.37:0/1313126407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa040101dc0 msgr2=0x7fa04019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa044c6d640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa040101dc0 0x7fa04019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa044c6d640 1 -- 172.21.15.37:0/1313126407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04019f260 con 0x7fa040100210 2024-03-26T06:22:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa04546e640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa040101dc0 0x7fa04019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa044c6d640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa040100210 0x7fa040197640 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa03400b450 tx=0x7fa03400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa03dffb640 1 -- 172.21.15.37:0/1313126407 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa034015020 con 0x7fa040100210 2024-03-26T06:22:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa03dffb640 1 -- 172.21.15.37:0/1313126407 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa034011d70 con 0x7fa040100210 2024-03-26T06:22:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa046ed1640 1 -- 172.21.15.37:0/1313126407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04019f550 con 0x7fa040100210 2024-03-26T06:22:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa03dffb640 1 -- 172.21.15.37:0/1313126407 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03400f340 con 0x7fa040100210 2024-03-26T06:22:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.570+0000 7fa046ed1640 1 -- 172.21.15.37:0/1313126407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04019fa50 con 0x7fa040100210 2024-03-26T06:22:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.571+0000 7fa046ed1640 1 -- 172.21.15.37:0/1313126407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04019fd10 con 0x7fa040100210 2024-03-26T06:22:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.572+0000 7fa03dffb640 1 -- 172.21.15.37:0/1313126407 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa034018020 con 0x7fa040100210 2024-03-26T06:22:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.573+0000 7fa03dffb640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa0300a2800 0x7fa0300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.573+0000 7fa03ffff640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa0300a2800 0x7fa0300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.573+0000 7fa03dffb640 1 -- 172.21.15.37:0/1313126407 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa0340ce600 con 0x7fa040100210 2024-03-26T06:22:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.573+0000 7fa03ffff640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa0300a2800 0x7fa0300a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa040198b50 tx=0x7fa02c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:40.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.577+0000 7fa03dffb640 1 -- 172.21.15.37:0/1313126407 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa03401f020 con 0x7fa040100210 2024-03-26T06:22:40.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:40 smithi066 ceph-mon[36504]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:22:40.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:40 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1186534532' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T06:22:40.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:40 smithi149 ceph-mon[36681]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:22:40.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:40 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1186534532' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T06:22:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.763+0000 7fa046ed1640 1 -- 172.21.15.37:0/1313126407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa04019fee0 con 0x7fa040100210 2024-03-26T06:22:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.763+0000 7fa03dffb640 1 -- 172.21.15.37:0/1313126407 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa03400d040 con 0x7fa040100210 2024-03-26T06:22:40.765 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-26T06:22:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 -- 172.21.15.37:0/1313126407 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa0300a2800 msgr2=0x7fa0300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa0300a2800 0x7fa0300a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa040198b50 tx=0x7fa02c00a4c0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 -- 172.21.15.37:0/1313126407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa040100210 msgr2=0x7fa040197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa040100210 0x7fa040197640 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa03400b450 tx=0x7fa03400b910 comp rx=0 tx=0).stop 2024-03-26T06:22:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 -- 172.21.15.37:0/1313126407 shutdown_connections 2024-03-26T06:22:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa0300a2800 0x7fa0300a4cc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa040101dc0 0x7fa04019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa040101400 0x7fa040197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 --2- 172.21.15.37:0/1313126407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa040100210 0x7fa040197640 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 -- 172.21.15.37:0/1313126407 >> 172.21.15.37:0/1313126407 conn(0x7fa0400fb7a0 msgr2=0x7fa0400fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 -- 172.21.15.37:0/1313126407 shutdown_connections 2024-03-26T06:22:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.765+0000 7fa046ed1640 1 -- 172.21.15.37:0/1313126407 wait complete. 2024-03-26T06:22:40.778 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T06:22:40.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.883+0000 7f4d9106f640 1 -- 172.21.15.37:0/2441181463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8c1046c0 msgr2=0x7f4d8c106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.883+0000 7f4d9106f640 1 --2- 172.21.15.37:0/2441181463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8c1046c0 0x7f4d8c106ab0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009900 tx=0x7f4d7402bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:40.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.883+0000 7f4d9106f640 1 -- 172.21.15.37:0/2441181463 shutdown_connections 2024-03-26T06:22:40.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.883+0000 7f4d9106f640 1 --2- 172.21.15.37:0/2441181463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d8c106ff0 0x7f4d8c109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.883+0000 7f4d9106f640 1 --2- 172.21.15.37:0/2441181463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8c1046c0 0x7f4d8c106ab0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.883+0000 7f4d9106f640 1 --2- 172.21.15.37:0/2441181463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d8c101d90 0x7f4d8c104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.883+0000 7f4d9106f640 1 -- 172.21.15.37:0/2441181463 >> 172.21.15.37:0/2441181463 conn(0x7f4d8c0fb7a0 msgr2=0x7f4d8c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:40.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.883+0000 7f4d9106f640 1 -- 172.21.15.37:0/2441181463 shutdown_connections 2024-03-26T06:22:40.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.883+0000 7f4d9106f640 1 -- 172.21.15.37:0/2441181463 wait complete. 2024-03-26T06:22:40.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d9106f640 1 Processor -- start 2024-03-26T06:22:40.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d9106f640 1 -- start start 2024-03-26T06:22:40.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d9106f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d8c101d90 0x7f4d8c1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d9106f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d8c1046c0 0x7f4d8c197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d9106f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8c106ff0 0x7f4d8c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d9106f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c10cfe0 con 0x7f4d8c106ff0 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d9106f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c10ce60 con 0x7f4d8c1046c0 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d9106f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c10d160 con 0x7f4d8c101d90 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d8b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8c106ff0 0x7f4d8c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d8a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d8c1046c0 0x7f4d8c197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d8b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8c106ff0 0x7f4d8c19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44588/0 (socket says 172.21.15.37:44588) 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d8b59e640 1 -- 172.21.15.37:0/2953749822 learned_addr learned my addr 172.21.15.37:0/2953749822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.884+0000 7f4d8ad9d640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d8c101d90 0x7f4d8c1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d8b59e640 1 -- 172.21.15.37:0/2953749822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d8c101d90 msgr2=0x7f4d8c1976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d8b59e640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d8c101d90 0x7f4d8c1976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d8b59e640 1 -- 172.21.15.37:0/2953749822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d8c1046c0 msgr2=0x7f4d8c197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d8b59e640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d8c1046c0 0x7f4d8c197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:40.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d8b59e640 1 -- 172.21.15.37:0/2953749822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c19f2d0 con 0x7f4d8c106ff0 2024-03-26T06:22:40.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d8a59c640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d8c1046c0 0x7f4d8c197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:22:40.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d8b59e640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8c106ff0 0x7f4d8c19ec60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00b970 tx=0x7f4d7c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:40.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d83fff640 1 -- 172.21.15.37:0/2953749822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c019070 con 0x7f4d8c106ff0 2024-03-26T06:22:40.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d9106f640 1 -- 172.21.15.37:0/2953749822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8c19f5c0 con 0x7f4d8c106ff0 2024-03-26T06:22:40.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d83fff640 1 -- 172.21.15.37:0/2953749822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d7c00cb40 con 0x7f4d8c106ff0 2024-03-26T06:22:40.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d83fff640 1 -- 172.21.15.37:0/2953749822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c007650 con 0x7f4d8c106ff0 2024-03-26T06:22:40.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.885+0000 7f4d9106f640 1 -- 172.21.15.37:0/2953749822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8c19fa90 con 0x7f4d8c106ff0 2024-03-26T06:22:40.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.886+0000 7f4d9106f640 1 -- 172.21.15.37:0/2953749822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8c0fd290 con 0x7f4d8c106ff0 2024-03-26T06:22:40.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.887+0000 7f4d83fff640 1 -- 172.21.15.37:0/2953749822 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4d7c01c070 con 0x7f4d8c106ff0 2024-03-26T06:22:40.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.888+0000 7f4d83fff640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4d780a2800 0x7f4d780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:40.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.888+0000 7f4d8ad9d640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4d780a2800 0x7f4d780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:40.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.888+0000 7f4d83fff640 1 -- 172.21.15.37:0/2953749822 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4d7c0ce080 con 0x7f4d8c106ff0 2024-03-26T06:22:40.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.888+0000 7f4d8ad9d640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4d780a2800 0x7f4d780a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400bbe0 tx=0x7f4d8400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:40.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:40.892+0000 7f4d83fff640 1 -- 172.21.15.37:0/2953749822 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4d7c09aa80 con 0x7f4d8c106ff0 2024-03-26T06:22:41.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.074+0000 7f4d9106f640 1 -- 172.21.15.37:0/2953749822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d8c061ef0 con 0x7f4d8c106ff0 2024-03-26T06:22:41.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.075+0000 7f4d83fff640 1 -- 172.21.15.37:0/2953749822 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d7c09eb20 con 0x7f4d8c106ff0 2024-03-26T06:22:41.076 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.077+0000 7f4d9106f640 1 -- 172.21.15.37:0/2953749822 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4d780a2800 msgr2=0x7f4d780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.077+0000 7f4d9106f640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4d780a2800 0x7f4d780a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400bbe0 tx=0x7f4d8400b390 comp rx=0 tx=0).stop 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.077+0000 7f4d9106f640 1 -- 172.21.15.37:0/2953749822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8c106ff0 msgr2=0x7f4d8c19ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.077+0000 7f4d9106f640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8c106ff0 0x7f4d8c19ec60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00b970 tx=0x7f4d7c00be30 comp rx=0 tx=0).stop 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.077+0000 7f4d9106f640 1 -- 172.21.15.37:0/2953749822 shutdown_connections 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.077+0000 7f4d9106f640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4d780a2800 0x7f4d780a4cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.077+0000 7f4d9106f640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8c106ff0 0x7f4d8c19ec60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.077+0000 7f4d9106f640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d8c1046c0 0x7f4d8c197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.077+0000 7f4d9106f640 1 --2- 172.21.15.37:0/2953749822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d8c101d90 0x7f4d8c1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.077+0000 7f4d9106f640 1 -- 172.21.15.37:0/2953749822 >> 172.21.15.37:0/2953749822 conn(0x7f4d8c0fb7a0 msgr2=0x7f4d8c1026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.078+0000 7f4d9106f640 1 -- 172.21.15.37:0/2953749822 shutdown_connections 2024-03-26T06:22:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.078+0000 7f4d9106f640 1 -- 172.21.15.37:0/2953749822 wait complete. 2024-03-26T06:22:41.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T06:22:41.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.194+0000 7f39cc08c640 1 -- 172.21.15.37:0/3034628884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39c4101300 msgr2=0x7f39c4101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.194+0000 7f39cc08c640 1 --2- 172.21.15.37:0/3034628884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39c4101300 0x7f39c4101720 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009900 tx=0x7f39bc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:41.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.194+0000 7f39cc08c640 1 -- 172.21.15.37:0/3034628884 shutdown_connections 2024-03-26T06:22:41.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.194+0000 7f39cc08c640 1 --2- 172.21.15.37:0/3034628884 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39c4102eb0 0x7f39c4109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.194+0000 7f39cc08c640 1 --2- 172.21.15.37:0/3034628884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39c41024f0 0x7f39c4102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.194+0000 7f39cc08c640 1 --2- 172.21.15.37:0/3034628884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39c4101300 0x7f39c4101720 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.194+0000 7f39cc08c640 1 -- 172.21.15.37:0/3034628884 >> 172.21.15.37:0/3034628884 conn(0x7f39c40fc890 msgr2=0x7f39c40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:41.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.194+0000 7f39cc08c640 1 -- 172.21.15.37:0/3034628884 shutdown_connections 2024-03-26T06:22:41.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.194+0000 7f39cc08c640 1 -- 172.21.15.37:0/3034628884 wait complete. 2024-03-26T06:22:41.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39cc08c640 1 Processor -- start 2024-03-26T06:22:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39cc08c640 1 -- start start 2024-03-26T06:22:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39cc08c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39c4101300 0x7f39c40750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39cc08c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39c41024f0 0x7f39c40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39cc08c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39c4102eb0 0x7f39c4073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39cc08c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c410cf90 con 0x7f39c4102eb0 2024-03-26T06:22:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39cc08c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c410ce10 con 0x7f39c41024f0 2024-03-26T06:22:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39cc08c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c410d110 con 0x7f39c4101300 2024-03-26T06:22:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39ca629640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39c4102eb0 0x7f39c4073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39ca629640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39c4102eb0 0x7f39c4073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44616/0 (socket says 172.21.15.37:44616) 2024-03-26T06:22:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39ca629640 1 -- 172.21.15.37:0/1223872947 learned_addr learned my addr 172.21.15.37:0/1223872947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39c9627640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39c41024f0 0x7f39c40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39ca629640 1 -- 172.21.15.37:0/1223872947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39c4101300 msgr2=0x7f39c40750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39ca629640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39c4101300 0x7f39c40750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39ca629640 1 -- 172.21.15.37:0/1223872947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39c41024f0 msgr2=0x7f39c40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39ca629640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39c41024f0 0x7f39c40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.195+0000 7f39ca629640 1 -- 172.21.15.37:0/1223872947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c40744b0 con 0x7f39c4102eb0 2024-03-26T06:22:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.196+0000 7f39ca629640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39c4102eb0 0x7f39c4073c20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f39c000eaa0 tx=0x7f39c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.196+0000 7f39baffd640 1 -- 172.21.15.37:0/1223872947 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000cdb0 con 0x7f39c4102eb0 2024-03-26T06:22:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.196+0000 7f39baffd640 1 -- 172.21.15.37:0/1223872947 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39c0005020 con 0x7f39c4102eb0 2024-03-26T06:22:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.196+0000 7f39cc08c640 1 -- 172.21.15.37:0/1223872947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c41a1770 con 0x7f39c4102eb0 2024-03-26T06:22:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.196+0000 7f39baffd640 1 -- 172.21.15.37:0/1223872947 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c00116b0 con 0x7f39c4102eb0 2024-03-26T06:22:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.196+0000 7f39cc08c640 1 -- 172.21.15.37:0/1223872947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c41a1c70 con 0x7f39c4102eb0 2024-03-26T06:22:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.197+0000 7f39cc08c640 1 -- 172.21.15.37:0/1223872947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c4108690 con 0x7f39c4102eb0 2024-03-26T06:22:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.198+0000 7f39baffd640 1 -- 172.21.15.37:0/1223872947 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f39c0005560 con 0x7f39c4102eb0 2024-03-26T06:22:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.198+0000 7f39baffd640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f39b00a2800 0x7f39b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.198+0000 7f39c9e28640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f39b00a2800 0x7f39b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.199+0000 7f39baffd640 1 -- 172.21.15.37:0/1223872947 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f39c00cf110 con 0x7f39c4102eb0 2024-03-26T06:22:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.199+0000 7f39c9e28640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f39b00a2800 0x7f39b00a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f39bc036040 tx=0x7f39bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.202+0000 7f39baffd640 1 -- 172.21.15.37:0/1223872947 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f39c0016030 con 0x7f39c4102eb0 2024-03-26T06:22:41.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.387+0000 7f39cc08c640 1 -- 172.21.15.37:0/1223872947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39c4061ef0 con 0x7f39c4102eb0 2024-03-26T06:22:41.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.387+0000 7f39baffd640 1 -- 172.21.15.37:0/1223872947 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f39c009c1c0 con 0x7f39c4102eb0 2024-03-26T06:22:41.388 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-26T06:22:41.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 -- 172.21.15.37:0/1223872947 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f39b00a2800 msgr2=0x7f39b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f39b00a2800 0x7f39b00a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f39bc036040 tx=0x7f39bc000bc0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 -- 172.21.15.37:0/1223872947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39c4102eb0 msgr2=0x7f39c4073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39c4102eb0 0x7f39c4073c20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f39c000eaa0 tx=0x7f39c000ef60 comp rx=0 tx=0).stop 2024-03-26T06:22:41.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 -- 172.21.15.37:0/1223872947 shutdown_connections 2024-03-26T06:22:41.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f39b00a2800 0x7f39b00a4cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39c4102eb0 0x7f39c4073c20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f39c41024f0 0x7f39c40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 --2- 172.21.15.37:0/1223872947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39c4101300 0x7f39c40750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 -- 172.21.15.37:0/1223872947 >> 172.21.15.37:0/1223872947 conn(0x7f39c40fc890 msgr2=0x7f39c4105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:41.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 -- 172.21.15.37:0/1223872947 shutdown_connections 2024-03-26T06:22:41.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.389+0000 7f39cc08c640 1 -- 172.21.15.37:0/1223872947 wait complete. 2024-03-26T06:22:41.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:41 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/4276751532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:41.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:41 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1313126407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:41.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:41 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2953749822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:41.401 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:22:41.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.503+0000 7fd9e77df640 1 -- 172.21.15.37:0/160386582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e0101310 msgr2=0x7fd9e0101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.503+0000 7fd9e77df640 1 --2- 172.21.15.37:0/160386582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e0101310 0x7fd9e0101730 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009900 tx=0x7fd9dc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:41.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.503+0000 7fd9e77df640 1 -- 172.21.15.37:0/160386582 shutdown_connections 2024-03-26T06:22:41.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.503+0000 7fd9e77df640 1 --2- 172.21.15.37:0/160386582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9e0102ec0 0x7fd9e0109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.503+0000 7fd9e77df640 1 --2- 172.21.15.37:0/160386582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9e0102500 0x7fd9e0102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.503+0000 7fd9e77df640 1 --2- 172.21.15.37:0/160386582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e0101310 0x7fd9e0101730 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.503+0000 7fd9e77df640 1 -- 172.21.15.37:0/160386582 >> 172.21.15.37:0/160386582 conn(0x7fd9e00fc8a0 msgr2=0x7fd9e00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:41.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.503+0000 7fd9e77df640 1 -- 172.21.15.37:0/160386582 shutdown_connections 2024-03-26T06:22:41.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.503+0000 7fd9e77df640 1 -- 172.21.15.37:0/160386582 wait complete. 2024-03-26T06:22:41.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e77df640 1 Processor -- start 2024-03-26T06:22:41.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e77df640 1 -- start start 2024-03-26T06:22:41.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e77df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e0101310 0x7fd9e00750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e77df640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9e0102500 0x7fd9e00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e77df640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9e0102ec0 0x7fd9e0073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e77df640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e010cfe0 con 0x7fd9e0101310 2024-03-26T06:22:41.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e77df640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e010ce60 con 0x7fd9e0102500 2024-03-26T06:22:41.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e77df640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e010d160 con 0x7fd9e0102ec0 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e557b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e0101310 0x7fd9e00750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e557b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e0101310 0x7fd9e00750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44630/0 (socket says 172.21.15.37:44630) 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e557b640 1 -- 172.21.15.37:0/2983704354 learned_addr learned my addr 172.21.15.37:0/2983704354 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e4d7a640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9e0102500 0x7fd9e00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e557b640 1 -- 172.21.15.37:0/2983704354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9e0102ec0 msgr2=0x7fd9e0073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e557b640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9e0102ec0 0x7fd9e0073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e557b640 1 -- 172.21.15.37:0/2983704354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9e0102500 msgr2=0x7fd9e00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e557b640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9e0102500 0x7fd9e00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e557b640 1 -- 172.21.15.37:0/2983704354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00744b0 con 0x7fd9e0101310 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e557b640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e0101310 0x7fd9e00750a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc005ab0 tx=0x7fd9dc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9d67fc640 1 -- 172.21.15.37:0/2983704354 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc041070 con 0x7fd9e0101310 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9d67fc640 1 -- 172.21.15.37:0/2983704354 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9dc0054e0 con 0x7fd9e0101310 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9e77df640 1 -- 172.21.15.37:0/2983704354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e01a1740 con 0x7fd9e0101310 2024-03-26T06:22:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.504+0000 7fd9d67fc640 1 -- 172.21.15.37:0/2983704354 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc034440 con 0x7fd9e0101310 2024-03-26T06:22:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.505+0000 7fd9e77df640 1 -- 172.21.15.37:0/2983704354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e01a1ce0 con 0x7fd9e0101310 2024-03-26T06:22:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.505+0000 7fd9e77df640 1 -- 172.21.15.37:0/2983704354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e0108630 con 0x7fd9e0101310 2024-03-26T06:22:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.506+0000 7fd9d67fc640 1 -- 172.21.15.37:0/2983704354 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd9dc044070 con 0x7fd9e0101310 2024-03-26T06:22:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.507+0000 7fd9d67fc640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd9cc0a2880 0x7fd9cc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.507+0000 7fd9e4d7a640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd9cc0a2880 0x7fd9cc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.507+0000 7fd9d67fc640 1 -- 172.21.15.37:0/2983704354 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd9dc0f0870 con 0x7fd9e0101310 2024-03-26T06:22:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.507+0000 7fd9e4d7a640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd9cc0a2880 0x7fd9cc0a4d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0074e10 tx=0x7fd9d000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:41.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.511+0000 7fd9d67fc640 1 -- 172.21.15.37:0/2983704354 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd9dc0bd920 con 0x7fd9e0101310 2024-03-26T06:22:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:41 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/4276751532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:41 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1313126407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:41.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:41 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2953749822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:41.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:41 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/4276751532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:41.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:41 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1313126407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:41.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:41 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2953749822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:41.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.732+0000 7fd9e77df640 1 -- 172.21.15.37:0/2983704354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9e002ce70 con 0x7fd9e0101310 2024-03-26T06:22:41.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.733+0000 7fd9d67fc640 1 -- 172.21.15.37:0/2983704354 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11385 (secure 0 0 0) 0x7fd9dc0bd220 con 0x7fd9e0101310 2024-03-26T06:22:41.734 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:41.734 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T06:20:53.631953+0000","modified":"2024-03-26T06:21:00.587013+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24494,"mds_2":24470,"mds_3":24469,"mds_4":14682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14670":{"gid":14670,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/1944002831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1944002831},{"type":"v1","addr":"172.21.15.37:6839","nonce":1944002831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14682":{"gid":14682,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6841/933565564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":933565564},{"type":"v1","addr":"172.21.15.37:6841","nonce":933565564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24394":{"gid":24394,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/1420321367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1420321367},{"type":"v1","addr":"172.21.15.66:6839","nonce":1420321367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-26T06:22:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-03-26T06:22:41.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 -- 172.21.15.37:0/2983704354 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd9cc0a2880 msgr2=0x7fd9cc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd9cc0a2880 0x7fd9cc0a4d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0074e10 tx=0x7fd9d000b390 comp rx=0 tx=0).stop 2024-03-26T06:22:41.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 -- 172.21.15.37:0/2983704354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e0101310 msgr2=0x7fd9e00750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e0101310 0x7fd9e00750a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc005ab0 tx=0x7fd9dc005dc0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 -- 172.21.15.37:0/2983704354 shutdown_connections 2024-03-26T06:22:41.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd9cc0a2880 0x7fd9cc0a4d40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9e0102ec0 0x7fd9e0073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9e0102500 0x7fd9e00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 --2- 172.21.15.37:0/2983704354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9e0101310 0x7fd9e00750a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 -- 172.21.15.37:0/2983704354 >> 172.21.15.37:0/2983704354 conn(0x7fd9e00fc8a0 msgr2=0x7fd9e0105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 -- 172.21.15.37:0/2983704354 shutdown_connections 2024-03-26T06:22:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.735+0000 7fd9e77df640 1 -- 172.21.15.37:0/2983704354 wait complete. 2024-03-26T06:22:41.748 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:22:41.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 -- 172.21.15.37:0/3793479011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa12c0ff390 msgr2=0x7fa12c0ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 --2- 172.21.15.37:0/3793479011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa12c0ff390 0x7fa12c0ff810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa118009900 tx=0x7fa11802bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 -- 172.21.15.37:0/3793479011 shutdown_connections 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 --2- 172.21.15.37:0/3793479011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c0ffd50 0x7fa12c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 --2- 172.21.15.37:0/3793479011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa12c0ff390 0x7fa12c0ff810 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 --2- 172.21.15.37:0/3793479011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa12c102920 0x7fa12c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 -- 172.21.15.37:0/3793479011 >> 172.21.15.37:0/3793479011 conn(0x7fa12c0faf30 msgr2=0x7fa12c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 -- 172.21.15.37:0/3793479011 shutdown_connections 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 -- 172.21.15.37:0/3793479011 wait complete. 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 Processor -- start 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 -- start start 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa12c0ff390 0x7fa12c197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa12c0ffd50 0x7fa12c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.850+0000 7fa132cdc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c102920 0x7fa12c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa132cdc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c10d000 con 0x7fa12c0ff390 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa132cdc640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c10ce80 con 0x7fa12c0ffd50 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa132cdc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c10d180 con 0x7fa12c102920 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa130a78640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa12c0ff390 0x7fa12c197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa130a78640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa12c0ff390 0x7fa12c197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44650/0 (socket says 172.21.15.37:44650) 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa130a78640 1 -- 172.21.15.37:0/123509831 learned_addr learned my addr 172.21.15.37:0/123509831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa12bfff640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa12c0ffd50 0x7fa12c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa131279640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c102920 0x7fa12c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa130a78640 1 -- 172.21.15.37:0/123509831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c102920 msgr2=0x7fa12c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa130a78640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c102920 0x7fa12c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa130a78640 1 -- 172.21.15.37:0/123509831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa12c0ffd50 msgr2=0x7fa12c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:41.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa130a78640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa12c0ffd50 0x7fa12c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa130a78640 1 -- 172.21.15.37:0/123509831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c19f260 con 0x7fa12c0ff390 2024-03-26T06:22:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa12bfff640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa12c0ffd50 0x7fa12c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa131279640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c102920 0x7fa12c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa130a78640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa12c0ff390 0x7fa12c197640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa12000e9c0 tx=0x7fa12000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa129ffb640 1 -- 172.21.15.37:0/123509831 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12000cd80 con 0x7fa12c0ff390 2024-03-26T06:22:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa129ffb640 1 -- 172.21.15.37:0/123509831 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1200035f0 con 0x7fa12c0ff390 2024-03-26T06:22:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa132cdc640 1 -- 172.21.15.37:0/123509831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12c19f550 con 0x7fa12c0ff390 2024-03-26T06:22:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa129ffb640 1 -- 172.21.15.37:0/123509831 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120010690 con 0x7fa12c0ff390 2024-03-26T06:22:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.851+0000 7fa132cdc640 1 -- 172.21.15.37:0/123509831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c19fac0 con 0x7fa12c0ff390 2024-03-26T06:22:41.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.852+0000 7fa132cdc640 1 -- 172.21.15.37:0/123509831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12c1087e0 con 0x7fa12c0ff390 2024-03-26T06:22:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.853+0000 7fa129ffb640 1 -- 172.21.15.37:0/123509831 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa1200037a0 con 0x7fa12c0ff390 2024-03-26T06:22:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.854+0000 7fa129ffb640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa11c0a2880 0x7fa11c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.854+0000 7fa12bfff640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa11c0a2880 0x7fa11c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.854+0000 7fa129ffb640 1 -- 172.21.15.37:0/123509831 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa1200cef50 con 0x7fa12c0ff390 2024-03-26T06:22:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.854+0000 7fa12bfff640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa11c0a2880 0x7fa11c0a4d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa118006010 tx=0x7fa11802c8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:41.858+0000 7fa129ffb640 1 -- 172.21.15.37:0/123509831 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa12009bfa0 con 0x7fa12c0ff390 2024-03-26T06:22:42.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.045+0000 7fa132cdc640 1 -- 172.21.15.37:0/123509831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa12c061ef0 con 0x7fa12c0ff390 2024-03-26T06:22:42.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.046+0000 7fa129ffb640 1 -- 172.21.15.37:0/123509831 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22580 (secure 0 0 0) 0x7fa12009b8a0 con 0x7fa12c0ff390 2024-03-26T06:22:42.048 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:42.048 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":87,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:21:48.711822+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:22:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 -- 172.21.15.37:0/123509831 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa11c0a2880 msgr2=0x7fa11c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa11c0a2880 0x7fa11c0a4d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa118006010 tx=0x7fa11802c8a0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 -- 172.21.15.37:0/123509831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa12c0ff390 msgr2=0x7fa12c197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa12c0ff390 0x7fa12c197640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa12000e9c0 tx=0x7fa12000ee80 comp rx=0 tx=0).stop 2024-03-26T06:22:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 -- 172.21.15.37:0/123509831 shutdown_connections 2024-03-26T06:22:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fa11c0a2880 0x7fa11c0a4d40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c102920 0x7fa12c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa12c0ffd50 0x7fa12c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 --2- 172.21.15.37:0/123509831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa12c0ff390 0x7fa12c197640 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 -- 172.21.15.37:0/123509831 >> 172.21.15.37:0/123509831 conn(0x7fa12c0faf30 msgr2=0x7fa12c1059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 -- 172.21.15.37:0/123509831 shutdown_connections 2024-03-26T06:22:42.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.049+0000 7fa132cdc640 1 -- 172.21.15.37:0/123509831 wait complete. 2024-03-26T06:22:42.062 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T06:16:17.005303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T06:20:51.878592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T06:20:52.836733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T06:22:42.062 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T06:22:42.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/2938018409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881012e0 msgr2=0x7fcf88101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 --2- 172.21.15.37:0/2938018409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881012e0 0x7fcf88101760 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009900 tx=0x7fcf7802bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/2938018409 shutdown_connections 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 --2- 172.21.15.37:0/2938018409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf88101ca0 0x7fcf88108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 --2- 172.21.15.37:0/2938018409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881012e0 0x7fcf88101760 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 --2- 172.21.15.37:0/2938018409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf881000f0 0x7fcf88100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/2938018409 >> 172.21.15.37:0/2938018409 conn(0x7fcf880fb660 msgr2=0x7fcf880fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/2938018409 shutdown_connections 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/2938018409 wait complete. 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 Processor -- start 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 -- start start 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.173+0000 7fcf8dfa2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881000f0 0x7fcf8806a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8dfa2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf881012e0 0x7fcf8819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8dfa2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf88101ca0 0x7fcf8819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8dfa2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8810bdd0 con 0x7fcf881000f0 2024-03-26T06:22:42.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8dfa2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8810bc50 con 0x7fcf88101ca0 2024-03-26T06:22:42.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8dfa2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8810bf50 con 0x7fcf881012e0 2024-03-26T06:22:42.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8cfa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881000f0 0x7fcf8806a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8cfa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881000f0 0x7fcf8806a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44664/0 (socket says 172.21.15.37:44664) 2024-03-26T06:22:42.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8cfa0640 1 -- 172.21.15.37:0/77681506 learned_addr learned my addr 172.21.15.37:0/77681506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:42.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8d7a1640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf88101ca0 0x7fcf8819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf87fff640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf881012e0 0x7fcf8819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8cfa0640 1 -- 172.21.15.37:0/77681506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf881012e0 msgr2=0x7fcf8819c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8cfa0640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf881012e0 0x7fcf8819c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8cfa0640 1 -- 172.21.15.37:0/77681506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf88101ca0 msgr2=0x7fcf8819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8cfa0640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf88101ca0 0x7fcf8819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8cfa0640 1 -- 172.21.15.37:0/77681506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8819f0c0 con 0x7fcf881000f0 2024-03-26T06:22:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8d7a1640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf88101ca0 0x7fcf8819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8cfa0640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881000f0 0x7fcf8806a3d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000eaa0 tx=0x7fcf8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf85ffb640 1 -- 172.21.15.37:0/77681506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80016070 con 0x7fcf881000f0 2024-03-26T06:22:42.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf85ffb640 1 -- 172.21.15.37:0/77681506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf80003e50 con 0x7fcf881000f0 2024-03-26T06:22:42.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/77681506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8819f3b0 con 0x7fcf881000f0 2024-03-26T06:22:42.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf85ffb640 1 -- 172.21.15.37:0/77681506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf800116d0 con 0x7fcf881000f0 2024-03-26T06:22:42.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.174+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/77681506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8819f8d0 con 0x7fcf881000f0 2024-03-26T06:22:42.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.175+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/77681506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf881073b0 con 0x7fcf881000f0 2024-03-26T06:22:42.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.176+0000 7fcf85ffb640 1 -- 172.21.15.37:0/77681506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fcf80019070 con 0x7fcf881000f0 2024-03-26T06:22:42.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.177+0000 7fcf85ffb640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcf740a2880 0x7fcf740a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.177+0000 7fcf87fff640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcf740a2880 0x7fcf740a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.177+0000 7fcf85ffb640 1 -- 172.21.15.37:0/77681506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcf800ceb80 con 0x7fcf881000f0 2024-03-26T06:22:42.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.177+0000 7fcf87fff640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcf740a2880 0x7fcf740a4d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcf7802c240 tx=0x7fcf7802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:42.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.181+0000 7fcf85ffb640 1 -- 172.21.15.37:0/77681506 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fcf8009bbd0 con 0x7fcf881000f0 2024-03-26T06:22:42.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.363+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/77681506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf88061ef0 con 0x7fcf881000f0 2024-03-26T06:22:42.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.363+0000 7fcf85ffb640 1 -- 172.21.15.37:0/77681506 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf8009b4d0 con 0x7fcf881000f0 2024-03-26T06:22:42.365 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.365+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/77681506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcf740a2880 msgr2=0x7fcf740a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.365+0000 7fcf8dfa2640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcf740a2880 0x7fcf740a4d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcf7802c240 tx=0x7fcf7802df90 comp rx=0 tx=0).stop 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.365+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/77681506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881000f0 msgr2=0x7fcf8806a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.365+0000 7fcf8dfa2640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881000f0 0x7fcf8806a3d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000eaa0 tx=0x7fcf8000ef60 comp rx=0 tx=0).stop 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.366+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/77681506 shutdown_connections 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.366+0000 7fcf8dfa2640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fcf740a2880 0x7fcf740a4d40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.366+0000 7fcf8dfa2640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf88101ca0 0x7fcf8819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.366+0000 7fcf8dfa2640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf881012e0 0x7fcf8819c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.366+0000 7fcf8dfa2640 1 --2- 172.21.15.37:0/77681506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881000f0 0x7fcf8806a3d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.366+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/77681506 >> 172.21.15.37:0/77681506 conn(0x7fcf880fb660 msgr2=0x7fcf88104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.366+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/77681506 shutdown_connections 2024-03-26T06:22:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.366+0000 7fcf8dfa2640 1 -- 172.21.15.37:0/77681506 wait complete. 2024-03-26T06:22:42.376 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:42 smithi037 ceph-mon[31974]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:22:42.377 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:42 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1223872947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:42.377 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:42 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2983704354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:42.377 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:42 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/123509831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:42.378 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T06:22:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.483+0000 7f4acb75b640 1 -- 172.21.15.37:0/3389314494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac4100200 msgr2=0x7f4ac4100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.483+0000 7f4acb75b640 1 --2- 172.21.15.37:0/3389314494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac4100200 0x7f4ac4100620 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009900 tx=0x7f4ac002bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.483+0000 7f4acb75b640 1 -- 172.21.15.37:0/3389314494 shutdown_connections 2024-03-26T06:22:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.483+0000 7f4acb75b640 1 --2- 172.21.15.37:0/3389314494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ac4101db0 0x7f4ac4108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.483+0000 7f4acb75b640 1 --2- 172.21.15.37:0/3389314494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac41013f0 0x7f4ac4101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.483+0000 7f4acb75b640 1 --2- 172.21.15.37:0/3389314494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac4100200 0x7f4ac4100620 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.483+0000 7f4acb75b640 1 -- 172.21.15.37:0/3389314494 >> 172.21.15.37:0/3389314494 conn(0x7f4ac40fb790 msgr2=0x7f4ac40fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.483+0000 7f4acb75b640 1 -- 172.21.15.37:0/3389314494 shutdown_connections 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.483+0000 7f4acb75b640 1 -- 172.21.15.37:0/3389314494 wait complete. 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.483+0000 7f4acb75b640 1 Processor -- start 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.483+0000 7f4acb75b640 1 -- start start 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4acb75b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac4100200 0x7f4ac41964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4acb75b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac41013f0 0x7f4ac41969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4acb75b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ac4101db0 0x7f4ac419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4acb75b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac410be90 con 0x7f4ac4100200 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4acb75b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac410bd10 con 0x7f4ac41013f0 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4acb75b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac410c010 con 0x7f4ac4101db0 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4ac94f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac4100200 0x7f4ac41964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4ac94f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac4100200 0x7f4ac41964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44688/0 (socket says 172.21.15.37:44688) 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4ac94f7640 1 -- 172.21.15.37:0/2924296816 learned_addr learned my addr 172.21.15.37:0/2924296816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4ac8cf6640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac41013f0 0x7f4ac41969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4ac9cf8640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ac4101db0 0x7f4ac419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4ac94f7640 1 -- 172.21.15.37:0/2924296816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ac4101db0 msgr2=0x7f4ac419da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4ac94f7640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ac4101db0 0x7f4ac419da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4ac94f7640 1 -- 172.21.15.37:0/2924296816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac41013f0 msgr2=0x7f4ac41969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4ac94f7640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac41013f0 0x7f4ac41969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4ac94f7640 1 -- 172.21.15.37:0/2924296816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac419e0d0 con 0x7f4ac4100200 2024-03-26T06:22:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4ac94f7640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac4100200 0x7f4ac41964b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0005ab0 tx=0x7f4ac0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4aba7fc640 1 -- 172.21.15.37:0/2924296816 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0041070 con 0x7f4ac4100200 2024-03-26T06:22:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4aba7fc640 1 -- 172.21.15.37:0/2924296816 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ac00054e0 con 0x7f4ac4100200 2024-03-26T06:22:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4acb75b640 1 -- 172.21.15.37:0/2924296816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac40698c0 con 0x7f4ac4100200 2024-03-26T06:22:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4aba7fc640 1 -- 172.21.15.37:0/2924296816 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0034440 con 0x7f4ac4100200 2024-03-26T06:22:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.484+0000 7f4acb75b640 1 -- 172.21.15.37:0/2924296816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac4069db0 con 0x7f4ac4100200 2024-03-26T06:22:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.485+0000 7f4acb75b640 1 -- 172.21.15.37:0/2924296816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac4068530 con 0x7f4ac4100200 2024-03-26T06:22:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.486+0000 7f4aba7fc640 1 -- 172.21.15.37:0/2924296816 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4ac0044070 con 0x7f4ac4100200 2024-03-26T06:22:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.487+0000 7f4aba7fc640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4ab00a2800 0x7f4ab00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.487+0000 7f4ac8cf6640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4ab00a2800 0x7f4ab00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.487+0000 7f4aba7fc640 1 -- 172.21.15.37:0/2924296816 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4ac00f05f0 con 0x7f4ac4100200 2024-03-26T06:22:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.487+0000 7f4ac8cf6640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4ab00a2800 0x7f4ab00a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4ac41979c0 tx=0x7f4ab400e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.491+0000 7f4aba7fc640 1 -- 172.21.15.37:0/2924296816 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4ac00bd6a0 con 0x7f4ac4100200 2024-03-26T06:22:42.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:42 smithi066 ceph-mon[36504]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:22:42.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:42 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1223872947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:42.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:42 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2983704354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:42.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:42 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/123509831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:42.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:42 smithi149 ceph-mon[36681]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:22:42.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:42 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1223872947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:42.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:42 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2983704354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:42.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:42 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/123509831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:42.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.676+0000 7f4acb75b640 1 -- 172.21.15.37:0/2924296816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ac419e360 con 0x7f4ac4100200 2024-03-26T06:22:42.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.676+0000 7f4aba7fc640 1 -- 172.21.15.37:0/2924296816 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ac00bcfa0 con 0x7f4ac4100200 2024-03-26T06:22:42.677 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 -- 172.21.15.37:0/2924296816 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4ab00a2800 msgr2=0x7f4ab00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4ab00a2800 0x7f4ab00a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4ac41979c0 tx=0x7f4ab400e040 comp rx=0 tx=0).stop 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 -- 172.21.15.37:0/2924296816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac4100200 msgr2=0x7f4ac41964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac4100200 0x7f4ac41964b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0005ab0 tx=0x7f4ac0005dc0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 -- 172.21.15.37:0/2924296816 shutdown_connections 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f4ab00a2800 0x7f4ab00a4cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ac4101db0 0x7f4ac419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ac41013f0 0x7f4ac41969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 --2- 172.21.15.37:0/2924296816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac4100200 0x7f4ac41964b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 -- 172.21.15.37:0/2924296816 >> 172.21.15.37:0/2924296816 conn(0x7f4ac40fb790 msgr2=0x7f4ac4104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 -- 172.21.15.37:0/2924296816 shutdown_connections 2024-03-26T06:22:42.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.678+0000 7f4acb75b640 1 -- 172.21.15.37:0/2924296816 wait complete. 2024-03-26T06:22:42.691 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T06:22:42.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.795+0000 7fb857f7e640 1 -- 172.21.15.37:0/498406559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb850101300 msgr2=0x7fb850101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.795+0000 7fb857f7e640 1 --2- 172.21.15.37:0/498406559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb850101300 0x7fb850101720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009900 tx=0x7fb84c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:42.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.795+0000 7fb857f7e640 1 -- 172.21.15.37:0/498406559 shutdown_connections 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.795+0000 7fb857f7e640 1 --2- 172.21.15.37:0/498406559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb850102eb0 0x7fb850109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.795+0000 7fb857f7e640 1 --2- 172.21.15.37:0/498406559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8501024f0 0x7fb850102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.795+0000 7fb857f7e640 1 --2- 172.21.15.37:0/498406559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb850101300 0x7fb850101720 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.795+0000 7fb857f7e640 1 -- 172.21.15.37:0/498406559 >> 172.21.15.37:0/498406559 conn(0x7fb8500fc890 msgr2=0x7fb8500fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.795+0000 7fb857f7e640 1 -- 172.21.15.37:0/498406559 shutdown_connections 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.795+0000 7fb857f7e640 1 -- 172.21.15.37:0/498406559 wait complete. 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb857f7e640 1 Processor -- start 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb857f7e640 1 -- start start 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb857f7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb850101300 0x7fb8500750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb857f7e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8501024f0 0x7fb8500736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb857f7e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb850102eb0 0x7fb850073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb857f7e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85010cf90 con 0x7fb850101300 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb857f7e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85010ce10 con 0x7fb850102eb0 2024-03-26T06:22:42.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb857f7e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85010d110 con 0x7fb8501024f0 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb855d1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb850101300 0x7fb8500750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb855d1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb850101300 0x7fb8500750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44700/0 (socket says 172.21.15.37:44700) 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb855d1a640 1 -- 172.21.15.37:0/3339394551 learned_addr learned my addr 172.21.15.37:0/3339394551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb85651b640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb850102eb0 0x7fb850073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb855519640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8501024f0 0x7fb8500736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb855d1a640 1 -- 172.21.15.37:0/3339394551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8501024f0 msgr2=0x7fb8500736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb855d1a640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8501024f0 0x7fb8500736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb855d1a640 1 -- 172.21.15.37:0/3339394551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb850102eb0 msgr2=0x7fb850073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb855d1a640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb850102eb0 0x7fb850073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb855d1a640 1 -- 172.21.15.37:0/3339394551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8500744b0 con 0x7fb850101300 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb855519640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8501024f0 0x7fb8500736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb85651b640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb850102eb0 0x7fb850073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.796+0000 7fb855d1a640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb850101300 0x7fb8500750a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb84c000c00 tx=0x7fb84c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.797+0000 7fb846ffd640 1 -- 172.21.15.37:0/3339394551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c041070 con 0x7fb850101300 2024-03-26T06:22:42.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.797+0000 7fb846ffd640 1 -- 172.21.15.37:0/3339394551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb84c0044d0 con 0x7fb850101300 2024-03-26T06:22:42.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.797+0000 7fb857f7e640 1 -- 172.21.15.37:0/3339394551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8501a16d0 con 0x7fb850101300 2024-03-26T06:22:42.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.797+0000 7fb846ffd640 1 -- 172.21.15.37:0/3339394551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c034440 con 0x7fb850101300 2024-03-26T06:22:42.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.797+0000 7fb857f7e640 1 -- 172.21.15.37:0/3339394551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8501a1c20 con 0x7fb850101300 2024-03-26T06:22:42.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.797+0000 7fb857f7e640 1 -- 172.21.15.37:0/3339394551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8501086a0 con 0x7fb850101300 2024-03-26T06:22:42.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.799+0000 7fb846ffd640 1 -- 172.21.15.37:0/3339394551 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb84c044070 con 0x7fb850101300 2024-03-26T06:22:42.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.799+0000 7fb846ffd640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb83c0a2800 0x7fb83c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:42.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.799+0000 7fb855519640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb83c0a2800 0x7fb83c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:42.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.799+0000 7fb846ffd640 1 -- 172.21.15.37:0/3339394551 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb84c0f08c0 con 0x7fb850101300 2024-03-26T06:22:42.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.800+0000 7fb855519640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb83c0a2800 0x7fb83c0a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb850074e10 tx=0x7fb840005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:42.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.803+0000 7fb846ffd640 1 -- 172.21.15.37:0/3339394551 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb84c0bd970 con 0x7fb850101300 2024-03-26T06:22:42.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.985+0000 7fb857f7e640 1 -- 172.21.15.37:0/3339394551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb85002d000 con 0x7fb850101300 2024-03-26T06:22:42.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.985+0000 7fb846ffd640 1 -- 172.21.15.37:0/3339394551 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb84c0bd270 con 0x7fb850101300 2024-03-26T06:22:42.986 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-26T06:22:42.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 -- 172.21.15.37:0/3339394551 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb83c0a2800 msgr2=0x7fb83c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb83c0a2800 0x7fb83c0a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb850074e10 tx=0x7fb840005f90 comp rx=0 tx=0).stop 2024-03-26T06:22:42.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 -- 172.21.15.37:0/3339394551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb850101300 msgr2=0x7fb8500750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:42.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb850101300 0x7fb8500750a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb84c000c00 tx=0x7fb84c005d50 comp rx=0 tx=0).stop 2024-03-26T06:22:42.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 -- 172.21.15.37:0/3339394551 shutdown_connections 2024-03-26T06:22:42.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fb83c0a2800 0x7fb83c0a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb850102eb0 0x7fb850073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8501024f0 0x7fb8500736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 --2- 172.21.15.37:0/3339394551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb850101300 0x7fb8500750a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:42.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 -- 172.21.15.37:0/3339394551 >> 172.21.15.37:0/3339394551 conn(0x7fb8500fc890 msgr2=0x7fb850105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:42.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 -- 172.21.15.37:0/3339394551 shutdown_connections 2024-03-26T06:22:42.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:42.987+0000 7fb857f7e640 1 -- 172.21.15.37:0/3339394551 wait complete. 2024-03-26T06:22:42.999 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:22:43.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.100+0000 7f3651748640 1 -- 172.21.15.37:0/1491899607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f364c101dc0 msgr2=0x7f364c108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.100+0000 7f3651748640 1 --2- 172.21.15.37:0/1491899607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f364c101dc0 0x7f364c108650 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f363c009ca0 tx=0x7f363c02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.100+0000 7f3651748640 1 -- 172.21.15.37:0/1491899607 shutdown_connections 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.100+0000 7f3651748640 1 --2- 172.21.15.37:0/1491899607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f364c101dc0 0x7f364c108650 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.100+0000 7f3651748640 1 --2- 172.21.15.37:0/1491899607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f364c101400 0x7f364c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.100+0000 7f3651748640 1 --2- 172.21.15.37:0/1491899607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f364c100210 0x7f364c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.100+0000 7f3651748640 1 -- 172.21.15.37:0/1491899607 >> 172.21.15.37:0/1491899607 conn(0x7f364c0fb780 msgr2=0x7f364c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.100+0000 7f3651748640 1 -- 172.21.15.37:0/1491899607 shutdown_connections 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.100+0000 7f3651748640 1 -- 172.21.15.37:0/1491899607 wait complete. 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f3651748640 1 Processor -- start 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f3651748640 1 -- start start 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f3651748640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f364c100210 0x7f364c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f3651748640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f364c101400 0x7f364c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f3651748640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f364c101dc0 0x7f364c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f3651748640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c10bea0 con 0x7f364c101400 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f3651748640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c10bd20 con 0x7f364c100210 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f3651748640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c10c020 con 0x7f364c101dc0 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f364a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f364c101400 0x7f364c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f364a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f364c101400 0x7f364c197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44714/0 (socket says 172.21.15.37:44714) 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f364a7fc640 1 -- 172.21.15.37:0/1543327368 learned_addr learned my addr 172.21.15.37:0/1543327368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.101+0000 7f364b7fe640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f364c101dc0 0x7f364c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f364affd640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f364c100210 0x7f364c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f364a7fc640 1 -- 172.21.15.37:0/1543327368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f364c101dc0 msgr2=0x7f364c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f364a7fc640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f364c101dc0 0x7f364c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f364a7fc640 1 -- 172.21.15.37:0/1543327368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f364c100210 msgr2=0x7f364c197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f364a7fc640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f364c100210 0x7f364c197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f364a7fc640 1 -- 172.21.15.37:0/1543327368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c19f2a0 con 0x7f364c101400 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f364a7fc640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f364c101400 0x7f364c197bc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f364000c970 tx=0x7f364000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f3633fff640 1 -- 172.21.15.37:0/1543327368 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640007bb0 con 0x7f364c101400 2024-03-26T06:22:43.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f3651748640 1 -- 172.21.15.37:0/1543327368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c19f590 con 0x7f364c101400 2024-03-26T06:22:43.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f3633fff640 1 -- 172.21.15.37:0/1543327368 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3640007d50 con 0x7f364c101400 2024-03-26T06:22:43.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f3633fff640 1 -- 172.21.15.37:0/1543327368 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364000f450 con 0x7f364c101400 2024-03-26T06:22:43.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.102+0000 7f3651748640 1 -- 172.21.15.37:0/1543327368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c19fa90 con 0x7f364c101400 2024-03-26T06:22:43.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.103+0000 7f3651748640 1 -- 172.21.15.37:0/1543327368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c19fd50 con 0x7f364c101400 2024-03-26T06:22:43.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.104+0000 7f3633fff640 1 -- 172.21.15.37:0/1543327368 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3640005020 con 0x7f364c101400 2024-03-26T06:22:43.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.105+0000 7f3633fff640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a2800 0x7f36340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.105+0000 7f364affd640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a2800 0x7f36340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.105+0000 7f3633fff640 1 -- 172.21.15.37:0/1543327368 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f36400ce820 con 0x7f364c101400 2024-03-26T06:22:43.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.105+0000 7f364affd640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a2800 0x7f36340a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f364c101260 tx=0x7f3644005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.109+0000 7f3633fff640 1 -- 172.21.15.37:0/1543327368 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3640014030 con 0x7f364c101400 2024-03-26T06:22:43.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.330+0000 7f3651748640 1 -- 172.21.15.37:0/1543327368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f364c19ff20 con 0x7f364c101400 2024-03-26T06:22:43.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.330+0000 7f3633fff640 1 -- 172.21.15.37:0/1543327368 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11385 (secure 0 0 0) 0x7f364009b8d0 con 0x7f364c101400 2024-03-26T06:22:43.332 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:43.332 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T06:20:53.631953+0000","modified":"2024-03-26T06:21:00.587013+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24494,"mds_2":24470,"mds_3":24469,"mds_4":14682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14670":{"gid":14670,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/1944002831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1944002831},{"type":"v1","addr":"172.21.15.37:6839","nonce":1944002831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14682":{"gid":14682,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6841/933565564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":933565564},{"type":"v1","addr":"172.21.15.37:6841","nonce":933565564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24394":{"gid":24394,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/801999981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":801999981},{"type":"v1","addr":"172.21.15.149:6837","nonce":801999981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/1420321367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1420321367},{"type":"v1","addr":"172.21.15.66:6839","nonce":1420321367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-26T06:22:43.332 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-03-26T06:22:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.332+0000 7f3651748640 1 -- 172.21.15.37:0/1543327368 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a2800 msgr2=0x7f36340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.332+0000 7f3651748640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a2800 0x7f36340a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f364c101260 tx=0x7f3644005f90 comp rx=0 tx=0).stop 2024-03-26T06:22:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.332+0000 7f3651748640 1 -- 172.21.15.37:0/1543327368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f364c101400 msgr2=0x7f364c197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.332+0000 7f3651748640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f364c101400 0x7f364c197bc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f364000c970 tx=0x7f364000ce30 comp rx=0 tx=0).stop 2024-03-26T06:22:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.332+0000 7f3651748640 1 -- 172.21.15.37:0/1543327368 shutdown_connections 2024-03-26T06:22:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.332+0000 7f3651748640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f36340a2800 0x7f36340a4cc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.332+0000 7f3651748640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f364c101dc0 0x7f364c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.332+0000 7f3651748640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f364c101400 0x7f364c197bc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.332+0000 7f3651748640 1 --2- 172.21.15.37:0/1543327368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f364c100210 0x7f364c197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.332+0000 7f3651748640 1 -- 172.21.15.37:0/1543327368 >> 172.21.15.37:0/1543327368 conn(0x7f364c0fb780 msgr2=0x7f364c0fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.333+0000 7f3651748640 1 -- 172.21.15.37:0/1543327368 shutdown_connections 2024-03-26T06:22:43.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.333+0000 7f3651748640 1 -- 172.21.15.37:0/1543327368 wait complete. 2024-03-26T06:22:43.345 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:22:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.447+0000 7fef6dc59640 1 -- 172.21.15.37:0/1409215325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef68101310 msgr2=0x7fef68101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.447+0000 7fef6dc59640 1 --2- 172.21.15.37:0/1409215325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef68101310 0x7fef68101730 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fef58009ca0 tx=0x7fef5802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.447+0000 7fef6dc59640 1 -- 172.21.15.37:0/1409215325 shutdown_connections 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.447+0000 7fef6dc59640 1 --2- 172.21.15.37:0/1409215325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef68102ec0 0x7fef68109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.447+0000 7fef6dc59640 1 --2- 172.21.15.37:0/1409215325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef68102500 0x7fef68102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.447+0000 7fef6dc59640 1 --2- 172.21.15.37:0/1409215325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef68101310 0x7fef68101730 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.447+0000 7fef6dc59640 1 -- 172.21.15.37:0/1409215325 >> 172.21.15.37:0/1409215325 conn(0x7fef680fc8a0 msgr2=0x7fef680fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.447+0000 7fef6dc59640 1 -- 172.21.15.37:0/1409215325 shutdown_connections 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.447+0000 7fef6dc59640 1 -- 172.21.15.37:0/1409215325 wait complete. 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef6dc59640 1 Processor -- start 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef6dc59640 1 -- start start 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef6dc59640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef68101310 0x7fef680750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef6dc59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef68102500 0x7fef680736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef6dc59640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef68102ec0 0x7fef68073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef6dc59640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6810cfe0 con 0x7fef68102500 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef6dc59640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6810ce60 con 0x7fef68101310 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef6dc59640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6810d160 con 0x7fef68102ec0 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef66ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef68102500 0x7fef680736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef66ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef68102500 0x7fef680736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44730/0 (socket says 172.21.15.37:44730) 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef66ffd640 1 -- 172.21.15.37:0/2480711979 learned_addr learned my addr 172.21.15.37:0/2480711979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef677fe640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef68101310 0x7fef680750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef66ffd640 1 -- 172.21.15.37:0/2480711979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef68102ec0 msgr2=0x7fef68073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef66ffd640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef68102ec0 0x7fef68073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef66ffd640 1 -- 172.21.15.37:0/2480711979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef68101310 msgr2=0x7fef680750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef66ffd640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef68101310 0x7fef680750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.448+0000 7fef66ffd640 1 -- 172.21.15.37:0/2480711979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef680744b0 con 0x7fef68102500 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.449+0000 7fef66ffd640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef68102500 0x7fef680736e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fef5000d950 tx=0x7fef5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.449+0000 7fef64ff9640 1 -- 172.21.15.37:0/2480711979 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50017070 con 0x7fef68102500 2024-03-26T06:22:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.449+0000 7fef64ff9640 1 -- 172.21.15.37:0/2480711979 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef50009c30 con 0x7fef68102500 2024-03-26T06:22:43.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.449+0000 7fef6dc59640 1 -- 172.21.15.37:0/2480711979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef681a17a0 con 0x7fef68102500 2024-03-26T06:22:43.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.449+0000 7fef64ff9640 1 -- 172.21.15.37:0/2480711979 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5000f6b0 con 0x7fef68102500 2024-03-26T06:22:43.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.449+0000 7fef6dc59640 1 -- 172.21.15.37:0/2480711979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef681a1cf0 con 0x7fef68102500 2024-03-26T06:22:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.450+0000 7fef6dc59640 1 -- 172.21.15.37:0/2480711979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef68108630 con 0x7fef68102500 2024-03-26T06:22:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.451+0000 7fef64ff9640 1 -- 172.21.15.37:0/2480711979 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fef500109d0 con 0x7fef68102500 2024-03-26T06:22:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.451+0000 7fef64ff9640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fef540a2800 0x7fef540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.452+0000 7fef677fe640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fef540a2800 0x7fef540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.452+0000 7fef64ff9640 1 -- 172.21.15.37:0/2480711979 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fef500ce700 con 0x7fef68102500 2024-03-26T06:22:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.452+0000 7fef677fe640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fef540a2800 0x7fef540a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fef5802c700 tx=0x7fef58032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.456+0000 7fef64ff9640 1 -- 172.21.15.37:0/2480711979 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fef50009de0 con 0x7fef68102500 2024-03-26T06:22:43.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:43 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/77681506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:43.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:43 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2924296816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:43.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:43 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3339394551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.638+0000 7fef6dc59640 1 -- 172.21.15.37:0/2480711979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef68061ef0 con 0x7fef68102500 2024-03-26T06:22:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.640+0000 7fef64ff9640 1 -- 172.21.15.37:0/2480711979 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22580 (secure 0 0 0) 0x7fef5009b7b0 con 0x7fef68102500 2024-03-26T06:22:43.641 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:22:43.641 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":87,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:21:48.711822+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:22:43.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.641+0000 7fef6dc59640 1 -- 172.21.15.37:0/2480711979 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fef540a2800 msgr2=0x7fef540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.642+0000 7fef6dc59640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fef540a2800 0x7fef540a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fef5802c700 tx=0x7fef58032590 comp rx=0 tx=0).stop 2024-03-26T06:22:43.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.642+0000 7fef6dc59640 1 -- 172.21.15.37:0/2480711979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef68102500 msgr2=0x7fef680736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.642+0000 7fef6dc59640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef68102500 0x7fef680736e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fef5000d950 tx=0x7fef5000de10 comp rx=0 tx=0).stop 2024-03-26T06:22:43.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.642+0000 7fef6dc59640 1 -- 172.21.15.37:0/2480711979 shutdown_connections 2024-03-26T06:22:43.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.642+0000 7fef6dc59640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fef540a2800 0x7fef540a4cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.642+0000 7fef6dc59640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef68102ec0 0x7fef68073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.642+0000 7fef6dc59640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef68102500 0x7fef680736e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.642+0000 7fef6dc59640 1 --2- 172.21.15.37:0/2480711979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef68101310 0x7fef680750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.642+0000 7fef6dc59640 1 -- 172.21.15.37:0/2480711979 >> 172.21.15.37:0/2480711979 conn(0x7fef680fc8a0 msgr2=0x7fef68105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.642+0000 7fef6dc59640 1 -- 172.21.15.37:0/2480711979 shutdown_connections 2024-03-26T06:22:43.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.642+0000 7fef6dc59640 1 -- 172.21.15.37:0/2480711979 wait complete. 2024-03-26T06:22:43.653 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:43 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/77681506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:43.653 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:43 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2924296816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:43.653 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:43 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3339394551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:43.655 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T06:20:53.631953+0000', 'modified': '2024-03-26T06:21:00.587013+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24545, 'mds_1': 24494, 'mds_2': 24470, 'mds_3': 24469, 'mds_4': 14682}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14670': {'gid': 14670, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.37:6839/1944002831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6838', 'nonce': 1944002831}, {'type': 'v1', 'addr': '172.21.15.37:6839', 'nonce': 1944002831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14682': {'gid': 14682, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.37:6841/933565564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6840', 'nonce': 933565564}, {'type': 'v1', 'addr': '172.21.15.37:6841', 'nonce': 933565564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24394': {'gid': 24394, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6833/2169208399', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 2169208399}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 2169208399}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6833/2811553067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 2811553067}, {'type': 'v1', 'addr': '172.21.15.66:6833', 'nonce': 2811553067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/1494796094', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 1494796094}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 1494796094}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/801999981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 801999981}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 801999981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.149:6839/1810727373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 1810727373}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 1810727373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24470': {'gid': 24470, 'name': 'e', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.66:6835/4186424754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 4186424754}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 4186424754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.66:6837/3870845253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 3870845253}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 3870845253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6839/1420321367', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 1420321367}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 1420321367}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-26T06:22:43.655 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-26T06:22:43.655 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-26T06:22:43.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:43 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/77681506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:22:43.659 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:43 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2924296816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:22:43.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:43 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3339394551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:22:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.758+0000 7f19f4b1b640 1 -- 172.21.15.37:0/2851453599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f01001f0 msgr2=0x7f19f0100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.758+0000 7f19f4b1b640 1 --2- 172.21.15.37:0/2851453599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f01001f0 0x7f19f0100610 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009900 tx=0x7f19e802bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.758+0000 7f19f4b1b640 1 -- 172.21.15.37:0/2851453599 shutdown_connections 2024-03-26T06:22:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.758+0000 7f19f4b1b640 1 --2- 172.21.15.37:0/2851453599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f0101da0 0x7f19f0108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.758+0000 7f19f4b1b640 1 --2- 172.21.15.37:0/2851453599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f01013e0 0x7f19f0101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.758+0000 7f19f4b1b640 1 --2- 172.21.15.37:0/2851453599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f01001f0 0x7f19f0100610 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.758+0000 7f19f4b1b640 1 -- 172.21.15.37:0/2851453599 >> 172.21.15.37:0/2851453599 conn(0x7f19f00fb780 msgr2=0x7f19f00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.758+0000 7f19f4b1b640 1 -- 172.21.15.37:0/2851453599 shutdown_connections 2024-03-26T06:22:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.758+0000 7f19f4b1b640 1 -- 172.21.15.37:0/2851453599 wait complete. 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.759+0000 7f19f4b1b640 1 Processor -- start 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.759+0000 7f19f4b1b640 1 -- start start 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.759+0000 7f19f4b1b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f01001f0 0x7f19f0069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.759+0000 7f19f4b1b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f01013e0 0x7f19f019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.759+0000 7f19f4b1b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f0101da0 0x7f19f019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.759+0000 7f19f4b1b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f010bec0 con 0x7f19f0101da0 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.759+0000 7f19f4b1b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f010bd40 con 0x7f19f01013e0 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.759+0000 7f19f4b1b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f010c040 con 0x7f19f01001f0 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.759+0000 7f19eed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f0101da0 0x7f19f019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.759+0000 7f19eed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f0101da0 0x7f19f019ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44744/0 (socket says 172.21.15.37:44744) 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.759+0000 7f19eed9d640 1 -- 172.21.15.37:0/278670611 learned_addr learned my addr 172.21.15.37:0/278670611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19ee59c640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f01001f0 0x7f19f0069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19edd9b640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f01013e0 0x7f19f019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19eed9d640 1 -- 172.21.15.37:0/278670611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f01001f0 msgr2=0x7f19f0069020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19eed9d640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f01001f0 0x7f19f0069020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19eed9d640 1 -- 172.21.15.37:0/278670611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f01013e0 msgr2=0x7f19f019c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19eed9d640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f01013e0 0x7f19f019c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19eed9d640 1 -- 172.21.15.37:0/278670611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f019f320 con 0x7f19f0101da0 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19ee59c640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f01001f0 0x7f19f0069020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19edd9b640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f01013e0 0x7f19f019c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19eed9d640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f0101da0 0x7f19f019ea60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f19e000cad0 tx=0x7f19e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19e77fe640 1 -- 172.21.15.37:0/278670611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e000ae20 con 0x7f19f0101da0 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19f4b1b640 1 -- 172.21.15.37:0/278670611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f019f610 con 0x7f19f0101da0 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19e77fe640 1 -- 172.21.15.37:0/278670611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19e0005020 con 0x7f19f0101da0 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19e77fe640 1 -- 172.21.15.37:0/278670611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e000f6f0 con 0x7f19f0101da0 2024-03-26T06:22:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.760+0000 7f19f4b1b640 1 -- 172.21.15.37:0/278670611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f019fb60 con 0x7f19f0101da0 2024-03-26T06:22:43.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19f77d2740 1 Processor -- start 2024-03-26T06:22:43.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19e77fe640 1 -- 172.21.15.37:0/278670611 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f19e00051d0 con 0x7f19f0101da0 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19f77d2740 1 -- start start 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6cfb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6cfbee0 0x55aec6d009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6d01050 0x55aec6d034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19f77d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6c5c7c0 con 0x55aec6cfb4f0 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19f77d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6c05b50 con 0x55aec6cfbee0 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19f77d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6c58860 con 0x55aec6d01050 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19edd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6cfb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19edd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6cfb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44752/0 (socket says 172.21.15.37:44752) 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19edd9b640 1 -- 172.21.15.37:0/2110128793 learned_addr learned my addr 172.21.15.37:0/2110128793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19ee59c640 1 --2- 172.21.15.37:0/2110128793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6cfbee0 0x55aec6d009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.763+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2110128793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6d01050 0x55aec6d034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19e77fe640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f19dc0a2800 0x7f19dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19edd9b640 1 -- 172.21.15.37:0/2110128793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6d01050 msgr2=0x55aec6d034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2110128793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6d01050 0x55aec6d034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19edd9b640 1 -- 172.21.15.37:0/2110128793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6cfbee0 msgr2=0x55aec6d009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2110128793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6cfbee0 0x55aec6d009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19edd9b640 1 -- 172.21.15.37:0/2110128793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aec6c199a0 con 0x55aec6cfb4f0 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19eed9d640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f19dc0a2800 0x7f19dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19e77fe640 1 -- 172.21.15.37:0/278670611 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f19e00ceb70 con 0x7f19f0101da0 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2110128793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6cfb910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f19d800af40 tx=0x7f19d800a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956e7412b755ab51 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19e4ff9640 1 -- 172.21.15.37:0/2110128793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d801b070 con 0x55aec6cfb4f0 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19e4ff9640 1 -- 172.21.15.37:0/2110128793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19d80168d0 con 0x55aec6cfb4f0 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.764+0000 7f19e4ff9640 1 -- 172.21.15.37:0/2110128793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d800f040 con 0x55aec6cfb4f0 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 -- 172.21.15.37:0/2110128793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 msgr2=0x55aec6cfb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2110128793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6cfb910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f19d800af40 tx=0x7f19d800a410 comp rx=0 tx=0).stop 2024-03-26T06:22:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19eed9d640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f19dc0a2800 0x7f19dc0a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f19e0097980 tx=0x7f19e0066230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 -- 172.21.15.37:0/2110128793 shutdown_connections 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2110128793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6d01050 0x55aec6d034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2110128793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6cfbee0 0x55aec6d009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2110128793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6cfb910 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 -- 172.21.15.37:0/2110128793 >> 172.21.15.37:0/2110128793 conn(0x55aec6cbf400 msgr2=0x55aec6cf2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 -- 172.21.15.37:0/2110128793 shutdown_connections 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 -- 172.21.15.37:0/2110128793 wait complete. 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 -- start start 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 Processor -- start 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfb4f0 0x55aec6d059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfbee0 0x55aec6d05ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6d01050 0x55aec6d0cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6d03fa0 con 0x55aec6cfbee0 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6c18f90 con 0x55aec6d01050 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6d04120 con 0x55aec6cfb4f0 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.765+0000 7f19f77d2740 10 client.0 _ll_register_callbacks cb 0x55aec6c54400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19ee59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfbee0 0x55aec6d05ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19ee59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfbee0 0x55aec6d05ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44764/0 (socket says 172.21.15.37:44764) 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19ee59c640 1 -- 172.21.15.37:0/2091970426 learned_addr learned my addr 172.21.15.37:0/2091970426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfb4f0 0x55aec6d059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6d01050 0x55aec6d0cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19ee59c640 1 -- 172.21.15.37:0/2091970426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfb4f0 msgr2=0x55aec6d059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19ee59c640 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfb4f0 0x55aec6d059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19ee59c640 1 -- 172.21.15.37:0/2091970426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6d01050 msgr2=0x55aec6d0cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19ee59c640 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6d01050 0x55aec6d0cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19ee59c640 1 -- 172.21.15.37:0/2091970426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aec69712d0 con 0x55aec6cfbee0 2024-03-26T06:22:43.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6d01050 0x55aec6d0cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19ee59c640 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfbee0 0x55aec6d05ee0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009900 tx=0x7f19e8004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efca2301c05bab08 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19d5ffb640 1 -- 172.21.15.37:0/2091970426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8043030 con 0x55aec6cfbee0 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19f77d2740 1 -- 172.21.15.37:0/2091970426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19d80168d0 con 0x55aec6cfbee0 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19d5ffb640 1 -- 172.21.15.37:0/2091970426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f19e8005000 con 0x55aec6cfbee0 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19d5ffb640 1 -- 172.21.15.37:0/2091970426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19e8005200 con 0x55aec6cfbee0 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19f77d2740 10 client.15297 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19f77d2740 1 -- 172.21.15.37:0/2091970426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55aec6a3b6b0 con 0x55aec6cfbee0 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.766+0000 7f19d5ffb640 1 -- 172.21.15.37:0/2091970426 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19d80168d0 con 0x55aec6cfbee0 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.767+0000 7f19d5ffb640 1 -- 172.21.15.37:0/2091970426 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f19e8003710 con 0x55aec6cfbee0 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.767+0000 7f19f77d2740 10 client.15297 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.767+0000 7f19f77d2740 10 client.15297 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T06:22:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.767+0000 7f19f77d2740 20 client.15297 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.767+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55aec6d12550 0x55aec6d115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.767+0000 7f19f77d2740 4 client.15297 mds_command: new command op to 14682 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.767+0000 7f19f77d2740 1 -- 172.21.15.37:0/2091970426 --> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aec6c69d00 con 0x55aec6d12550 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.767+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55aec6d12550 0x55aec6d115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.767+0000 7f19d5ffb640 10 client.15297 ms_handle_connect on v2:172.21.15.37:6840/933565564 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.767+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55aec6d12550 0x55aec6d115f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98fe4118d4d104b4 server_cookie=433d693a5c9a6455 in_seq=0 out_seq=0 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19d5ffb640 1 -- 172.21.15.37:0/2091970426 <== mds.4 v2:172.21.15.37:6840/933565564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aec6c69d00 con 0x55aec6d12550 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19d5ffb640 10 client.15297 handle_command_reply: tid=0 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 1 client.15297 shutdown 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 1 -- 172.21.15.37:0/2091970426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfbee0 msgr2=0x55aec6d05ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfbee0 0x55aec6d05ee0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009900 tx=0x7f19e8004250 comp rx=0 tx=0).stop 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 1 -- 172.21.15.37:0/2091970426 shutdown_connections 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55aec6d12550 0x55aec6d115f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6d01050 0x55aec6d0cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfbee0 0x55aec6d05ee0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2091970426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfb4f0 0x55aec6d059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19d5ffb640 0 client.15297 ms_handle_reset on v2:172.21.15.37:6840/933565564 2024-03-26T06:22:43.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 1 -- 172.21.15.37:0/2091970426 >> 172.21.15.37:0/2091970426 conn(0x55aec6cbf400 msgr2=0x55aec6cf1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 1 -- 172.21.15.37:0/2091970426 shutdown_connections 2024-03-26T06:22:43.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 1 -- 172.21.15.37:0/2091970426 wait complete. 2024-03-26T06:22:43.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.768+0000 7f19f77d2740 20 client.15297 trim_cache size 0 max 16384 2024-03-26T06:22:43.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19f77d2740 1 Processor -- start 2024-03-26T06:22:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19f77d2740 1 -- start start 2024-03-26T06:22:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6cfb4f0 0x55aec6cf5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfbee0 0x55aec6d0cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6d01050 0x55aec6cf61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19f77d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6d11d70 con 0x55aec6d01050 2024-03-26T06:22:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19f77d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6d123d0 con 0x55aec6cfb4f0 2024-03-26T06:22:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19f77d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6d12a30 con 0x55aec6cfbee0 2024-03-26T06:22:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19edd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6d01050 0x55aec6cf61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19edd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6d01050 0x55aec6cf61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44776/0 (socket says 172.21.15.37:44776) 2024-03-26T06:22:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19edd9b640 1 -- 172.21.15.37:0/2682207665 learned_addr learned my addr 172.21.15.37:0/2682207665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19edd9b640 1 -- 172.21.15.37:0/2682207665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfbee0 msgr2=0x55aec6d0cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2682207665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfbee0 0x55aec6d0cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19edd9b640 1 -- 172.21.15.37:0/2682207665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6cfb4f0 msgr2=0x55aec6cf5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2682207665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6cfb4f0 0x55aec6cf5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19edd9b640 1 -- 172.21.15.37:0/2682207665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aec6d00da0 con 0x55aec6d01050 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19edd9b640 1 --2- 172.21.15.37:0/2682207665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6d01050 0x55aec6cf61d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19d800aee0 tx=0x7f19d800a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103ff2ecebcbd1d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19d7fff640 1 -- 172.21.15.37:0/2682207665 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d801b040 con 0x55aec6d01050 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19d7fff640 1 -- 172.21.15.37:0/2682207665 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19d8016cf0 con 0x55aec6d01050 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.784+0000 7f19d7fff640 1 -- 172.21.15.37:0/2682207665 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8022430 con 0x55aec6d01050 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.785+0000 7f19f77d2740 1 -- 172.21.15.37:0/2682207665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6d01050 msgr2=0x55aec6cf61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.785+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2682207665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6d01050 0x55aec6cf61d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19d800aee0 tx=0x7f19d800a1c0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.785+0000 7f19f77d2740 1 -- 172.21.15.37:0/2682207665 shutdown_connections 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.785+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2682207665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6d01050 0x55aec6cf61d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.785+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2682207665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfbee0 0x55aec6d0cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.785+0000 7f19f77d2740 1 --2- 172.21.15.37:0/2682207665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6cfb4f0 0x55aec6cf5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.785+0000 7f19f77d2740 1 -- 172.21.15.37:0/2682207665 >> 172.21.15.37:0/2682207665 conn(0x55aec6cbf400 msgr2=0x55aec6cfd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.785+0000 7f19f77d2740 1 -- 172.21.15.37:0/2682207665 shutdown_connections 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.785+0000 7f19f77d2740 1 -- 172.21.15.37:0/2682207665 wait complete. 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.785+0000 7f19f77d2740 1 -- start start 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.785+0000 7f19f77d2740 1 Processor -- start 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6d0a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfbee0 0x55aec6d0a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19f77d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6d01050 0x55aec6d0b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19f77d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6d136f0 con 0x55aec6cfb4f0 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19f77d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6d13090 con 0x55aec6d01050 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19f77d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec6d13d50 con 0x55aec6cfbee0 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19f77d2740 10 client.0 _ll_register_callbacks cb 0x7f19d800a270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19edd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6d0a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19edd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6d0a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44786/0 (socket says 172.21.15.37:44786) 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19edd9b640 1 -- 172.21.15.37:0/1849993104 learned_addr learned my addr 172.21.15.37:0/1849993104 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19ee59c640 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfbee0 0x55aec6d0a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19edd9b640 1 -- 172.21.15.37:0/1849993104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfbee0 msgr2=0x55aec6d0a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19edd9b640 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfbee0 0x55aec6d0a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19edd9b640 1 -- 172.21.15.37:0/1849993104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6d01050 msgr2=0x55aec6d0b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19edd9b640 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6d01050 0x55aec6d0b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19edd9b640 1 -- 172.21.15.37:0/1849993104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aec6d0bba0 con 0x55aec6cfb4f0 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19ee59c640 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfbee0 0x55aec6d0a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19edd9b640 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6d0a0b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f19d8004810 tx=0x7f19d800abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8de0dbc113afdc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19d5ffb640 1 -- 172.21.15.37:0/1849993104 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d801b040 con 0x55aec6cfb4f0 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19d5ffb640 1 -- 172.21.15.37:0/1849993104 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f19d8023040 con 0x55aec6cfb4f0 2024-03-26T06:22:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19f77d2740 1 -- 172.21.15.37:0/1849993104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19d8016cf0 con 0x55aec6cfb4f0 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19d5ffb640 1 -- 172.21.15.37:0/1849993104 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19d8023240 con 0x55aec6cfb4f0 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19f77d2740 10 client.15303 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19f77d2740 1 -- 172.21.15.37:0/1849993104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55aec6d0cea0 con 0x55aec6cfb4f0 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.786+0000 7f19d5ffb640 1 -- 172.21.15.37:0/1849993104 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19d8016cf0 con 0x55aec6cfb4f0 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19d5ffb640 1 -- 172.21.15.37:0/1849993104 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f19d8015030 con 0x55aec6cfb4f0 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 10 client.15303 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 20 client.15303 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55aec6d6dac0 0x55aec6d6a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 4 client.15303 mds_command: new command op to 14682 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 -- 172.21.15.37:0/1849993104 --> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aec6c64050 con 0x55aec6d6dac0 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19ee59c640 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55aec6d6dac0 0x55aec6d6a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19ee59c640 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55aec6d6dac0 0x55aec6d6a160 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b11115fc6d33534e server_cookie=3b8124844906647 in_seq=0 out_seq=0 2024-03-26T06:22:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19d5ffb640 10 client.15303 ms_handle_connect on v2:172.21.15.37:6840/933565564 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19d5ffb640 1 -- 172.21.15.37:0/1849993104 <== mds.4 v2:172.21.15.37:6840/933565564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55aec6c64050 con 0x55aec6d6dac0 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19d5ffb640 10 client.15303 handle_command_reply: tid=0 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 client.15303 shutdown 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 -- 172.21.15.37:0/1849993104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 msgr2=0x55aec6d0a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6d0a0b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f19d8004810 tx=0x7f19d800abd0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 -- 172.21.15.37:0/1849993104 shutdown_connections 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.37:6840/933565564,v1:172.21.15.37:6841/933565564] conn(0x55aec6d6dac0 0x55aec6d6a160 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec6d01050 0x55aec6d0b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19d5ffb640 0 client.15303 ms_handle_reset on v2:172.21.15.37:6840/933565564 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aec6cfbee0 0x55aec6d0a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 --2- 172.21.15.37:0/1849993104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aec6cfb4f0 0x55aec6d0a0b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 -- 172.21.15.37:0/1849993104 >> 172.21.15.37:0/1849993104 conn(0x55aec6cbf400 msgr2=0x55aec6cf5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 -- 172.21.15.37:0/1849993104 shutdown_connections 2024-03-26T06:22:43.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.787+0000 7f19f77d2740 1 -- 172.21.15.37:0/1849993104 wait complete. 2024-03-26T06:22:43.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f77d2740 20 client.15303 trim_cache size 0 max 16384 2024-03-26T06:22:43.800 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:22:43.800 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:22:43.800 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 4, 2024-03-26T06:22:43.800 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 14682, 2024-03-26T06:22:43.800 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-26T06:22:43.800 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-26T06:22:43.800 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-26T06:22:43.800 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 104.20505076400001, 2024-03-26T06:22:43.801 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-03-26T06:22:43.801 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 87, 2024-03-26T06:22:43.801 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T06:22:43.801 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 117.505990688 2024-03-26T06:22:43.801 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:22:43.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 -- 172.21.15.37:0/278670611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f19dc0a2800 msgr2=0x7f19dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f19dc0a2800 0x7f19dc0a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f19e0097980 tx=0x7f19e0066230 comp rx=0 tx=0).stop 2024-03-26T06:22:43.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 -- 172.21.15.37:0/278670611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f0101da0 msgr2=0x7f19f019ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f0101da0 0x7f19f019ea60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f19e000cad0 tx=0x7f19e000cf90 comp rx=0 tx=0).stop 2024-03-26T06:22:43.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 -- 172.21.15.37:0/278670611 shutdown_connections 2024-03-26T06:22:43.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f19dc0a2800 0x7f19dc0a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f0101da0 0x7f19f019ea60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19f01013e0 0x7f19f019c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 --2- 172.21.15.37:0/278670611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f01001f0 0x7f19f0069020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 -- 172.21.15.37:0/278670611 >> 172.21.15.37:0/278670611 conn(0x7f19f00fb780 msgr2=0x7f19f0104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 -- 172.21.15.37:0/278670611 shutdown_connections 2024-03-26T06:22:43.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.788+0000 7f19f4b1b640 1 -- 172.21.15.37:0/278670611 wait complete. 2024-03-26T06:22:43.803 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-26T06:22:43.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.905+0000 7f318a220640 1 -- 172.21.15.37:0/639014872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31841001f0 msgr2=0x7f3184100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.905+0000 7f318a220640 1 --2- 172.21.15.37:0/639014872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31841001f0 0x7f3184100610 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f316c009900 tx=0x7f316c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:43.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.905+0000 7f318a220640 1 -- 172.21.15.37:0/639014872 shutdown_connections 2024-03-26T06:22:43.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.905+0000 7f318a220640 1 --2- 172.21.15.37:0/639014872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3184101da0 0x7f3184108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.905+0000 7f318a220640 1 --2- 172.21.15.37:0/639014872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31841013e0 0x7f3184101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.905+0000 7f318a220640 1 --2- 172.21.15.37:0/639014872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31841001f0 0x7f3184100610 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.905+0000 7f318a220640 1 -- 172.21.15.37:0/639014872 >> 172.21.15.37:0/639014872 conn(0x7f31840fb780 msgr2=0x7f31840fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.906+0000 7f318a220640 1 -- 172.21.15.37:0/639014872 shutdown_connections 2024-03-26T06:22:43.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.906+0000 7f318a220640 1 -- 172.21.15.37:0/639014872 wait complete. 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.906+0000 7f318a220640 1 Processor -- start 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.906+0000 7f318a220640 1 -- start start 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.906+0000 7f318a220640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31841001f0 0x7f3184197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.906+0000 7f318a220640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31841013e0 0x7f3184197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.906+0000 7f318a220640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184101da0 0x7f318419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.906+0000 7f318a220640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318410be80 con 0x7f3184101da0 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.906+0000 7f318a220640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318410bd00 con 0x7f31841013e0 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.906+0000 7f318a220640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318410c000 con 0x7f31841001f0 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.906+0000 7f3183fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184101da0 0x7f318419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3183fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184101da0 0x7f318419ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44806/0 (socket says 172.21.15.37:44806) 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3183fff640 1 -- 172.21.15.37:0/2052437455 learned_addr learned my addr 172.21.15.37:0/2052437455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f31837fe640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31841001f0 0x7f3184197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3182ffd640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31841013e0 0x7f3184197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3183fff640 1 -- 172.21.15.37:0/2052437455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31841001f0 msgr2=0x7f3184197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3183fff640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31841001f0 0x7f3184197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3183fff640 1 -- 172.21.15.37:0/2052437455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31841013e0 msgr2=0x7f3184197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3183fff640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31841013e0 0x7f3184197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3183fff640 1 -- 172.21.15.37:0/2052437455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318419f250 con 0x7f3184101da0 2024-03-26T06:22:43.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3183fff640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184101da0 0x7f318419ebe0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f317c00cad0 tx=0x7f317c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3180ff9640 1 -- 172.21.15.37:0/2052437455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c00ae20 con 0x7f3184101da0 2024-03-26T06:22:43.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3180ff9640 1 -- 172.21.15.37:0/2052437455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f317c005020 con 0x7f3184101da0 2024-03-26T06:22:43.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f318a220640 1 -- 172.21.15.37:0/2052437455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318419f540 con 0x7f3184101da0 2024-03-26T06:22:43.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f3180ff9640 1 -- 172.21.15.37:0/2052437455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c00f6f0 con 0x7f3184101da0 2024-03-26T06:22:43.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.907+0000 7f318a220640 1 -- 172.21.15.37:0/2052437455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318419fae0 con 0x7f3184101da0 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.909+0000 7f3180ff9640 1 -- 172.21.15.37:0/2052437455 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f317c0051d0 con 0x7f3184101da0 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.909+0000 7f318ce04740 1 Processor -- start 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.909+0000 7f318ce04740 1 -- start start 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f318ce04740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08af0e0 0x55a7d08af500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f318ce04740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f318ce04740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08b4c40 0x55a7d08b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f3180ff9640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f31700a2880 0x7f31700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f318ce04740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d08147c0 con 0x55a7d08afad0 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f318ce04740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d07bdb50 con 0x55a7d08b4c40 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f318ce04740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d0810860 con 0x55a7d08af0e0 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f31837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f31837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44812/0 (socket says 172.21.15.37:44812) 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f31837fe640 1 -- 172.21.15.37:0/371270707 learned_addr learned my addr 172.21.15.37:0/371270707 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f3183fff640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f31700a2880 0x7f31700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f3180ff9640 1 -- 172.21.15.37:0/2052437455 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f317c0ced50 con 0x7f3184101da0 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f3182ffd640 1 --2- 172.21.15.37:0/371270707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08b4c40 0x55a7d08b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f3182ffd640 1 --2- 172.21.15.37:0/371270707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08af0e0 0x55a7d08af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f31837fe640 1 -- 172.21.15.37:0/371270707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08af0e0 msgr2=0x55a7d08af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f31837fe640 1 --2- 172.21.15.37:0/371270707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08af0e0 0x55a7d08af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f31837fe640 1 -- 172.21.15.37:0/371270707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08b4c40 msgr2=0x55a7d08b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f31837fe640 1 --2- 172.21.15.37:0/371270707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08b4c40 0x55a7d08b70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f31837fe640 1 -- 172.21.15.37:0/371270707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7d07d19a0 con 0x55a7d08afad0 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f3182ffd640 1 --2- 172.21.15.37:0/371270707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08af0e0 0x55a7d08af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f31837fe640 1 --2- 172.21.15.37:0/371270707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08b45d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f316c005ab0 tx=0x7f316c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9deb42e90969c60 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f3183fff640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f31700a2880 0x7f31700a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f317c09b400 tx=0x7f317c0664c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f3179ffb640 1 -- 172.21.15.37:0/371270707 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c042070 con 0x55a7d08afad0 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f3179ffb640 1 -- 172.21.15.37:0/371270707 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f316c0046c0 con 0x55a7d08afad0 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.910+0000 7f3179ffb640 1 -- 172.21.15.37:0/371270707 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c005300 con 0x55a7d08afad0 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.911+0000 7f318ce04740 1 -- 172.21.15.37:0/371270707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 msgr2=0x55a7d08b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.911+0000 7f318ce04740 1 --2- 172.21.15.37:0/371270707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08b45d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f316c005ab0 tx=0x7f316c005e80 comp rx=0 tx=0).stop 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.911+0000 7f318ce04740 1 -- 172.21.15.37:0/371270707 shutdown_connections 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.911+0000 7f318ce04740 1 --2- 172.21.15.37:0/371270707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08b4c40 0x55a7d08b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.911+0000 7f318ce04740 1 --2- 172.21.15.37:0/371270707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08b45d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.911+0000 7f318ce04740 1 --2- 172.21.15.37:0/371270707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08af0e0 0x55a7d08af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.911+0000 7f318ce04740 1 -- 172.21.15.37:0/371270707 >> 172.21.15.37:0/371270707 conn(0x55a7d0877400 msgr2=0x55a7d08aaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.911+0000 7f318ce04740 1 -- 172.21.15.37:0/371270707 shutdown_connections 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.911+0000 7f318ce04740 1 -- 172.21.15.37:0/371270707 wait complete. 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.911+0000 7f318ce04740 1 -- start start 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f318ce04740 1 Processor -- start 2024-03-26T06:22:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f318ce04740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08b9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f318ce04740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08afad0 0x55a7d08b9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f318ce04740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08b4c40 0x55a7d08c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f318ce04740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d08b7b90 con 0x55a7d08af0e0 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f318ce04740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d07d0f90 con 0x55a7d08afad0 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f318ce04740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d08b7d10 con 0x55a7d08b4c40 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f318ce04740 10 client.0 _ll_register_callbacks cb 0x55a7d080c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f3182ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f3182ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08b9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44818/0 (socket says 172.21.15.37:44818) 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f3182ffd640 1 -- 172.21.15.37:0/3448511426 learned_addr learned my addr 172.21.15.37:0/3448511426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f31837fe640 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08afad0 0x55a7d08b9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f3182ffd640 1 -- 172.21.15.37:0/3448511426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08b4c40 msgr2=0x55a7d08c0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f3182ffd640 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08b4c40 0x55a7d08c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f3182ffd640 1 -- 172.21.15.37:0/3448511426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08afad0 msgr2=0x55a7d08b9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f3182ffd640 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08afad0 0x55a7d08b9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f3182ffd640 1 -- 172.21.15.37:0/3448511426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7d05292d0 con 0x55a7d08af0e0 2024-03-26T06:22:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f31837fe640 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08afad0 0x55a7d08b9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f3182ffd640 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08b9590 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f317400df00 tx=0x7f3174012a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b52bf889e13f4f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.912+0000 7f3162ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f3162ffd640 1 -- 172.21.15.37:0/3448511426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317400f9f0 con 0x55a7d08af0e0 2024-03-26T06:22:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f3162ffd640 1 -- 172.21.15.37:0/3448511426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f317400fb90 con 0x55a7d08af0e0 2024-03-26T06:22:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f318ce04740 1 -- 172.21.15.37:0/3448511426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f316c0046c0 con 0x55a7d08af0e0 2024-03-26T06:22:43.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f3162ffd640 1 -- 172.21.15.37:0/3448511426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f317400fd90 con 0x55a7d08af0e0 2024-03-26T06:22:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f318ce04740 10 client.15315 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f318ce04740 1 -- 172.21.15.37:0/3448511426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a7d05f36b0 con 0x55a7d08af0e0 2024-03-26T06:22:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f3162ffd640 1 -- 172.21.15.37:0/3448511426 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f316c0046c0 con 0x55a7d08af0e0 2024-03-26T06:22:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f3162ffd640 1 -- 172.21.15.37:0/3448511426 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3174010590 con 0x55a7d08af0e0 2024-03-26T06:22:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f318ce04740 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f318ce04740 10 client.15315 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T06:22:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f318ce04740 20 client.15315 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f318ce04740 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55a7d08c6100 0x55a7d08c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f318ce04740 4 client.15315 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f318ce04740 1 -- 172.21.15.37:0/3448511426 --> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7d0821d00 con 0x55a7d08c6100 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.913+0000 7f31837fe640 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55a7d08c6100 0x55a7d08c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.914+0000 7f31837fe640 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55a7d08c6100 0x55a7d08c51b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f4cf5bd6dff6309 server_cookie=bf840c77ee8fd453 in_seq=0 out_seq=0 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.914+0000 7f3162ffd640 10 client.15315 ms_handle_connect on v2:172.21.15.149:6838/1810727373 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.914+0000 7f3162ffd640 1 -- 172.21.15.37:0/3448511426 <== mds.3 v2:172.21.15.149:6838/1810727373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a7d0821d00 con 0x55a7d08c6100 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.914+0000 7f3162ffd640 10 client.15315 handle_command_reply: tid=0 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.914+0000 7f318ce04740 1 client.15315 shutdown 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f318ce04740 1 -- 172.21.15.37:0/3448511426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 msgr2=0x55a7d08b9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f318ce04740 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08b9590 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f317400df00 tx=0x7f3174012a40 comp rx=0 tx=0).stop 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f318ce04740 1 -- 172.21.15.37:0/3448511426 shutdown_connections 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f318ce04740 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55a7d08c6100 0x55a7d08c51b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f318ce04740 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08b4c40 0x55a7d08c0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f318ce04740 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08afad0 0x55a7d08b9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f3162ffd640 0 client.15315 ms_handle_reset on v2:172.21.15.149:6838/1810727373 2024-03-26T06:22:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f318ce04740 1 --2- 172.21.15.37:0/3448511426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08b9590 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f318ce04740 1 -- 172.21.15.37:0/3448511426 >> 172.21.15.37:0/3448511426 conn(0x55a7d0877400 msgr2=0x55a7d08a9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f318ce04740 1 -- 172.21.15.37:0/3448511426 shutdown_connections 2024-03-26T06:22:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f318ce04740 1 -- 172.21.15.37:0/3448511426 wait complete. 2024-03-26T06:22:43.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.915+0000 7f318ce04740 20 client.15315 trim_cache size 0 max 16384 2024-03-26T06:22:43.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.930+0000 7f318ce04740 1 Processor -- start 2024-03-26T06:22:43.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.930+0000 7f318ce04740 1 -- start start 2024-03-26T06:22:43.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f318ce04740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08af0e0 0x55a7d08b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f318ce04740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08c0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f318ce04740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08b4c40 0x55a7d08adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f318ce04740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d08c5920 con 0x55a7d08afad0 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f318ce04740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d08c5f80 con 0x55a7d08af0e0 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f318ce04740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d08c65e0 con 0x55a7d08b4c40 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08c0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08c0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44828/0 (socket says 172.21.15.37:44828) 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31837fe640 1 -- 172.21.15.37:0/3965857343 learned_addr learned my addr 172.21.15.37:0/3965857343 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31837fe640 1 -- 172.21.15.37:0/3965857343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08b4c40 msgr2=0x55a7d08adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31837fe640 1 --2- 172.21.15.37:0/3965857343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08b4c40 0x55a7d08adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31837fe640 1 -- 172.21.15.37:0/3965857343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08af0e0 msgr2=0x55a7d08b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31837fe640 1 --2- 172.21.15.37:0/3965857343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08af0e0 0x55a7d08b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31837fe640 1 -- 172.21.15.37:0/3965857343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7d08aaff0 con 0x55a7d08afad0 2024-03-26T06:22:43.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31837fe640 1 --2- 172.21.15.37:0/3965857343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08c0aa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f316c005ab0 tx=0x7f316c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3537bb80e9e005 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31797fa640 1 -- 172.21.15.37:0/3965857343 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c054070 con 0x55a7d08afad0 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31797fa640 1 -- 172.21.15.37:0/3965857343 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f316c0046c0 con 0x55a7d08afad0 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.931+0000 7f31797fa640 1 -- 172.21.15.37:0/3965857343 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c042040 con 0x55a7d08afad0 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 -- 172.21.15.37:0/3965857343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 msgr2=0x55a7d08c0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 --2- 172.21.15.37:0/3965857343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08c0aa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f316c005ab0 tx=0x7f316c005fd0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 -- 172.21.15.37:0/3965857343 shutdown_connections 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 --2- 172.21.15.37:0/3965857343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08b4c40 0x55a7d08adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 --2- 172.21.15.37:0/3965857343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08afad0 0x55a7d08c0aa0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 --2- 172.21.15.37:0/3965857343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08af0e0 0x55a7d08b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 -- 172.21.15.37:0/3965857343 >> 172.21.15.37:0/3965857343 conn(0x55a7d0877400 msgr2=0x55a7d08b91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 -- 172.21.15.37:0/3965857343 shutdown_connections 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 -- 172.21.15.37:0/3965857343 wait complete. 2024-03-26T06:22:43.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 -- start start 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 Processor -- start 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08bd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08afad0 0x55a7d08bddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08b4c40 0x55a7d091bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d08c72a0 con 0x55a7d08af0e0 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d08c6c40 con 0x55a7d08b4c40 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7d08c7900 con 0x55a7d08afad0 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.932+0000 7f318ce04740 10 client.0 _ll_register_callbacks cb 0x7f316c02c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3182ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08bd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3182ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08bd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44844/0 (socket says 172.21.15.37:44844) 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3182ffd640 1 -- 172.21.15.37:0/745967640 learned_addr learned my addr 172.21.15.37:0/745967640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f31837fe640 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08afad0 0x55a7d08bddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3182ffd640 1 -- 172.21.15.37:0/745967640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08afad0 msgr2=0x55a7d08bddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3182ffd640 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08afad0 0x55a7d08bddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3182ffd640 1 -- 172.21.15.37:0/745967640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08b4c40 msgr2=0x55a7d091bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3182ffd640 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08b4c40 0x55a7d091bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3182ffd640 1 -- 172.21.15.37:0/745967640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7d08be420 con 0x55a7d08af0e0 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3182ffd640 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08bd870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3174010a00 tx=0x7f317400f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc96564d91a1fd3f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3162ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3162ffd640 1 -- 172.21.15.37:0/745967640 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3174029030 con 0x55a7d08af0e0 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3162ffd640 1 -- 172.21.15.37:0/745967640 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f31740103d0 con 0x55a7d08af0e0 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f318ce04740 1 -- 172.21.15.37:0/745967640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f316c0046c0 con 0x55a7d08af0e0 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3162ffd640 1 -- 172.21.15.37:0/745967640 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f317400d7e0 con 0x55a7d08af0e0 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f318ce04740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:43.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f318ce04740 1 -- 172.21.15.37:0/745967640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a7d091d390 con 0x55a7d08af0e0 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3162ffd640 1 -- 172.21.15.37:0/745967640 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f316c0046c0 con 0x55a7d08af0e0 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.933+0000 7f3162ffd640 1 -- 172.21.15.37:0/745967640 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f317401a7f0 con 0x55a7d08af0e0 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f318ce04740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f318ce04740 10 client.15321 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f318ce04740 20 client.15321 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f318ce04740 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55a7d08c6100 0x55a7d091d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f318ce04740 4 client.15321 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f318ce04740 1 -- 172.21.15.37:0/745967640 --> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a7d081c050 con 0x55a7d08c6100 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f31837fe640 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55a7d08c6100 0x55a7d091d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f31837fe640 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55a7d08c6100 0x55a7d091d680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=13663ddc6d360cca server_cookie=9fbf84b3ae46e770 in_seq=0 out_seq=0 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f3162ffd640 10 client.15321 ms_handle_connect on v2:172.21.15.149:6838/1810727373 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f3162ffd640 1 -- 172.21.15.37:0/745967640 <== mds.3 v2:172.21.15.149:6838/1810727373 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55a7d081c050 con 0x55a7d08c6100 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f3162ffd640 10 client.15321 handle_command_reply: tid=0 2024-03-26T06:22:43.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.934+0000 7f318ce04740 1 client.15321 shutdown 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318ce04740 1 -- 172.21.15.37:0/745967640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 msgr2=0x55a7d08bd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318ce04740 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08bd870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3174010a00 tx=0x7f317400f430 comp rx=0 tx=0).stop 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318ce04740 1 -- 172.21.15.37:0/745967640 shutdown_connections 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318ce04740 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.149:6838/1810727373,v1:172.21.15.149:6839/1810727373] conn(0x55a7d08c6100 0x55a7d091d680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318ce04740 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a7d08b4c40 0x55a7d091bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318ce04740 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7d08afad0 0x55a7d08bddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f3162ffd640 0 client.15321 ms_handle_reset on v2:172.21.15.149:6838/1810727373 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318ce04740 1 --2- 172.21.15.37:0/745967640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7d08af0e0 0x55a7d08bd870 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318ce04740 1 -- 172.21.15.37:0/745967640 >> 172.21.15.37:0/745967640 conn(0x55a7d0877400 msgr2=0x55a7d08b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318ce04740 1 -- 172.21.15.37:0/745967640 shutdown_connections 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318ce04740 1 -- 172.21.15.37:0/745967640 wait complete. 2024-03-26T06:22:43.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318ce04740 20 client.15321 trim_cache size 0 max 16384 2024-03-26T06:22:43.943 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:22:43.943 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:22:43.943 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 3, 2024-03-26T06:22:43.943 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24469, 2024-03-26T06:22:43.943 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-26T06:22:43.944 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-26T06:22:43.944 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-26T06:22:43.944 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 105.322505101, 2024-03-26T06:22:43.944 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-03-26T06:22:43.944 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 87, 2024-03-26T06:22:43.944 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T06:22:43.944 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 123.133386591 2024-03-26T06:22:43.944 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:22:43.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 -- 172.21.15.37:0/2052437455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f31700a2880 msgr2=0x7f31700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f31700a2880 0x7f31700a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f317c09b400 tx=0x7f317c0664c0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 -- 172.21.15.37:0/2052437455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184101da0 msgr2=0x7f318419ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:43.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184101da0 0x7f318419ebe0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f317c00cad0 tx=0x7f317c00cf90 comp rx=0 tx=0).stop 2024-03-26T06:22:43.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 -- 172.21.15.37:0/2052437455 shutdown_connections 2024-03-26T06:22:43.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f31700a2880 0x7f31700a4d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184101da0 0x7f318419ebe0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31841013e0 0x7f3184197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 --2- 172.21.15.37:0/2052437455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31841001f0 0x7f3184197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:43.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 -- 172.21.15.37:0/2052437455 >> 172.21.15.37:0/2052437455 conn(0x7f31840fb780 msgr2=0x7f31840fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:43.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 -- 172.21.15.37:0/2052437455 shutdown_connections 2024-03-26T06:22:43.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:43.935+0000 7f318a220640 1 -- 172.21.15.37:0/2052437455 wait complete. 2024-03-26T06:22:43.947 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-26T06:22:44.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.049+0000 7f64344ab640 1 -- 172.21.15.37:0/3185896613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f642c101dc0 msgr2=0x7f642c108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.049+0000 7f64344ab640 1 --2- 172.21.15.37:0/3185896613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f642c101dc0 0x7f642c108650 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f641c009ca0 tx=0x7f641c02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.049+0000 7f64344ab640 1 -- 172.21.15.37:0/3185896613 shutdown_connections 2024-03-26T06:22:44.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.049+0000 7f64344ab640 1 --2- 172.21.15.37:0/3185896613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f642c101dc0 0x7f642c108650 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.049+0000 7f64344ab640 1 --2- 172.21.15.37:0/3185896613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c101400 0x7f642c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.049+0000 7f64344ab640 1 --2- 172.21.15.37:0/3185896613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f642c100210 0x7f642c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.049+0000 7f64344ab640 1 -- 172.21.15.37:0/3185896613 >> 172.21.15.37:0/3185896613 conn(0x7f642c0fb7a0 msgr2=0x7f642c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.049+0000 7f64344ab640 1 -- 172.21.15.37:0/3185896613 shutdown_connections 2024-03-26T06:22:44.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.049+0000 7f64344ab640 1 -- 172.21.15.37:0/3185896613 wait complete. 2024-03-26T06:22:44.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f64344ab640 1 Processor -- start 2024-03-26T06:22:44.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f64344ab640 1 -- start start 2024-03-26T06:22:44.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f64344ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f642c100210 0x7f642c1041a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f64344ab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c101400 0x7f642c1027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f64344ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f642c101dc0 0x7f642c102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f64344ab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c10bea0 con 0x7f642c100210 2024-03-26T06:22:44.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f64344ab640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c10bd20 con 0x7f642c101dc0 2024-03-26T06:22:44.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f64344ab640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c10c020 con 0x7f642c101400 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f6432247640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f642c100210 0x7f642c1041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f6432247640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f642c100210 0x7f642c1041a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44854/0 (socket says 172.21.15.37:44854) 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f6432247640 1 -- 172.21.15.37:0/1849250640 learned_addr learned my addr 172.21.15.37:0/1849250640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f6431a46640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c101400 0x7f642c1027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f6432247640 1 -- 172.21.15.37:0/1849250640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c101400 msgr2=0x7f642c1027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f6432247640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c101400 0x7f642c1027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.050+0000 7f6432247640 1 -- 172.21.15.37:0/1849250640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f642c101dc0 msgr2=0x7f642c102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.051+0000 7f6432247640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f642c101dc0 0x7f642c102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.051+0000 7f6432247640 1 -- 172.21.15.37:0/1849250640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c1035b0 con 0x7f642c100210 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.051+0000 7f6431a46640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c101400 0x7f642c1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.051+0000 7f6432247640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f642c100210 0x7f642c1041a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f642400e9c0 tx=0x7f642400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.051+0000 7f642b7fe640 1 -- 172.21.15.37:0/1849250640 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642400cd80 con 0x7f642c100210 2024-03-26T06:22:44.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.051+0000 7f64344ab640 1 -- 172.21.15.37:0/1849250640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c1a1750 con 0x7f642c100210 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.051+0000 7f642b7fe640 1 -- 172.21.15.37:0/1849250640 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64240035f0 con 0x7f642c100210 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.051+0000 7f642b7fe640 1 -- 172.21.15.37:0/1849250640 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424010690 con 0x7f642c100210 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.051+0000 7f64344ab640 1 -- 172.21.15.37:0/1849250640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c1a1cf0 con 0x7f642c100210 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.053+0000 7f642b7fe640 1 -- 172.21.15.37:0/1849250640 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f64240037a0 con 0x7f642c100210 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f643716f740 1 Processor -- start 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f642b7fe640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64140a2800 0x7f64140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f6432a48640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64140a2800 0x7f64140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f642b7fe640 1 -- 172.21.15.37:0/1849250640 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f64240cf030 con 0x7f642c100210 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f643716f740 1 -- start start 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f643716f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd50e0 0x5600aebd5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f643716f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebda5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f643716f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebdac40 0x5600aebdd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f643716f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aeb3a7c0 con 0x5600aebd5ad0 2024-03-26T06:22:44.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f643716f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aeae3b50 con 0x5600aebdac40 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f643716f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aeb36860 con 0x5600aebd50e0 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f6431a46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebda5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f6431a46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebda5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44856/0 (socket says 172.21.15.37:44856) 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f6431a46640 1 -- 172.21.15.37:0/3504866227 learned_addr learned my addr 172.21.15.37:0/3504866227 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f6432a48640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64140a2800 0x7f64140a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f641c009430 tx=0x7f641c032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f6432247640 1 --2- 172.21.15.37:0/3504866227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd50e0 0x5600aebd5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.054+0000 7f6432a48640 1 --2- 172.21.15.37:0/3504866227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebdac40 0x5600aebdd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.055+0000 7f6431a46640 1 -- 172.21.15.37:0/3504866227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd50e0 msgr2=0x5600aebd5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.055+0000 7f6431a46640 1 --2- 172.21.15.37:0/3504866227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd50e0 0x5600aebd5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.055+0000 7f6431a46640 1 -- 172.21.15.37:0/3504866227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebdac40 msgr2=0x5600aebdd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.055+0000 7f6431a46640 1 --2- 172.21.15.37:0/3504866227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebdac40 0x5600aebdd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.055+0000 7f6431a46640 1 -- 172.21.15.37:0/3504866227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600aeaf79a0 con 0x5600aebd5ad0 2024-03-26T06:22:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.055+0000 7f6431a46640 1 --2- 172.21.15.37:0/3504866227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebda5d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f642000bb40 tx=0x7f642000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc238c5d531dddd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.055+0000 7f6428bfe640 1 -- 172.21.15.37:0/3504866227 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420017420 con 0x5600aebd5ad0 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.055+0000 7f6428bfe640 1 -- 172.21.15.37:0/3504866227 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6420017aa0 con 0x5600aebd5ad0 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.055+0000 7f6428bfe640 1 -- 172.21.15.37:0/3504866227 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642000fb90 con 0x5600aebd5ad0 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 -- 172.21.15.37:0/3504866227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 msgr2=0x5600aebda5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 --2- 172.21.15.37:0/3504866227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebda5d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f642000bb40 tx=0x7f642000d340 comp rx=0 tx=0).stop 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 -- 172.21.15.37:0/3504866227 shutdown_connections 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 --2- 172.21.15.37:0/3504866227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebdac40 0x5600aebdd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 --2- 172.21.15.37:0/3504866227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebda5d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 --2- 172.21.15.37:0/3504866227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd50e0 0x5600aebd5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 -- 172.21.15.37:0/3504866227 >> 172.21.15.37:0/3504866227 conn(0x5600aeb9d400 msgr2=0x5600aebd0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 -- 172.21.15.37:0/3504866227 shutdown_connections 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 -- 172.21.15.37:0/3504866227 wait complete. 2024-03-26T06:22:44.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 -- start start 2024-03-26T06:22:44.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 Processor -- start 2024-03-26T06:22:44.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd50e0 0x5600aebdf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebd5ad0 0x5600aebdfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebdac40 0x5600aebe6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aebddbf0 con 0x5600aebd50e0 2024-03-26T06:22:44.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aeaf6f90 con 0x5600aebd5ad0 2024-03-26T06:22:44.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aebddd70 con 0x5600aebdac40 2024-03-26T06:22:44.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f643716f740 10 client.0 _ll_register_callbacks cb 0x5600aeb32400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:44.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f6432247640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd50e0 0x5600aebdf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f6432247640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd50e0 0x5600aebdf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44864/0 (socket says 172.21.15.37:44864) 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f6432247640 1 -- 172.21.15.37:0/572291992 learned_addr learned my addr 172.21.15.37:0/572291992 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.056+0000 7f6431a46640 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebd5ad0 0x5600aebdfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6432247640 1 -- 172.21.15.37:0/572291992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebdac40 msgr2=0x5600aebe6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6432247640 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebdac40 0x5600aebe6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6432247640 1 -- 172.21.15.37:0/572291992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebd5ad0 msgr2=0x5600aebdfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6432247640 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebd5ad0 0x5600aebdfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6432247640 1 -- 172.21.15.37:0/572291992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600ae84f2d0 con 0x5600aebd50e0 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6432247640 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd50e0 0x5600aebdf560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f64240ccae0 tx=0x7f642409ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458979f67327b2ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6411ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6411ffb640 1 -- 172.21.15.37:0/572291992 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64240a4120 con 0x5600aebd50e0 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6411ffb640 1 -- 172.21.15.37:0/572291992 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6424003b30 con 0x5600aebd50e0 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f643716f740 1 -- 172.21.15.37:0/572291992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6420017aa0 con 0x5600aebd50e0 2024-03-26T06:22:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6411ffb640 1 -- 172.21.15.37:0/572291992 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64240ba190 con 0x5600aebd50e0 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f643716f740 10 client.15333 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f643716f740 1 -- 172.21.15.37:0/572291992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5600ae9196b0 con 0x5600aebd50e0 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6411ffb640 1 -- 172.21.15.37:0/572291992 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6424065030 con 0x5600aebd50e0 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f6411ffb640 1 -- 172.21.15.37:0/572291992 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f642409a840 con 0x5600aebd50e0 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f643716f740 10 client.15333 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f643716f740 10 client.15333 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f643716f740 20 client.15333 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f643716f740 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x5600aebec100 0x5600aebeb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f643716f740 4 client.15333 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.057+0000 7f643716f740 1 -- 172.21.15.37:0/572291992 --> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600aeb47d00 con 0x5600aebec100 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.058+0000 7f6431a46640 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x5600aebec100 0x5600aebeb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.058+0000 7f6431a46640 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x5600aebec100 0x5600aebeb1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed343ad9cb3a5677 server_cookie=f10f592f8916dfa2 in_seq=0 out_seq=0 2024-03-26T06:22:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.058+0000 7f6411ffb640 10 client.15333 ms_handle_connect on v2:172.21.15.66:6834/4186424754 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f6411ffb640 1 -- 172.21.15.37:0/572291992 <== mds.2 v2:172.21.15.66:6834/4186424754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600aeb47d00 con 0x5600aebec100 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f6411ffb640 10 client.15333 handle_command_reply: tid=0 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 1 client.15333 shutdown 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 1 -- 172.21.15.37:0/572291992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd50e0 msgr2=0x5600aebdf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd50e0 0x5600aebdf560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f64240ccae0 tx=0x7f642409ba40 comp rx=0 tx=0).stop 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 1 -- 172.21.15.37:0/572291992 shutdown_connections 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x5600aebec100 0x5600aebeb1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebdac40 0x5600aebe6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebd5ad0 0x5600aebdfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 1 --2- 172.21.15.37:0/572291992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd50e0 0x5600aebdf560 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f6411ffb640 0 client.15333 ms_handle_reset on v2:172.21.15.66:6834/4186424754 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 1 -- 172.21.15.37:0/572291992 >> 172.21.15.37:0/572291992 conn(0x5600aeb9d400 msgr2=0x5600aebcfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 1 -- 172.21.15.37:0/572291992 shutdown_connections 2024-03-26T06:22:44.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 1 -- 172.21.15.37:0/572291992 wait complete. 2024-03-26T06:22:44.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.059+0000 7f643716f740 20 client.15333 trim_cache size 0 max 16384 2024-03-26T06:22:44.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f643716f740 1 Processor -- start 2024-03-26T06:22:44.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f643716f740 1 -- start start 2024-03-26T06:22:44.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f643716f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebd50e0 0x5600aebda560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f643716f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd5ad0 0x5600aebe6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f643716f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebdac40 0x5600aebd6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f643716f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aebeb920 con 0x5600aebdac40 2024-03-26T06:22:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f643716f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aebebf80 con 0x5600aebd50e0 2024-03-26T06:22:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f643716f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aebec5e0 con 0x5600aebd5ad0 2024-03-26T06:22:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f6432a48640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebdac40 0x5600aebd6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f6432a48640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebdac40 0x5600aebd6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44878/0 (socket says 172.21.15.37:44878) 2024-03-26T06:22:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f6432a48640 1 -- 172.21.15.37:0/3587838609 learned_addr learned my addr 172.21.15.37:0/3587838609 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f6432247640 1 --2- 172.21.15.37:0/3587838609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebd50e0 0x5600aebda560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f6432a48640 1 -- 172.21.15.37:0/3587838609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd5ad0 msgr2=0x5600aebe6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f6432a48640 1 --2- 172.21.15.37:0/3587838609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd5ad0 0x5600aebe6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f6432a48640 1 -- 172.21.15.37:0/3587838609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebd50e0 msgr2=0x5600aebda560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f6432a48640 1 --2- 172.21.15.37:0/3587838609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebd50e0 0x5600aebda560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f6432a48640 1 -- 172.21.15.37:0/3587838609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600aebf4e30 con 0x5600aebdac40 2024-03-26T06:22:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f6432247640 1 --2- 172.21.15.37:0/3587838609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebd50e0 0x5600aebda560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.075+0000 7f6432a48640 1 --2- 172.21.15.37:0/3587838609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebdac40 0x5600aebd6f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f641c036000 tx=0x7f641c03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c3bf97cda28b00 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f6413fff640 1 -- 172.21.15.37:0/3587838609 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c039040 con 0x5600aebdac40 2024-03-26T06:22:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f6413fff640 1 -- 172.21.15.37:0/3587838609 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f641c050070 con 0x5600aebdac40 2024-03-26T06:22:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f6413fff640 1 -- 172.21.15.37:0/3587838609 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0046c0 con 0x5600aebdac40 2024-03-26T06:22:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f643716f740 1 -- 172.21.15.37:0/3587838609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebdac40 msgr2=0x5600aebd6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f643716f740 1 --2- 172.21.15.37:0/3587838609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebdac40 0x5600aebd6f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f641c036000 tx=0x7f641c03b000 comp rx=0 tx=0).stop 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f643716f740 1 -- 172.21.15.37:0/3587838609 shutdown_connections 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f643716f740 1 --2- 172.21.15.37:0/3587838609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebdac40 0x5600aebd6f30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f643716f740 1 --2- 172.21.15.37:0/3587838609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd5ad0 0x5600aebe6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f643716f740 1 --2- 172.21.15.37:0/3587838609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebd50e0 0x5600aebda560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f643716f740 1 -- 172.21.15.37:0/3587838609 >> 172.21.15.37:0/3587838609 conn(0x5600aeb9d400 msgr2=0x5600aebeb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f643716f740 1 -- 172.21.15.37:0/3587838609 shutdown_connections 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f643716f740 1 -- 172.21.15.37:0/3587838609 wait complete. 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f643716f740 1 -- start start 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.076+0000 7f643716f740 1 Processor -- start 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f643716f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd50e0 0x5600aebe37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f643716f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebe3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f643716f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebdac40 0x5600aec41d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f643716f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aebed2a0 con 0x5600aebd5ad0 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f643716f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aebecc40 con 0x5600aebdac40 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f643716f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600aebed900 con 0x5600aebd50e0 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f643716f740 10 client.0 _ll_register_callbacks cb 0x7f642000b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6431a46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebe3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6431a46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebe3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44882/0 (socket says 172.21.15.37:44882) 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6431a46640 1 -- 172.21.15.37:0/1344355255 learned_addr learned my addr 172.21.15.37:0/1344355255 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6431a46640 1 -- 172.21.15.37:0/1344355255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd50e0 msgr2=0x5600aebe37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6431a46640 1 --2- 172.21.15.37:0/1344355255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd50e0 0x5600aebe37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6431a46640 1 -- 172.21.15.37:0/1344355255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebdac40 msgr2=0x5600aec41d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6431a46640 1 --2- 172.21.15.37:0/1344355255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebdac40 0x5600aec41d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6431a46640 1 -- 172.21.15.37:0/1344355255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600aebe6e00 con 0x5600aebd5ad0 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6431a46640 1 --2- 172.21.15.37:0/1344355255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebe3d30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f642000b310 tx=0x7f6420005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21d7258ce0a00be server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6411ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6411ffb640 1 -- 172.21.15.37:0/1344355255 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420017e60 con 0x5600aebd5ad0 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6411ffb640 1 -- 172.21.15.37:0/1344355255 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6420026b00 con 0x5600aebd5ad0 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f643716f740 1 -- 172.21.15.37:0/1344355255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f641c050070 con 0x5600aebd5ad0 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f6411ffb640 1 -- 172.21.15.37:0/1344355255 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6420038050 con 0x5600aebd5ad0 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f643716f740 10 client.15339 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.077+0000 7f643716f740 1 -- 172.21.15.37:0/1344355255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5600aec43280 con 0x5600aebd5ad0 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.078+0000 7f6411ffb640 1 -- 172.21.15.37:0/1344355255 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f641c050070 con 0x5600aebd5ad0 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.078+0000 7f6411ffb640 1 -- 172.21.15.37:0/1344355255 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f64200262b0 con 0x5600aebd5ad0 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.078+0000 7f643716f740 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.078+0000 7f643716f740 10 client.15339 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.078+0000 7f643716f740 20 client.15339 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.078+0000 7f643716f740 1 --2- 172.21.15.37:0/1344355255 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x5600aec47740 0x5600aec43570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.078+0000 7f643716f740 4 client.15339 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.078+0000 7f643716f740 1 -- 172.21.15.37:0/1344355255 --> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5600aeb42050 con 0x5600aec47740 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.078+0000 7f6432247640 1 --2- 172.21.15.37:0/1344355255 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x5600aec47740 0x5600aec43570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.078+0000 7f6432247640 1 --2- 172.21.15.37:0/1344355255 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x5600aec47740 0x5600aec43570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4a9bd8cf512a7da server_cookie=c0faa6468b8d7a51 in_seq=0 out_seq=0 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.078+0000 7f6411ffb640 10 client.15339 ms_handle_connect on v2:172.21.15.66:6834/4186424754 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f6411ffb640 1 -- 172.21.15.37:0/1344355255 <== mds.2 v2:172.21.15.66:6834/4186424754 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5600aeb42050 con 0x5600aec47740 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f6411ffb640 10 client.15339 handle_command_reply: tid=0 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 1 client.15339 shutdown 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 1 -- 172.21.15.37:0/1344355255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 msgr2=0x5600aebe3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 1 --2- 172.21.15.37:0/1344355255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebe3d30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f642000b310 tx=0x7f6420005ea0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 1 -- 172.21.15.37:0/1344355255 shutdown_connections 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 1 --2- 172.21.15.37:0/1344355255 >> [v2:172.21.15.66:6834/4186424754,v1:172.21.15.66:6835/4186424754] conn(0x5600aec47740 0x5600aec43570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 1 --2- 172.21.15.37:0/1344355255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600aebdac40 0x5600aec41d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f6411ffb640 0 client.15339 ms_handle_reset on v2:172.21.15.66:6834/4186424754 2024-03-26T06:22:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 1 --2- 172.21.15.37:0/1344355255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600aebd5ad0 0x5600aebe3d30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 1 --2- 172.21.15.37:0/1344355255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600aebd50e0 0x5600aebe37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 1 -- 172.21.15.37:0/1344355255 >> 172.21.15.37:0/1344355255 conn(0x5600aeb9d400 msgr2=0x5600aebda130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 1 -- 172.21.15.37:0/1344355255 shutdown_connections 2024-03-26T06:22:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 1 -- 172.21.15.37:0/1344355255 wait complete. 2024-03-26T06:22:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f643716f740 20 client.15339 trim_cache size 0 max 16384 2024-03-26T06:22:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:22:44.093 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:22:44.093 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-03-26T06:22:44.093 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24470, 2024-03-26T06:22:44.093 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-26T06:22:44.093 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-26T06:22:44.094 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-26T06:22:44.094 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 106.473241566, 2024-03-26T06:22:44.094 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-03-26T06:22:44.094 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 87, 2024-03-26T06:22:44.094 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T06:22:44.094 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 133.930472675 2024-03-26T06:22:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:22:44.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 -- 172.21.15.37:0/1849250640 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64140a2800 msgr2=0x7f64140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64140a2800 0x7f64140a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f641c009430 tx=0x7f641c032590 comp rx=0 tx=0).stop 2024-03-26T06:22:44.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 -- 172.21.15.37:0/1849250640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f642c100210 msgr2=0x7f642c1041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f642c100210 0x7f642c1041a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f642400e9c0 tx=0x7f642400ee80 comp rx=0 tx=0).stop 2024-03-26T06:22:44.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 -- 172.21.15.37:0/1849250640 shutdown_connections 2024-03-26T06:22:44.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f64140a2800 0x7f64140a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f642c101dc0 0x7f642c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f642c101400 0x7f642c1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 --2- 172.21.15.37:0/1849250640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f642c100210 0x7f642c1041a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 -- 172.21.15.37:0/1849250640 >> 172.21.15.37:0/1849250640 conn(0x7f642c0fb7a0 msgr2=0x7f642c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 -- 172.21.15.37:0/1849250640 shutdown_connections 2024-03-26T06:22:44.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.079+0000 7f64344ab640 1 -- 172.21.15.37:0/1849250640 wait complete. 2024-03-26T06:22:44.097 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-26T06:22:44.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 -- 172.21.15.37:0/2533497986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18c101da0 msgr2=0x7fe18c108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 --2- 172.21.15.37:0/2533497986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18c101da0 0x7fe18c108630 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe184009ca0 tx=0x7fe18402c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 -- 172.21.15.37:0/2533497986 shutdown_connections 2024-03-26T06:22:44.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 --2- 172.21.15.37:0/2533497986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18c101da0 0x7fe18c108630 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 --2- 172.21.15.37:0/2533497986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c1013e0 0x7fe18c101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 --2- 172.21.15.37:0/2533497986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe18c1001f0 0x7fe18c100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 -- 172.21.15.37:0/2533497986 >> 172.21.15.37:0/2533497986 conn(0x7fe18c0fb780 msgr2=0x7fe18c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 -- 172.21.15.37:0/2533497986 shutdown_connections 2024-03-26T06:22:44.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 -- 172.21.15.37:0/2533497986 wait complete. 2024-03-26T06:22:44.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 Processor -- start 2024-03-26T06:22:44.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 -- start start 2024-03-26T06:22:44.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c1001f0 0x7fe18c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe18c1013e0 0x7fe18c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18c101da0 0x7fe18c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c10be80 con 0x7fe18c101da0 2024-03-26T06:22:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c10bd00 con 0x7fe18c1001f0 2024-03-26T06:22:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.200+0000 7fe193897640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c10c000 con 0x7fe18c1013e0 2024-03-26T06:22:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe191e34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18c101da0 0x7fe18c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe191e34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18c101da0 0x7fe18c19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44894/0 (socket says 172.21.15.37:44894) 2024-03-26T06:22:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe191e34640 1 -- 172.21.15.37:0/4229633314 learned_addr learned my addr 172.21.15.37:0/4229633314 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe191633640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c1001f0 0x7fe18c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe190e32640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe18c1013e0 0x7fe18c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe191e34640 1 -- 172.21.15.37:0/4229633314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe18c1013e0 msgr2=0x7fe18c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe191e34640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe18c1013e0 0x7fe18c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe191e34640 1 -- 172.21.15.37:0/4229633314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c1001f0 msgr2=0x7fe18c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe191e34640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c1001f0 0x7fe18c1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe191e34640 1 -- 172.21.15.37:0/4229633314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c19f1e0 con 0x7fe18c101da0 2024-03-26T06:22:44.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe191e34640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18c101da0 0x7fe18c19eb70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe18402c700 tx=0x7fe184032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe1827fc640 1 -- 172.21.15.37:0/4229633314 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184039040 con 0x7fe18c101da0 2024-03-26T06:22:44.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe1827fc640 1 -- 172.21.15.37:0/4229633314 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe184044070 con 0x7fe18c101da0 2024-03-26T06:22:44.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe193897640 1 -- 172.21.15.37:0/4229633314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c19f470 con 0x7fe18c101da0 2024-03-26T06:22:44.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe1827fc640 1 -- 172.21.15.37:0/4229633314 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840042a0 con 0x7fe18c101da0 2024-03-26T06:22:44.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.201+0000 7fe193897640 1 -- 172.21.15.37:0/4229633314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c19f960 con 0x7fe18c101da0 2024-03-26T06:22:44.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.203+0000 7fe1827fc640 1 -- 172.21.15.37:0/4229633314 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe184003970 con 0x7fe18c101da0 2024-03-26T06:22:44.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe19647a740 1 Processor -- start 2024-03-26T06:22:44.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe19647a740 1 -- start start 2024-03-26T06:22:44.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe19647a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af54fd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe19647a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af55025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe19647a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 0x559af55050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe19647a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af54627c0 con 0x559af54fd0e0 2024-03-26T06:22:44.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe19647a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af540bb50 con 0x559af54fdad0 2024-03-26T06:22:44.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe19647a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af545e860 con 0x559af5502c40 2024-03-26T06:22:44.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe1827fc640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe1780a2800 0x7fe1780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe190e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af54fd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe190e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af54fd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44908/0 (socket says 172.21.15.37:44908) 2024-03-26T06:22:44.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe190e32640 1 -- 172.21.15.37:0/2885893521 learned_addr learned my addr 172.21.15.37:0/2885893521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe191e34640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe1780a2800 0x7fe1780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe191633640 1 --2- 172.21.15.37:0/2885893521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af55025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe1827fc640 1 -- 172.21.15.37:0/4229633314 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe1840f0fc0 con 0x7fe18c101da0 2024-03-26T06:22:44.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe190e32640 1 --2- 172.21.15.37:0/2885893521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 0x559af55050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe190e32640 1 -- 172.21.15.37:0/2885893521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 msgr2=0x559af55050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe190e32640 1 --2- 172.21.15.37:0/2885893521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 0x559af55050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe190e32640 1 -- 172.21.15.37:0/2885893521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 msgr2=0x559af55025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe190e32640 1 --2- 172.21.15.37:0/2885893521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af55025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.204+0000 7fe190e32640 1 -- 172.21.15.37:0/2885893521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559af541f9a0 con 0x559af54fd0e0 2024-03-26T06:22:44.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe190e32640 1 --2- 172.21.15.37:0/2885893521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af54fd500 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe17c013550 tx=0x7fe17c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5973348dcf333d66 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe191e34640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe1780a2800 0x7fe1780a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe18404d000 tx=0x7fe1840e8360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe1777fe640 1 -- 172.21.15.37:0/2885893521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c019030 con 0x559af54fd0e0 2024-03-26T06:22:44.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe1777fe640 1 -- 172.21.15.37:0/2885893521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe17c0119a0 con 0x559af54fd0e0 2024-03-26T06:22:44.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe1777fe640 1 -- 172.21.15.37:0/2885893521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c00f900 con 0x559af54fd0e0 2024-03-26T06:22:44.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe19647a740 1 -- 172.21.15.37:0/2885893521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 msgr2=0x559af54fd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe19647a740 1 --2- 172.21.15.37:0/2885893521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af54fd500 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe17c013550 tx=0x7fe17c00b670 comp rx=0 tx=0).stop 2024-03-26T06:22:44.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe19647a740 1 -- 172.21.15.37:0/2885893521 shutdown_connections 2024-03-26T06:22:44.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe19647a740 1 --2- 172.21.15.37:0/2885893521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 0x559af55050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe19647a740 1 --2- 172.21.15.37:0/2885893521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af55025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe19647a740 1 --2- 172.21.15.37:0/2885893521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af54fd500 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.205+0000 7fe19647a740 1 -- 172.21.15.37:0/2885893521 >> 172.21.15.37:0/2885893521 conn(0x559af54c5400 msgr2=0x559af54f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe19647a740 1 -- 172.21.15.37:0/2885893521 shutdown_connections 2024-03-26T06:22:44.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe19647a740 1 -- 172.21.15.37:0/2885893521 wait complete. 2024-03-26T06:22:44.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe19647a740 1 -- start start 2024-03-26T06:22:44.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe19647a740 1 Processor -- start 2024-03-26T06:22:44.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe19647a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af55076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe19647a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af54fdad0 0x559af5507bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe19647a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af5502c40 0x559af550ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe19647a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af5505bf0 con 0x559af54fd0e0 2024-03-26T06:22:44.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe19647a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af541ef90 con 0x559af5502c40 2024-03-26T06:22:44.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe19647a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af5505d70 con 0x559af54fdad0 2024-03-26T06:22:44.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe19647a740 10 client.0 _ll_register_callbacks cb 0x559af545a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:44.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe190e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af55076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe190e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af55076b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44918/0 (socket says 172.21.15.37:44918) 2024-03-26T06:22:44.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe190e32640 1 -- 172.21.15.37:0/953708979 learned_addr learned my addr 172.21.15.37:0/953708979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.206+0000 7fe191633640 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af54fdad0 0x559af5507bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe190e32640 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af5502c40 0x559af550ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe190e32640 1 -- 172.21.15.37:0/953708979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af54fdad0 msgr2=0x559af5507bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe190e32640 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af54fdad0 0x559af5507bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe190e32640 1 -- 172.21.15.37:0/953708979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af5502c40 msgr2=0x559af550ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe190e32640 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af5502c40 0x559af550ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe190e32640 1 -- 172.21.15.37:0/953708979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559af51772d0 con 0x559af54fd0e0 2024-03-26T06:22:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe190e32640 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af55076b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00b4a0 tx=0x7fe17c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9ae4967dc4b6d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe174ff9640 1 -- 172.21.15.37:0/953708979 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c019040 con 0x559af54fd0e0 2024-03-26T06:22:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe174ff9640 1 -- 172.21.15.37:0/953708979 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe17c011dd0 con 0x559af54fd0e0 2024-03-26T06:22:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe19647a740 1 -- 172.21.15.37:0/953708979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe17c0119a0 con 0x559af54fd0e0 2024-03-26T06:22:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe174ff9640 1 -- 172.21.15.37:0/953708979 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe17c004030 con 0x559af54fd0e0 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe19647a740 10 client.15351 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe19647a740 1 -- 172.21.15.37:0/953708979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559af52416b0 con 0x559af54fd0e0 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe174ff9640 1 -- 172.21.15.37:0/953708979 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe17c0119a0 con 0x559af54fd0e0 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe174ff9640 1 -- 172.21.15.37:0/953708979 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe17c004880 con 0x559af54fd0e0 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe19647a740 10 client.15351 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe19647a740 10 client.15351 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe19647a740 20 client.15351 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe19647a740 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x559af5513830 0x559af5513160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe19647a740 4 client.15351 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.207+0000 7fe19647a740 1 -- 172.21.15.37:0/953708979 --> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559af546fd00 con 0x559af5513830 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.208+0000 7fe191633640 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x559af5513830 0x559af5513160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.208+0000 7fe174ff9640 10 client.15351 ms_handle_connect on v2:172.21.15.66:6836/3870845253 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.208+0000 7fe191633640 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x559af5513830 0x559af5513160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e6d4fdabb76872b server_cookie=8c09f91c955df2ed in_seq=0 out_seq=0 2024-03-26T06:22:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe174ff9640 1 -- 172.21.15.37:0/953708979 <== mds.1 v2:172.21.15.66:6836/3870845253 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559af546fd00 con 0x559af5513830 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe174ff9640 10 client.15351 handle_command_reply: tid=0 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 1 client.15351 shutdown 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 1 -- 172.21.15.37:0/953708979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 msgr2=0x559af55076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af55076b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00b4a0 tx=0x7fe17c005fd0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 1 -- 172.21.15.37:0/953708979 shutdown_connections 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af5502c40 0x559af550ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x559af5513830 0x559af5513160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af54fdad0 0x559af5507bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 1 --2- 172.21.15.37:0/953708979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af55076b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe174ff9640 0 client.15351 ms_handle_reset on v2:172.21.15.66:6836/3870845253 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 1 -- 172.21.15.37:0/953708979 >> 172.21.15.37:0/953708979 conn(0x559af54c5400 msgr2=0x559af54f7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 1 -- 172.21.15.37:0/953708979 shutdown_connections 2024-03-26T06:22:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 1 -- 172.21.15.37:0/953708979 wait complete. 2024-03-26T06:22:44.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.209+0000 7fe19647a740 20 client.15351 trim_cache size 0 max 16384 2024-03-26T06:22:44.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe19647a740 1 Processor -- start 2024-03-26T06:22:44.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe19647a740 1 -- start start 2024-03-26T06:22:44.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe19647a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af551c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe19647a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af550eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe19647a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 0x559af54f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe19647a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af55136b0 con 0x559af54fd0e0 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe19647a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af5513cf0 con 0x559af54fdad0 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe19647a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af5514350 con 0x559af5502c40 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe190e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af551c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe190e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af551c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44934/0 (socket says 172.21.15.37:44934) 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe190e32640 1 -- 172.21.15.37:0/1673140316 learned_addr learned my addr 172.21.15.37:0/1673140316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe190e32640 1 -- 172.21.15.37:0/1673140316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 msgr2=0x559af54f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe190e32640 1 --2- 172.21.15.37:0/1673140316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 0x559af54f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe190e32640 1 -- 172.21.15.37:0/1673140316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 msgr2=0x559af550eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe190e32640 1 --2- 172.21.15.37:0/1673140316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af550eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe190e32640 1 -- 172.21.15.37:0/1673140316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559af54f97a0 con 0x559af54fd0e0 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe190e32640 1 --2- 172.21.15.37:0/1673140316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af551c310 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe17c013550 tx=0x7fe17c011550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358343a0a38d6815 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe176ffd640 1 -- 172.21.15.37:0/1673140316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c019050 con 0x559af54fd0e0 2024-03-26T06:22:44.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.225+0000 7fe176ffd640 1 -- 172.21.15.37:0/1673140316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe17c0043d0 con 0x559af54fd0e0 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe176ffd640 1 -- 172.21.15.37:0/1673140316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c032070 con 0x559af54fd0e0 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe19647a740 1 -- 172.21.15.37:0/1673140316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 msgr2=0x559af551c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe19647a740 1 --2- 172.21.15.37:0/1673140316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af551c310 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe17c013550 tx=0x7fe17c011550 comp rx=0 tx=0).stop 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe19647a740 1 -- 172.21.15.37:0/1673140316 shutdown_connections 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe19647a740 1 --2- 172.21.15.37:0/1673140316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 0x559af54f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe19647a740 1 --2- 172.21.15.37:0/1673140316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af550eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe19647a740 1 --2- 172.21.15.37:0/1673140316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af551c310 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe19647a740 1 -- 172.21.15.37:0/1673140316 >> 172.21.15.37:0/1673140316 conn(0x559af54c5400 msgr2=0x559af5506110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe19647a740 1 -- 172.21.15.37:0/1673140316 shutdown_connections 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe19647a740 1 -- 172.21.15.37:0/1673140316 wait complete. 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe19647a740 1 -- start start 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.226+0000 7fe19647a740 1 Processor -- start 2024-03-26T06:22:44.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe19647a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af550e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe19647a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af550b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe19647a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 0x559af550b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe19647a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af5515010 con 0x559af54fd0e0 2024-03-26T06:22:44.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe19647a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af55149b0 con 0x559af54fdad0 2024-03-26T06:22:44.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe19647a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af5515670 con 0x559af5502c40 2024-03-26T06:22:44.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe19647a740 10 client.0 _ll_register_callbacks cb 0x7fe17c00b4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:44.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe190e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af550e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe190e32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af550e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44936/0 (socket says 172.21.15.37:44936) 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe190e32640 1 -- 172.21.15.37:0/2083499809 learned_addr learned my addr 172.21.15.37:0/2083499809 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe191633640 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af550b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe190e32640 1 -- 172.21.15.37:0/2083499809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 msgr2=0x559af550b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe190e32640 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 0x559af550b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe190e32640 1 -- 172.21.15.37:0/2083499809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 msgr2=0x559af550b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe190e32640 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af550b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe190e32640 1 -- 172.21.15.37:0/2083499809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559af550bfa0 con 0x559af54fd0e0 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe191633640 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af550b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe190e32640 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af550e820 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00b4a0 tx=0x7fe17c00f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a693a2f7b328391 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe174ff9640 1 -- 172.21.15.37:0/2083499809 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c043060 con 0x559af54fd0e0 2024-03-26T06:22:44.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe174ff9640 1 -- 172.21.15.37:0/2083499809 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe17c011ad0 con 0x559af54fd0e0 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.227+0000 7fe19647a740 1 -- 172.21.15.37:0/2083499809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe17c0043d0 con 0x559af54fd0e0 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe174ff9640 1 -- 172.21.15.37:0/2083499809 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe17c032030 con 0x559af54fd0e0 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe19647a740 10 client.15357 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe19647a740 1 -- 172.21.15.37:0/2083499809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559af556ab00 con 0x559af54fd0e0 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe174ff9640 1 -- 172.21.15.37:0/2083499809 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe17c0043d0 con 0x559af54fd0e0 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe174ff9640 1 -- 172.21.15.37:0/2083499809 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe17c004f60 con 0x559af54fd0e0 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe19647a740 10 client.15357 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe19647a740 10 client.15357 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe19647a740 20 client.15357 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe19647a740 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x559af556ef10 0x559af556ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe19647a740 4 client.15357 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe19647a740 1 -- 172.21.15.37:0/2083499809 --> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559af546a050 con 0x559af556ef10 2024-03-26T06:22:44.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.228+0000 7fe191633640 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x559af556ef10 0x559af556ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe174ff9640 10 client.15357 ms_handle_connect on v2:172.21.15.66:6836/3870845253 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe191633640 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x559af556ef10 0x559af556ad80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f936504b54cdaa17 server_cookie=e7861a4e8c3d85d0 in_seq=0 out_seq=0 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe174ff9640 1 -- 172.21.15.37:0/2083499809 <== mds.1 v2:172.21.15.66:6836/3870845253 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x559af546a050 con 0x559af556ef10 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe174ff9640 10 client.15357 handle_command_reply: tid=0 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 1 client.15357 shutdown 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 1 -- 172.21.15.37:0/2083499809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 msgr2=0x559af550e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af550e820 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00b4a0 tx=0x7fe17c00f5f0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 1 -- 172.21.15.37:0/2083499809 shutdown_connections 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.66:6836/3870845253,v1:172.21.15.66:6837/3870845253] conn(0x559af556ef10 0x559af556ad80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559af5502c40 0x559af550b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559af54fdad0 0x559af550b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe174ff9640 0 client.15357 ms_handle_reset on v2:172.21.15.66:6836/3870845253 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 1 --2- 172.21.15.37:0/2083499809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559af54fd0e0 0x559af550e820 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 1 -- 172.21.15.37:0/2083499809 >> 172.21.15.37:0/2083499809 conn(0x559af54c5400 msgr2=0x559af551c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 1 -- 172.21.15.37:0/2083499809 shutdown_connections 2024-03-26T06:22:44.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 1 -- 172.21.15.37:0/2083499809 wait complete. 2024-03-26T06:22:44.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.229+0000 7fe19647a740 20 client.15357 trim_cache size 0 max 16384 2024-03-26T06:22:44.245 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:22:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:22:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-03-26T06:22:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24494, 2024-03-26T06:22:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-26T06:22:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-26T06:22:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-26T06:22:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 107.636166531, 2024-03-26T06:22:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-03-26T06:22:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 87, 2024-03-26T06:22:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T06:22:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 126.060220411 2024-03-26T06:22:44.247 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:22:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 -- 172.21.15.37:0/4229633314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe1780a2800 msgr2=0x7fe1780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe1780a2800 0x7fe1780a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe18404d000 tx=0x7fe1840e8360 comp rx=0 tx=0).stop 2024-03-26T06:22:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 -- 172.21.15.37:0/4229633314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18c101da0 msgr2=0x7fe18c19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18c101da0 0x7fe18c19eb70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe18402c700 tx=0x7fe184032f20 comp rx=0 tx=0).stop 2024-03-26T06:22:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 -- 172.21.15.37:0/4229633314 shutdown_connections 2024-03-26T06:22:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fe1780a2800 0x7fe1780a4cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18c101da0 0x7fe18c19eb70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe18c1013e0 0x7fe18c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 --2- 172.21.15.37:0/4229633314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe18c1001f0 0x7fe18c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 -- 172.21.15.37:0/4229633314 >> 172.21.15.37:0/4229633314 conn(0x7fe18c0fb780 msgr2=0x7fe18c0fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 -- 172.21.15.37:0/4229633314 shutdown_connections 2024-03-26T06:22:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.230+0000 7fe193897640 1 -- 172.21.15.37:0/4229633314 wait complete. 2024-03-26T06:22:44.250 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T06:22:44.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.354+0000 7ff385b59640 1 -- 172.21.15.37:0/3968197368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff380101400 msgr2=0x7ff380101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.354+0000 7ff385b59640 1 --2- 172.21.15.37:0/3968197368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff380101400 0x7ff380101880 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff374009900 tx=0x7ff37402bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:44.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.354+0000 7ff385b59640 1 -- 172.21.15.37:0/3968197368 shutdown_connections 2024-03-26T06:22:44.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.354+0000 7ff385b59640 1 --2- 172.21.15.37:0/3968197368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380101dc0 0x7ff380108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.354+0000 7ff385b59640 1 --2- 172.21.15.37:0/3968197368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff380101400 0x7ff380101880 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.354+0000 7ff385b59640 1 --2- 172.21.15.37:0/3968197368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff380100210 0x7ff380100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.354+0000 7ff385b59640 1 -- 172.21.15.37:0/3968197368 >> 172.21.15.37:0/3968197368 conn(0x7ff3800fb7a0 msgr2=0x7ff3800fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.354+0000 7ff385b59640 1 -- 172.21.15.37:0/3968197368 shutdown_connections 2024-03-26T06:22:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.354+0000 7ff385b59640 1 -- 172.21.15.37:0/3968197368 wait complete. 2024-03-26T06:22:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.354+0000 7ff385b59640 1 Processor -- start 2024-03-26T06:22:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff385b59640 1 -- start start 2024-03-26T06:22:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff385b59640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff380100210 0x7ff380197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff385b59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff380101400 0x7ff380197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff385b59640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380101dc0 0x7ff38019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff385b59640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38010bea0 con 0x7ff380101400 2024-03-26T06:22:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff385b59640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38010bd20 con 0x7ff380100210 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff385b59640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38010c020 con 0x7ff380101dc0 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff37effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff380101400 0x7ff380197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff37effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff380101400 0x7ff380197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44940/0 (socket says 172.21.15.37:44940) 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff37effd640 1 -- 172.21.15.37:0/3999461060 learned_addr learned my addr 172.21.15.37:0/3999461060 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff380100210 0x7ff380197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff37effd640 1 -- 172.21.15.37:0/3999461060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380101dc0 msgr2=0x7ff38019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff37effd640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380101dc0 0x7ff38019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff37effd640 1 -- 172.21.15.37:0/3999461060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff380100210 msgr2=0x7ff380197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff37effd640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff380100210 0x7ff380197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff37effd640 1 -- 172.21.15.37:0/3999461060 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38019f250 con 0x7ff380101400 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff37effd640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff380101400 0x7ff380197bc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff374009230 tx=0x7ff374005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.355+0000 7ff37cff9640 1 -- 172.21.15.37:0/3999461060 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff374041070 con 0x7ff380101400 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.356+0000 7ff37cff9640 1 -- 172.21.15.37:0/3999461060 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3740054e0 con 0x7ff380101400 2024-03-26T06:22:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.356+0000 7ff385b59640 1 -- 172.21.15.37:0/3999461060 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38019f4e0 con 0x7ff380101400 2024-03-26T06:22:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.356+0000 7ff37cff9640 1 -- 172.21.15.37:0/3999461060 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff374034440 con 0x7ff380101400 2024-03-26T06:22:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.356+0000 7ff385b59640 1 -- 172.21.15.37:0/3999461060 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38019f9d0 con 0x7ff380101400 2024-03-26T06:22:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff37cff9640 1 -- 172.21.15.37:0/3999461060 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff374044070 con 0x7ff380101400 2024-03-26T06:22:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff388807740 1 Processor -- start 2024-03-26T06:22:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff388807740 1 -- start start 2024-03-26T06:22:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff37cff9640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff3680a2800 0x7ff3680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff388807740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86f4f0 0x55cbec86f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff388807740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86fee0 0x55cbec8749e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff388807740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec875050 0x55cbec8774e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff388807740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec7d07c0 con 0x55cbec86f4f0 2024-03-26T06:22:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff388807740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec779b50 con 0x55cbec86fee0 2024-03-26T06:22:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff388807740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec7cc860 con 0x55cbec875050 2024-03-26T06:22:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff37ffff640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff3680a2800 0x7ff3680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86f4f0 0x55cbec86f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff37cff9640 1 -- 172.21.15.37:0/3999461060 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3740f0760 con 0x7ff380101400 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86f4f0 0x55cbec86f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44942/0 (socket says 172.21.15.37:44942) 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.358+0000 7ff37f7fe640 1 -- 172.21.15.37:0/732688901 learned_addr learned my addr 172.21.15.37:0/732688901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff37ffff640 1 --2- 172.21.15.37:0/732688901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec875050 0x55cbec8774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff37effd640 1 --2- 172.21.15.37:0/732688901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86fee0 0x55cbec8749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff37f7fe640 1 -- 172.21.15.37:0/732688901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec875050 msgr2=0x55cbec8774e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/732688901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec875050 0x55cbec8774e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff37f7fe640 1 -- 172.21.15.37:0/732688901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86fee0 msgr2=0x55cbec8749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/732688901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86fee0 0x55cbec8749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff37f7fe640 1 -- 172.21.15.37:0/732688901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbec78d9a0 con 0x55cbec86f4f0 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff37ffff640 1 --2- 172.21.15.37:0/732688901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec875050 0x55cbec8774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff37effd640 1 --2- 172.21.15.37:0/732688901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86fee0 0x55cbec8749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/732688901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86f4f0 0x55cbec86f910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff370003a70 tx=0x7ff370003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb7fd5753d45716 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff37ffff640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff3680a2800 0x7ff3680a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff378005fd0 tx=0x7ff37800b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff365ffb640 1 -- 172.21.15.37:0/732688901 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370012450 con 0x55cbec86f4f0 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff365ffb640 1 -- 172.21.15.37:0/732688901 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff370012ad0 con 0x55cbec86f4f0 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.359+0000 7ff365ffb640 1 -- 172.21.15.37:0/732688901 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370011c70 con 0x55cbec86f4f0 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 -- 172.21.15.37:0/732688901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86f4f0 msgr2=0x55cbec86f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 --2- 172.21.15.37:0/732688901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86f4f0 0x55cbec86f910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff370003a70 tx=0x7ff370003f30 comp rx=0 tx=0).stop 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 -- 172.21.15.37:0/732688901 shutdown_connections 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 --2- 172.21.15.37:0/732688901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec875050 0x55cbec8774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 --2- 172.21.15.37:0/732688901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86fee0 0x55cbec8749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 --2- 172.21.15.37:0/732688901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86f4f0 0x55cbec86f910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 -- 172.21.15.37:0/732688901 >> 172.21.15.37:0/732688901 conn(0x55cbec833400 msgr2=0x55cbec866a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 -- 172.21.15.37:0/732688901 shutdown_connections 2024-03-26T06:22:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 -- 172.21.15.37:0/732688901 wait complete. 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 -- start start 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 Processor -- start 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 0x55cbec879910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec879e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec875050 0x55cbec880e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec878000 con 0x55cbec86fee0 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec78cf90 con 0x55cbec86f4f0 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.360+0000 7ff388807740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec878180 con 0x55cbec875050 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff388807740 10 client.0 _ll_register_callbacks cb 0x55cbec7c8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff37effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec879e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff37effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec879e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44946/0 (socket says 172.21.15.37:44946) 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff37effd640 1 -- 172.21.15.37:0/3430861032 learned_addr learned my addr 172.21.15.37:0/3430861032 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff37effd640 1 -- 172.21.15.37:0/3430861032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec875050 msgr2=0x55cbec880e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff37effd640 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec875050 0x55cbec880e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 0x55cbec879910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff37effd640 1 -- 172.21.15.37:0/3430861032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 msgr2=0x55cbec879910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff37effd640 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 0x55cbec879910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff37effd640 1 -- 172.21.15.37:0/3430861032 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbec4e52d0 con 0x55cbec86fee0 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 0x55cbec879910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff37effd640 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec879e50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff374003fd0 tx=0x7ff3740bcc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7c924a26e13584 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff35effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff35effd640 1 -- 172.21.15.37:0/3430861032 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37404b070 con 0x55cbec86fee0 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff35effd640 1 -- 172.21.15.37:0/3430861032 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3740bb410 con 0x55cbec86fee0 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff388807740 1 -- 172.21.15.37:0/3430861032 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff370012ad0 con 0x55cbec86fee0 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff35effd640 1 -- 172.21.15.37:0/3430861032 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff374004c60 con 0x55cbec86fee0 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff388807740 10 client.15369 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff388807740 1 -- 172.21.15.37:0/3430861032 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cbec5af6b0 con 0x55cbec86fee0 2024-03-26T06:22:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.361+0000 7ff35effd640 1 -- 172.21.15.37:0/3430861032 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff370012ad0 con 0x55cbec86fee0 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.362+0000 7ff35effd640 1 -- 172.21.15.37:0/3430861032 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff3740c5420 con 0x55cbec86fee0 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.362+0000 7ff388807740 10 client.15369 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.362+0000 7ff388807740 10 client.15369 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.362+0000 7ff388807740 20 client.15369 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.362+0000 7ff388807740 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbec8864d0 0x55cbec8852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.362+0000 7ff388807740 4 client.15369 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.362+0000 7ff388807740 1 -- 172.21.15.37:0/3430861032 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbec7ddd00 con 0x55cbec8864d0 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.362+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbec8864d0 0x55cbec8852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.362+0000 7ff35effd640 10 client.15369 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.362+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbec8864d0 0x55cbec8852f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b207783d48c6bd1 server_cookie=4d40db98851fd96c in_seq=0 out_seq=0 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff35effd640 1 -- 172.21.15.37:0/3430861032 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbec7ddd00 con 0x55cbec8864d0 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff35effd640 10 client.15369 handle_command_reply: tid=0 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 1 client.15369 shutdown 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 1 -- 172.21.15.37:0/3430861032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 msgr2=0x55cbec879e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec879e50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff374003fd0 tx=0x7ff3740bcc50 comp rx=0 tx=0).stop 2024-03-26T06:22:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 1 -- 172.21.15.37:0/3430861032 shutdown_connections 2024-03-26T06:22:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbec8864d0 0x55cbec8852f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec875050 0x55cbec880e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec879e50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff35effd640 0 client.15369 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:22:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 1 --2- 172.21.15.37:0/3430861032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 0x55cbec879910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 1 -- 172.21.15.37:0/3430861032 >> 172.21.15.37:0/3430861032 conn(0x55cbec833400 msgr2=0x55cbec865c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 1 -- 172.21.15.37:0/3430861032 shutdown_connections 2024-03-26T06:22:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 1 -- 172.21.15.37:0/3430861032 wait complete. 2024-03-26T06:22:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.363+0000 7ff388807740 20 client.15369 trim_cache size 0 max 16384 2024-03-26T06:22:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.379+0000 7ff388807740 1 Processor -- start 2024-03-26T06:22:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.379+0000 7ff388807740 1 -- start start 2024-03-26T06:22:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.379+0000 7ff388807740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 0x55cbec879910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.379+0000 7ff388807740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec86fee0 0x55cbec879e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.379+0000 7ff388807740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec875050 0x55cbec880e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.379+0000 7ff388807740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec885cf0 con 0x55cbec875050 2024-03-26T06:22:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.379+0000 7ff388807740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec886350 con 0x55cbec86f4f0 2024-03-26T06:22:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.379+0000 7ff388807740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec8869b0 con 0x55cbec86fee0 2024-03-26T06:22:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.379+0000 7ff37ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec875050 0x55cbec880e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.379+0000 7ff37ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec875050 0x55cbec880e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44952/0 (socket says 172.21.15.37:44952) 2024-03-26T06:22:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.379+0000 7ff37ffff640 1 -- 172.21.15.37:0/336434178 learned_addr learned my addr 172.21.15.37:0/336434178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff37effd640 1 --2- 172.21.15.37:0/336434178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec86fee0 0x55cbec879e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/336434178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 0x55cbec879910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff37ffff640 1 -- 172.21.15.37:0/336434178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec86fee0 msgr2=0x55cbec879e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff37ffff640 1 --2- 172.21.15.37:0/336434178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec86fee0 0x55cbec879e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff37ffff640 1 -- 172.21.15.37:0/336434178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 msgr2=0x55cbec879910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff37ffff640 1 --2- 172.21.15.37:0/336434178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 0x55cbec879910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff37ffff640 1 -- 172.21.15.37:0/336434178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbec8702c0 con 0x55cbec875050 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/336434178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 0x55cbec879910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff37ffff640 1 --2- 172.21.15.37:0/336434178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec875050 0x55cbec880e50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff37800dd60 tx=0x7ff378005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aeca91a02f2e570 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff3657fa640 1 -- 172.21.15.37:0/336434178 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37801a050 con 0x55cbec875050 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff3657fa640 1 -- 172.21.15.37:0/336434178 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff378024070 con 0x55cbec875050 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.380+0000 7ff3657fa640 1 -- 172.21.15.37:0/336434178 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378012760 con 0x55cbec875050 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 -- 172.21.15.37:0/336434178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec875050 msgr2=0x55cbec880e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 --2- 172.21.15.37:0/336434178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec875050 0x55cbec880e50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff37800dd60 tx=0x7ff378005ed0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 -- 172.21.15.37:0/336434178 shutdown_connections 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 --2- 172.21.15.37:0/336434178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec875050 0x55cbec880e50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 --2- 172.21.15.37:0/336434178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec86fee0 0x55cbec879e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 --2- 172.21.15.37:0/336434178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec86f4f0 0x55cbec879910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 -- 172.21.15.37:0/336434178 >> 172.21.15.37:0/336434178 conn(0x55cbec833400 msgr2=0x55cbec88e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 -- 172.21.15.37:0/336434178 shutdown_connections 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 -- 172.21.15.37:0/336434178 wait complete. 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 -- start start 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 Processor -- start 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec86f4f0 0x55cbec880bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec87d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec875050 0x55cbec87dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec887670 con 0x55cbec86fee0 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec887010 con 0x55cbec875050 2024-03-26T06:22:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec887cd0 con 0x55cbec86f4f0 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff388807740 10 client.0 _ll_register_callbacks cb 0x7ff370009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff37effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec87d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.381+0000 7ff37effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec87d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44964/0 (socket says 172.21.15.37:44964) 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff37effd640 1 -- 172.21.15.37:0/3622986491 learned_addr learned my addr 172.21.15.37:0/3622986491 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec86f4f0 0x55cbec880bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff37ffff640 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec875050 0x55cbec87dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff37effd640 1 -- 172.21.15.37:0/3622986491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec86f4f0 msgr2=0x55cbec880bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff37effd640 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec86f4f0 0x55cbec880bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff37effd640 1 -- 172.21.15.37:0/3622986491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec875050 msgr2=0x55cbec87dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff37effd640 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec875050 0x55cbec87dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff37effd640 1 -- 172.21.15.37:0/3622986491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbec87e250 con 0x55cbec86fee0 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff37ffff640 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec875050 0x55cbec87dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:22:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff37effd640 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec87d670 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff37402c7b0 tx=0x7ff374005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19916526cb012d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff35effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff35effd640 1 -- 172.21.15.37:0/3622986491 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37404b040 con 0x55cbec86fee0 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff388807740 1 -- 172.21.15.37:0/3622986491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff378024070 con 0x55cbec86fee0 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff35effd640 1 -- 172.21.15.37:0/3622986491 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3740bc560 con 0x55cbec86fee0 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff35effd640 1 -- 172.21.15.37:0/3622986491 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3740dd0b0 con 0x55cbec86fee0 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff388807740 10 client.15375 fetch_fsmap learned FSMap version 26 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff388807740 1 -- 172.21.15.37:0/3622986491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cbec8dd270 con 0x55cbec86fee0 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.382+0000 7ff35effd640 1 -- 172.21.15.37:0/3622986491 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff378024070 con 0x55cbec86fee0 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.383+0000 7ff35effd640 1 -- 172.21.15.37:0/3622986491 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff3740bc060 con 0x55cbec86fee0 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.383+0000 7ff388807740 10 client.15375 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.383+0000 7ff388807740 10 client.15375 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.383+0000 7ff388807740 20 client.15375 populate_metadata read hostname 'smithi037' 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.383+0000 7ff388807740 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbec8e1890 0x55cbec8dd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.383+0000 7ff388807740 4 client.15375 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.383+0000 7ff388807740 1 -- 172.21.15.37:0/3622986491 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cbec7d8050 con 0x55cbec8e1890 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.383+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbec8e1890 0x55cbec8dd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.383+0000 7ff35effd640 10 client.15375 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.383+0000 7ff37f7fe640 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbec8e1890 0x55cbec8dd500 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90488b30d434aeb5 server_cookie=f556f3af69d90d06 in_seq=0 out_seq=0 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff35effd640 1 -- 172.21.15.37:0/3622986491 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55cbec7d8050 con 0x55cbec8e1890 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff35effd640 10 client.15375 handle_command_reply: tid=0 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 1 client.15375 shutdown 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 1 -- 172.21.15.37:0/3622986491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 msgr2=0x55cbec87d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec87d670 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff37402c7b0 tx=0x7ff374005bd0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 1 -- 172.21.15.37:0/3622986491 shutdown_connections 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbec8e1890 0x55cbec8dd500 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbec875050 0x55cbec87dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbec86fee0 0x55cbec87d670 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff35effd640 0 client.15375 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:22:44.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 1 --2- 172.21.15.37:0/3622986491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec86f4f0 0x55cbec880bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 1 -- 172.21.15.37:0/3622986491 >> 172.21.15.37:0/3622986491 conn(0x55cbec833400 msgr2=0x55cbec879910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 1 -- 172.21.15.37:0/3622986491 shutdown_connections 2024-03-26T06:22:44.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 1 -- 172.21.15.37:0/3622986491 wait complete. 2024-03-26T06:22:44.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff388807740 20 client.15375 trim_cache size 0 max 16384 2024-03-26T06:22:44.393 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:22:44.393 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a86fb218-eb37-11ee-95d0-87774f69a715", 2024-03-26T06:22:44.393 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-03-26T06:22:44.393 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24545, 2024-03-26T06:22:44.393 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-03-26T06:22:44.394 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-03-26T06:22:44.394 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-03-26T06:22:44.394 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 110.11322058499999, 2024-03-26T06:22:44.394 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-03-26T06:22:44.394 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 87, 2024-03-26T06:22:44.394 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T06:22:44.394 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 115.253494313 2024-03-26T06:22:44.394 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff385b59640 1 -- 172.21.15.37:0/3999461060 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff3680a2800 msgr2=0x7ff3680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff385b59640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff3680a2800 0x7ff3680a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff378005fd0 tx=0x7ff37800b740 comp rx=0 tx=0).stop 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff385b59640 1 -- 172.21.15.37:0/3999461060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff380101400 msgr2=0x7ff380197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff385b59640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff380101400 0x7ff380197bc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff374009230 tx=0x7ff374005dd0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff385b59640 1 -- 172.21.15.37:0/3999461060 shutdown_connections 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff385b59640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7ff3680a2800 0x7ff3680a4cc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff385b59640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff380101dc0 0x7ff38019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff385b59640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff380101400 0x7ff380197bc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff385b59640 1 --2- 172.21.15.37:0/3999461060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff380100210 0x7ff380197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.384+0000 7ff385b59640 1 -- 172.21.15.37:0/3999461060 >> 172.21.15.37:0/3999461060 conn(0x7ff3800fb7a0 msgr2=0x7ff380069900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.385+0000 7ff385b59640 1 -- 172.21.15.37:0/3999461060 shutdown_connections 2024-03-26T06:22:44.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:44.385+0000 7ff385b59640 1 -- 172.21.15.37:0/3999461060 wait complete. 2024-03-26T06:22:44.396 INFO:teuthology.run_tasks:Running task kclient... 2024-03-26T06:22:44.421 INFO:tasks.kclient:Mounting kernel clients... 2024-03-26T06:22:44.421 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-26T06:22:44.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:44.422 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-03-26T06:22:44.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:44.437 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-03-26T06:22:44.500 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:44 smithi037 ceph-mon[31974]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-26T06:22:44.500 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:44 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1543327368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:44.500 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:44 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2480711979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:44.505 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-03-26T06:22:44.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:22:44.507 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-26T06:22:44.507 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-26T06:22:44.507 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-26T06:22:44.507 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-26T06:22:44.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:44 smithi066 ceph-mon[36504]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-26T06:22:44.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:44 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1543327368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:44.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:44 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2480711979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:44.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:44 smithi149 ceph-mon[36681]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-26T06:22:44.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:44 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1543327368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:22:44.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:44 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2480711979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:22:44.789 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:22:46.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.523+0000 7f078667a700 1 -- 172.21.15.37:0/4250861097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0780101330 msgr2=0x7f0780106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:46.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.523+0000 7f078667a700 1 --2- 172.21.15.37:0/4250861097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0780101330 0x7f0780106a70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f07740099c0 tx=0x7f0774009cd0 comp rx=0 tx=0).stop 2024-03-26T06:22:46.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.523+0000 7f078667a700 1 -- 172.21.15.37:0/4250861097 shutdown_connections 2024-03-26T06:22:46.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.523+0000 7f078667a700 1 --2- 172.21.15.37:0/4250861097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07801071a0 0x7f078010ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:46.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.523+0000 7f078667a700 1 --2- 172.21.15.37:0/4250861097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780100980 0x7f0780100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:46.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.523+0000 7f078667a700 1 --2- 172.21.15.37:0/4250861097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0780101330 0x7f0780106a70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:46.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.523+0000 7f078667a700 1 -- 172.21.15.37:0/4250861097 >> 172.21.15.37:0/4250861097 conn(0x7f0780076020 msgr2=0x7f0780076430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:46.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.523+0000 7f078667a700 1 -- 172.21.15.37:0/4250861097 shutdown_connections 2024-03-26T06:22:46.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.523+0000 7f078667a700 1 -- 172.21.15.37:0/4250861097 wait complete. 2024-03-26T06:22:46.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.524+0000 7f078667a700 1 Processor -- start 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.524+0000 7f078667a700 1 -- start start 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.524+0000 7f078667a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780100980 0x7f0780198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.524+0000 7f078667a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0780101330 0x7f0780198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.524+0000 7f078667a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07801071a0 0x7f078019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.524+0000 7f078667a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07801120f0 con 0x7f0780101330 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.524+0000 7f078667a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0780111f70 con 0x7f0780100980 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.524+0000 7f078667a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0780112270 con 0x7f07801071a0 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.524+0000 7f077f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0780101330 0x7f0780198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.524+0000 7f077f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0780101330 0x7f0780198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44984/0 (socket says 172.21.15.37:44984) 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.524+0000 7f077f7fe700 1 -- 172.21.15.37:0/3779946638 learned_addr learned my addr 172.21.15.37:0/3779946638 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f0784c17700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07801071a0 0x7f078019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f077ffff700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780100980 0x7f0780198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f077f7fe700 1 -- 172.21.15.37:0/3779946638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07801071a0 msgr2=0x7f078019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f077f7fe700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07801071a0 0x7f078019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f077f7fe700 1 -- 172.21.15.37:0/3779946638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780100980 msgr2=0x7f0780198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f077f7fe700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780100980 0x7f0780198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f077f7fe700 1 -- 172.21.15.37:0/3779946638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078019d510 con 0x7f0780101330 2024-03-26T06:22:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f077f7fe700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0780101330 0x7f0780198a40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f077400b580 tx=0x7f0774003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f077d7fa700 1 -- 172.21.15.37:0/3779946638 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774027070 con 0x7f0780101330 2024-03-26T06:22:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f078667a700 1 -- 172.21.15.37:0/3779946638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078019d7a0 con 0x7f0780101330 2024-03-26T06:22:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f077d7fa700 1 -- 172.21.15.37:0/3779946638 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0774004050 con 0x7f0780101330 2024-03-26T06:22:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f078667a700 1 -- 172.21.15.37:0/3779946638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07801a5080 con 0x7f0780101330 2024-03-26T06:22:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.525+0000 7f077d7fa700 1 -- 172.21.15.37:0/3779946638 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774022df0 con 0x7f0780101330 2024-03-26T06:22:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.527+0000 7f078667a700 1 -- 172.21.15.37:0/3779946638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078010a3e0 con 0x7f0780101330 2024-03-26T06:22:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.527+0000 7f077d7fa700 1 -- 172.21.15.37:0/3779946638 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f077402a070 con 0x7f0780101330 2024-03-26T06:22:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.528+0000 7f077d7fa700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f07680a2910 0x7f07680a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.528+0000 7f077ffff700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f07680a2910 0x7f07680a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.528+0000 7f077d7fa700 1 -- 172.21.15.37:0/3779946638 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f07740cf6e0 con 0x7f0780101330 2024-03-26T06:22:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.528+0000 7f077ffff700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f07680a2910 0x7f07680a4dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0770005fd0 tx=0x7f077000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:46.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.532+0000 7f077d7fa700 1 -- 172.21.15.37:0/3779946638 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0774013070 con 0x7f0780101330 2024-03-26T06:22:46.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.786+0000 7f078667a700 1 -- 172.21.15.37:0/3779946638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0780199860 con 0x7f0780101330 2024-03-26T06:22:46.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.786+0000 7f077d7fa700 1 -- 172.21.15.37:0/3779946638 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f077409c820 con 0x7f0780101330 2024-03-26T06:22:46.787 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-26T06:22:46.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.788+0000 7f078667a700 1 -- 172.21.15.37:0/3779946638 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f07680a2910 msgr2=0x7f07680a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:46.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.788+0000 7f078667a700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f07680a2910 0x7f07680a4dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0770005fd0 tx=0x7f077000a560 comp rx=0 tx=0).stop 2024-03-26T06:22:46.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.788+0000 7f078667a700 1 -- 172.21.15.37:0/3779946638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0780101330 msgr2=0x7f0780198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:46.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.788+0000 7f078667a700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0780101330 0x7f0780198a40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f077400b580 tx=0x7f0774003890 comp rx=0 tx=0).stop 2024-03-26T06:22:46.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.788+0000 7f078667a700 1 -- 172.21.15.37:0/3779946638 shutdown_connections 2024-03-26T06:22:46.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.788+0000 7f078667a700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f07680a2910 0x7f07680a4dd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:46.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.788+0000 7f078667a700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07801071a0 0x7f078019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:46.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.788+0000 7f078667a700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780100980 0x7f0780198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:46.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.788+0000 7f078667a700 1 --2- 172.21.15.37:0/3779946638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0780101330 0x7f0780198a40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:46.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.788+0000 7f078667a700 1 -- 172.21.15.37:0/3779946638 >> 172.21.15.37:0/3779946638 conn(0x7f0780076020 msgr2=0x7f07800fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:46.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.789+0000 7f078667a700 1 -- 172.21.15.37:0/3779946638 shutdown_connections 2024-03-26T06:22:46.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:46.789+0000 7f078667a700 1 -- 172.21.15.37:0/3779946638 wait complete. 2024-03-26T06:22:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:46 smithi149 ceph-mon[36681]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:22:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:46 smithi037 ceph-mon[31974]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:22:47.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:46 smithi066 ceph-mon[36504]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-26T06:22:47.515 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-26T06:22:47.515 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-26T06:22:47.515 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-03-26T06:22:47.515 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-26T06:22:47.515 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-26T06:22:47.516 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-26T06:22:47.516 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-26T06:22:47.516 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-26T06:22:47.516 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-26T06:22:47.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:47.516 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-03-26T06:22:47.533 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-26T06:22:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-26T06:22:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-26T06:22:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42067sec preferred_lft 42067sec 2024-03-26T06:22:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-03-26T06:22:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-26T06:22:47.535 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T06:22:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-03-26T06:22:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-03-26T06:22:47.535 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-26T06:22:47.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:47.536 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-26T06:22:47.536 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-26T06:22:47.536 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-03-26T06:22:47.536 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-03-26T06:22:47.536 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-03-26T06:22:47.536 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-26T06:22:47.536 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-26T06:22:47.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:47.702 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-26T06:22:47.732 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-03-26T06:22:47.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:47.733 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-03-26T06:22:47.801 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-03-26T06:22:47.801 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-26T06:22:47.801 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-26T06:22:47.801 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-26T06:22:47.801 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-26T06:22:47.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:47.802 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-26T06:22:47.802 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-26T06:22:47.802 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-26T06:22:47.802 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-26T06:22:47.802 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-26T06:22:47.802 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-26T06:22:47.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:47 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3779946638' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T06:22:47.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:47 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3779946638' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T06:22:48.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:48.060 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-03-26T06:22:48.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:47 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3779946638' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T06:22:48.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:48.116 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-03-26T06:22:48.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:48.170 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-26T06:22:48.170 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-26T06:22:48.170 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T06:22:48.170 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-26T06:22:48.170 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-26T06:22:48.253 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-26T06:22:48.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:48.254 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-26T06:22:48.254 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-26T06:22:48.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-26T06:22:48.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-26T06:22:48.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-26T06:22:48.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-26T06:22:48.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-26T06:22:48.254 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-26T06:22:48.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:48.395 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-26T06:22:48.395 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-26T06:22:48.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-03-26T06:22:48.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-03-26T06:22:48.395 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-26T06:22:48.437 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T06:22:48.437 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-26T06:22:48.495 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-26T06:22:48.496 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T06:22:48.496 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-26T06:22:48.550 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-26T06:22:48.550 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-26T06:22:48.550 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/c1037191-717f-4157-93f3-2e04e0bac5ca 2024-03-26T06:22:48.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:48.550 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/c1037191-717f-4157-93f3-2e04e0bac5ca /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-26T06:22:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-26T06:22:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-26T06:22:48.634 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-03-26T06:22:48.765 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-26T06:22:48.765 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-26T06:22:48.765 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-03-26T06:22:48.765 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:6789,172.21.15.66:6789,172.21.15.149:6789" 2024-03-26T06:22:48.765 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:6789,172.21.15.66:6789,172.21.15.149:6789:/volumes/qa/sv_1/c1037191-717f-4157-93f3-2e04e0bac5ca 2024-03-26T06:22:48.765 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=a86fb218-eb37-11ee-95d0-87774f69a715" will pass to kernel 2024-03-26T06:22:48.765 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-26T06:22:48.766 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-03-26T06:22:48.766 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-26T06:22:48.766 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-03-26T06:22:48.766 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-26T06:22:48.767 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-26T06:22:48.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:48.768 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-26T06:22:48.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:48.796 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T06:22:48.852 INFO:teuthology.orchestra.run.smithi037.stdout:client24995 2024-03-26T06:22:48.852 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:22:48.852 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T06:22:48.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:48.909 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T06:22:48.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:48 smithi149 ceph-mon[36681]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:48.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:48 smithi037 ceph-mon[31974]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:48.964 INFO:teuthology.orchestra.run.smithi037.stdout:client24995 2024-03-26T06:22:48.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:48.964 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T06:22:49.020 INFO:teuthology.orchestra.run.smithi037.stdout:client24995 2024-03-26T06:22:49.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:22:49.020 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/a86fb218-eb37-11ee-95d0-87774f69a715.client24995/status') 2024-03-26T06:22:49.086 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.24995 (3)172.21.15.37:0/322672061 2024-03-26T06:22:49.086 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-03-26T06:22:49.086 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-03-26T06:22:49.087 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-03-26T06:22:49.087 INFO:teuthology.orchestra.run.smithi037.stderr:70 bytes copied, 2.1553e-05 s, 3.2 MB/s 2024-03-26T06:22:49.088 INFO:teuthology.run_tasks:Running task exec... 2024-03-26T06:22:49.098 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T06:22:49.098 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-26T06:22:49.098 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-26T06:22:49.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:48 smithi066 ceph-mon[36504]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T06:22:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 -- 172.21.15.37:0/3503673461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff4102ef0 msgr2=0x7f5ff4109780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 --2- 172.21.15.37:0/3503673461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff4102ef0 0x7f5ff4109780 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009ca0 tx=0x7f5fe402c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 -- 172.21.15.37:0/3503673461 shutdown_connections 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 --2- 172.21.15.37:0/3503673461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff4102ef0 0x7f5ff4109780 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 --2- 172.21.15.37:0/3503673461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ff4102530 0x7f5ff41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 --2- 172.21.15.37:0/3503673461 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff4101340 0x7f5ff4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 -- 172.21.15.37:0/3503673461 >> 172.21.15.37:0/3503673461 conn(0x7f5ff40fc8d0 msgr2=0x7f5ff40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 -- 172.21.15.37:0/3503673461 shutdown_connections 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 -- 172.21.15.37:0/3503673461 wait complete. 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 Processor -- start 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 -- start start 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff4101340 0x7f5ff4075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff4102530 0x7f5ff40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.231+0000 7f5ffc797640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ff4102ef0 0x7f5ff4073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ffc797640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff410cfd0 con 0x7f5ff4102530 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ffc797640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff410ce50 con 0x7f5ff4101340 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ffc797640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff410d150 con 0x7f5ff4102ef0 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff9d32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff4102530 0x7f5ff40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff9d32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff4102530 0x7f5ff40736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41006/0 (socket says 172.21.15.37:41006) 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff9d32640 1 -- 172.21.15.37:0/3981180305 learned_addr learned my addr 172.21.15.37:0/3981180305 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ffa533640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff4101340 0x7f5ff4075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ffad34640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ff4102ef0 0x7f5ff4073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff9d32640 1 -- 172.21.15.37:0/3981180305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ff4102ef0 msgr2=0x7f5ff4073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff9d32640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ff4102ef0 0x7f5ff4073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff9d32640 1 -- 172.21.15.37:0/3981180305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff4101340 msgr2=0x7f5ff4075050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff9d32640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff4101340 0x7f5ff4075050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff9d32640 1 -- 172.21.15.37:0/3981180305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff40744e0 con 0x7f5ff4102530 2024-03-26T06:22:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ffa533640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff4101340 0x7f5ff4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:22:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff9d32640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff4102530 0x7f5ff40736e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800c970 tx=0x7f5fe800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff37fe640 1 -- 172.21.15.37:0/3981180305 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8007bb0 con 0x7f5ff4102530 2024-03-26T06:22:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ffc797640 1 -- 172.21.15.37:0/3981180305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff41a1720 con 0x7f5ff4102530 2024-03-26T06:22:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff37fe640 1 -- 172.21.15.37:0/3981180305 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fe8007d50 con 0x7f5ff4102530 2024-03-26T06:22:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ff37fe640 1 -- 172.21.15.37:0/3981180305 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe800f450 con 0x7f5ff4102530 2024-03-26T06:22:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.232+0000 7f5ffc797640 1 -- 172.21.15.37:0/3981180305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff41a1bf0 con 0x7f5ff4102530 2024-03-26T06:22:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.233+0000 7f5ffc797640 1 -- 172.21.15.37:0/3981180305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff4108590 con 0x7f5ff4102530 2024-03-26T06:22:49.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.234+0000 7f5ff37fe640 1 -- 172.21.15.37:0/3981180305 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5fe8005020 con 0x7f5ff4102530 2024-03-26T06:22:49.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.235+0000 7f5ff37fe640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5fdc0a28a0 0x7f5fdc0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:49.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.235+0000 7f5ff37fe640 1 -- 172.21.15.37:0/3981180305 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5fe80ce860 con 0x7f5ff4102530 2024-03-26T06:22:49.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.235+0000 7f5ffa533640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5fdc0a28a0 0x7f5fdc0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:49.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.235+0000 7f5ffa533640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5fdc0a28a0 0x7f5fdc0a4d60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0058c0 tx=0x7f5fec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:49.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.239+0000 7f5ff37fe640 1 -- 172.21.15.37:0/3981180305 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5fe8014030 con 0x7f5ff4102530 2024-03-26T06:22:49.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.465+0000 7f5ffc797640 1 -- 172.21.15.37:0/3981180305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f5ff4061ef0 con 0x7f5ff4102530 2024-03-26T06:22:49.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.614+0000 7f5ff37fe640 1 -- 172.21.15.37:0/3981180305 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) v1 ==== 117+0+0 (secure 0 0 0) 0x7f5fe809b910 con 0x7f5ff4102530 2024-03-26T06:22:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 -- 172.21.15.37:0/3981180305 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5fdc0a28a0 msgr2=0x7f5fdc0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5fdc0a28a0 0x7f5fdc0a4d60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0058c0 tx=0x7f5fec005850 comp rx=0 tx=0).stop 2024-03-26T06:22:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 -- 172.21.15.37:0/3981180305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff4102530 msgr2=0x7f5ff40736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff4102530 0x7f5ff40736e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800c970 tx=0x7f5fe800ce30 comp rx=0 tx=0).stop 2024-03-26T06:22:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 -- 172.21.15.37:0/3981180305 shutdown_connections 2024-03-26T06:22:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f5fdc0a28a0 0x7f5fdc0a4d60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ff4102ef0 0x7f5ff4073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff4102530 0x7f5ff40736e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 --2- 172.21.15.37:0/3981180305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ff4101340 0x7f5ff4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 -- 172.21.15.37:0/3981180305 >> 172.21.15.37:0/3981180305 conn(0x7f5ff40fc8d0 msgr2=0x7f5ff4105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 -- 172.21.15.37:0/3981180305 shutdown_connections 2024-03-26T06:22:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:49.617+0000 7f5ffc797640 1 -- 172.21.15.37:0/3981180305 wait complete. 2024-03-26T06:22:49.698 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-26T06:22:49.950 INFO:teuthology.run_tasks:Running task exec... 2024-03-26T06:22:50.657 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T06:22:50.657 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-26T06:22:50.657 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-26T06:22:50.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:49 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3981180305' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-26T06:22:50.658 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:49 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3981180305' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-26T06:22:50.659 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:49 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3981180305' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-26T06:22:50.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.771+0000 7fecc8f22640 1 -- 172.21.15.37:0/3950189264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4100260 msgr2=0x7fecc4100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:50.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.771+0000 7fecc8f22640 1 --2- 172.21.15.37:0/3950189264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4100260 0x7fecc4100680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009900 tx=0x7fecb402bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:50.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.771+0000 7fecc8f22640 1 -- 172.21.15.37:0/3950189264 shutdown_connections 2024-03-26T06:22:50.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.771+0000 7fecc8f22640 1 --2- 172.21.15.37:0/3950189264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc4101e10 0x7fecc41086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:50.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.771+0000 7fecc8f22640 1 --2- 172.21.15.37:0/3950189264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc4101450 0x7fecc41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:50.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.771+0000 7fecc8f22640 1 --2- 172.21.15.37:0/3950189264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4100260 0x7fecc4100680 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:50.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.771+0000 7fecc8f22640 1 -- 172.21.15.37:0/3950189264 >> 172.21.15.37:0/3950189264 conn(0x7fecc40fb7d0 msgr2=0x7fecc40fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.771+0000 7fecc8f22640 1 -- 172.21.15.37:0/3950189264 shutdown_connections 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.771+0000 7fecc8f22640 1 -- 172.21.15.37:0/3950189264 wait complete. 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecc8f22640 1 Processor -- start 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecc8f22640 1 -- start start 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecc8f22640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc4100260 0x7fecc41041f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecc8f22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4101450 0x7fecc4102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecc8f22640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc4101e10 0x7fecc4102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecc8f22640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc410bef0 con 0x7fecc4101450 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecc8f22640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc410bd70 con 0x7fecc4101e10 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecc8f22640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc410c070 con 0x7fecc4100260 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4101450 0x7fecc4102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:50.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4101450 0x7fecc4102830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41030/0 (socket says 172.21.15.37:41030) 2024-03-26T06:22:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecbbfff640 1 -- 172.21.15.37:0/2624227887 learned_addr learned my addr 172.21.15.37:0/2624227887 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecc259c640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc4100260 0x7fecc41041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.772+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc4101e10 0x7fecc4102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.773+0000 7fecbbfff640 1 -- 172.21.15.37:0/2624227887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc4100260 msgr2=0x7fecc41041f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.773+0000 7fecbbfff640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc4100260 0x7fecc41041f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.773+0000 7fecbbfff640 1 -- 172.21.15.37:0/2624227887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc4101e10 msgr2=0x7fecc4102d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.773+0000 7fecbbfff640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc4101e10 0x7fecc4102d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.773+0000 7fecbbfff640 1 -- 172.21.15.37:0/2624227887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc4103600 con 0x7fecc4101450 2024-03-26T06:22:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.773+0000 7fecbbfff640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4101450 0x7fecc4102830 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fecac00d770 tx=0x7fecac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.773+0000 7fecbb7fe640 1 -- 172.21.15.37:0/2624227887 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac017070 con 0x7fecc4101450 2024-03-26T06:22:50.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.773+0000 7fecbb7fe640 1 -- 172.21.15.37:0/2624227887 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecac005500 con 0x7fecc4101450 2024-03-26T06:22:50.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.773+0000 7fecc8f22640 1 -- 172.21.15.37:0/2624227887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc41a16f0 con 0x7fecc4101450 2024-03-26T06:22:50.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.773+0000 7fecbb7fe640 1 -- 172.21.15.37:0/2624227887 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac004e00 con 0x7fecc4101450 2024-03-26T06:22:50.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.773+0000 7fecc8f22640 1 -- 172.21.15.37:0/2624227887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc41a1c40 con 0x7fecc4101450 2024-03-26T06:22:50.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.774+0000 7fecc8f22640 1 -- 172.21.15.37:0/2624227887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc41a1f00 con 0x7fecc4101450 2024-03-26T06:22:50.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.775+0000 7fecbb7fe640 1 -- 172.21.15.37:0/2624227887 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fecac005000 con 0x7fecc4101450 2024-03-26T06:22:50.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.776+0000 7fecbb7fe640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fecb00a2920 0x7fecb00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:50.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.776+0000 7fecc259c640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fecb00a2920 0x7fecb00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:50.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.776+0000 7fecbb7fe640 1 -- 172.21.15.37:0/2624227887 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fecac0ce360 con 0x7fecc4101450 2024-03-26T06:22:50.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.776+0000 7fecc259c640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fecb00a2920 0x7fecb00a4de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fecb4000c00 tx=0x7fecb4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:50.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:50.780+0000 7fecbb7fe640 1 -- 172.21.15.37:0/2624227887 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fecac09adc0 con 0x7fecc4101450 2024-03-26T06:22:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:50 smithi149 ceph-mon[36681]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:22:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:50 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3981180305' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-26T06:22:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:50 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:22:50.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:50 smithi037 ceph-mon[31974]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:22:50.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:50 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3981180305' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-26T06:22:50.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:50 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:22:51.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.015+0000 7fecc8f22640 1 -- 172.21.15.37:0/2624227887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fecc41a2290 con 0x7fecc4101450 2024-03-26T06:22:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:50 smithi066 ceph-mon[36504]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T06:22:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:50 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3981180305' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-26T06:22:51.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:50 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:22:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.624+0000 7fecbb7fe640 1 -- 172.21.15.37:0/2624227887 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7fecac09ee60 con 0x7fecc4101450 2024-03-26T06:22:51.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 -- 172.21.15.37:0/2624227887 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fecb00a2920 msgr2=0x7fecb00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:51.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fecb00a2920 0x7fecb00a4de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fecb4000c00 tx=0x7fecb4005fd0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 -- 172.21.15.37:0/2624227887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4101450 msgr2=0x7fecc4102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:51.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4101450 0x7fecc4102830 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fecac00d770 tx=0x7fecac00dc30 comp rx=0 tx=0).stop 2024-03-26T06:22:51.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 -- 172.21.15.37:0/2624227887 shutdown_connections 2024-03-26T06:22:51.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fecb00a2920 0x7fecb00a4de0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc4101e10 0x7fecc4102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4101450 0x7fecc4102830 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 --2- 172.21.15.37:0/2624227887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc4100260 0x7fecc41041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 -- 172.21.15.37:0/2624227887 >> 172.21.15.37:0/2624227887 conn(0x7fecc40fb7d0 msgr2=0x7fecc40fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:51.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 -- 172.21.15.37:0/2624227887 shutdown_connections 2024-03-26T06:22:51.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.627+0000 7fecc8f22640 1 -- 172.21.15.37:0/2624227887 wait complete. 2024-03-26T06:22:51.646 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-26T06:22:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.773+0000 7fd059632640 1 -- 172.21.15.37:0/1127638606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd054100210 msgr2=0x7fd054100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.773+0000 7fd059632640 1 --2- 172.21.15.37:0/1127638606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd054100210 0x7fd054100630 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009900 tx=0x7fd04c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.773+0000 7fd059632640 1 -- 172.21.15.37:0/1127638606 shutdown_connections 2024-03-26T06:22:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.773+0000 7fd059632640 1 --2- 172.21.15.37:0/1127638606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd054101dc0 0x7fd054108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.773+0000 7fd059632640 1 --2- 172.21.15.37:0/1127638606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd054101400 0x7fd054101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.773+0000 7fd059632640 1 --2- 172.21.15.37:0/1127638606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd054100210 0x7fd054100630 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.773+0000 7fd059632640 1 -- 172.21.15.37:0/1127638606 >> 172.21.15.37:0/1127638606 conn(0x7fd0540fb780 msgr2=0x7fd0540fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.773+0000 7fd059632640 1 -- 172.21.15.37:0/1127638606 shutdown_connections 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.773+0000 7fd059632640 1 -- 172.21.15.37:0/1127638606 wait complete. 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd059632640 1 Processor -- start 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd059632640 1 -- start start 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd059632640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd054100210 0x7fd0541975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd059632640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd054101400 0x7fd054197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd059632640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd054101dc0 0x7fd05419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd059632640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05410bea0 con 0x7fd054101400 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd059632640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05410bd20 con 0x7fd054100210 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd059632640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05410c020 con 0x7fd054101dc0 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd0527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd054101400 0x7fd054197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd0527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd054101400 0x7fd054197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41080/0 (socket says 172.21.15.37:41080) 2024-03-26T06:22:51.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd052ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd054100210 0x7fd0541975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd0527fc640 1 -- 172.21.15.37:0/2017286357 learned_addr learned my addr 172.21.15.37:0/2017286357 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd0537fe640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd054101dc0 0x7fd05419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd0527fc640 1 -- 172.21.15.37:0/2017286357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd054101dc0 msgr2=0x7fd05419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd0527fc640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd054101dc0 0x7fd05419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd0527fc640 1 -- 172.21.15.37:0/2017286357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd054100210 msgr2=0x7fd0541975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd0527fc640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd054100210 0x7fd0541975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd0527fc640 1 -- 172.21.15.37:0/2017286357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05419f200 con 0x7fd054101400 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.774+0000 7fd052ffd640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd054100210 0x7fd0541975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.775+0000 7fd0527fc640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd054101400 0x7fd054197b20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e9f0 tx=0x7fd04800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.775+0000 7fd03bfff640 1 -- 172.21.15.37:0/2017286357 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04800cd80 con 0x7fd054101400 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.775+0000 7fd059632640 1 -- 172.21.15.37:0/2017286357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05419f4f0 con 0x7fd054101400 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.775+0000 7fd03bfff640 1 -- 172.21.15.37:0/2017286357 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd048005500 con 0x7fd054101400 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.775+0000 7fd059632640 1 -- 172.21.15.37:0/2017286357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05419fa40 con 0x7fd054101400 2024-03-26T06:22:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.775+0000 7fd03bfff640 1 -- 172.21.15.37:0/2017286357 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048010690 con 0x7fd054101400 2024-03-26T06:22:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.776+0000 7fd059632640 1 -- 172.21.15.37:0/2017286357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05419fd60 con 0x7fd054101400 2024-03-26T06:22:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.777+0000 7fd03bfff640 1 -- 172.21.15.37:0/2017286357 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 137307+0+0 (secure 0 0 0) 0x7fd0480046c0 con 0x7fd054101400 2024-03-26T06:22:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.778+0000 7fd03bfff640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd03c0a28f0 0x7fd03c0a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:51.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.778+0000 7fd052ffd640 1 -- 172.21.15.37:0/2017286357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd03c0a28f0 msgr2=0x7fd03c0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/641545212 2024-03-26T06:22:51.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.778+0000 7fd03bfff640 1 -- 172.21.15.37:0/2017286357 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd0480cf130 con 0x7fd054101400 2024-03-26T06:22:51.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.778+0000 7fd052ffd640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd03c0a28f0 0x7fd03c0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T06:22:51.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.785+0000 7fd03bfff640 1 -- 172.21.15.37:0/2017286357 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd048053260 con 0x7fd054101400 2024-03-26T06:22:51.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:51 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2624227887' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-26T06:22:51.910 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:51 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: ignoring --setuser ceph since I am not root 2024-03-26T06:22:51.910 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:51 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: ignoring --setgroup ceph since I am not root 2024-03-26T06:22:51.910 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:51 smithi149 ceph-mgr[38361]: -- 172.21.15.149:0/3686631858 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55601cbb45a0 con 0x55601cb89000 2024-03-26T06:22:51.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:51 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2624227887' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-26T06:22:51.948 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:51 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: ignoring --setuser ceph since I am not root 2024-03-26T06:22:51.949 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:51 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: ignoring --setgroup ceph since I am not root 2024-03-26T06:22:51.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.970+0000 7fd059632640 1 -- 172.21.15.37:0/2017286357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fd0541a0430 con 0x7fd054101400 2024-03-26T06:22:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.974+0000 7fd03bfff640 1 -- 172.21.15.37:0/2017286357 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd04809c180 con 0x7fd054101400 2024-03-26T06:22:51.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 -- 172.21.15.37:0/2017286357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd03c0a28f0 msgr2=0x7fd03c0a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T06:22:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd03c0a28f0 0x7fd03c0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 -- 172.21.15.37:0/2017286357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd054101400 msgr2=0x7fd054197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd054101400 0x7fd054197b20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e9f0 tx=0x7fd04800edf0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 -- 172.21.15.37:0/2017286357 shutdown_connections 2024-03-26T06:22:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7fd03c0a28f0 0x7fd03c0a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd054101dc0 0x7fd05419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd054101400 0x7fd054197b20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 --2- 172.21.15.37:0/2017286357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd054100210 0x7fd0541975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 -- 172.21.15.37:0/2017286357 >> 172.21.15.37:0/2017286357 conn(0x7fd0540fb780 msgr2=0x7fd0540fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 -- 172.21.15.37:0/2017286357 shutdown_connections 2024-03-26T06:22:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:51.976+0000 7fd059632640 1 -- 172.21.15.37:0/2017286357 wait complete. 2024-03-26T06:22:51.989 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-26T06:22:52.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.092+0000 7f41cdf0a640 1 -- 172.21.15.37:0/3884540210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8102530 msgr2=0x7f41c81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:52.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.092+0000 7f41cdf0a640 1 --2- 172.21.15.37:0/3884540210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8102530 0x7f41c81029b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009900 tx=0x7f41b802bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:52.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.092+0000 7f41cdf0a640 1 -- 172.21.15.37:0/3884540210 shutdown_connections 2024-03-26T06:22:52.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.092+0000 7f41cdf0a640 1 --2- 172.21.15.37:0/3884540210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c8102ef0 0x7f41c8109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.092+0000 7f41cdf0a640 1 --2- 172.21.15.37:0/3884540210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8102530 0x7f41c81029b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.092+0000 7f41cdf0a640 1 --2- 172.21.15.37:0/3884540210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101340 0x7f41c8101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.092+0000 7f41cdf0a640 1 -- 172.21.15.37:0/3884540210 >> 172.21.15.37:0/3884540210 conn(0x7f41c80fc8b0 msgr2=0x7f41c80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.092+0000 7f41cdf0a640 1 -- 172.21.15.37:0/3884540210 shutdown_connections 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.092+0000 7f41cdf0a640 1 -- 172.21.15.37:0/3884540210 wait complete. 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.092+0000 7f41cdf0a640 1 Processor -- start 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41cdf0a640 1 -- start start 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41cdf0a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101340 0x7f41c8199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41cdf0a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c8102530 0x7f41c8199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41cdf0a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8102ef0 0x7f41c81a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41cdf0a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c810cfd0 con 0x7f41c8102ef0 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41cdf0a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c810ce50 con 0x7f41c8102530 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41cdf0a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c810d150 con 0x7f41c8101340 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8102ef0 0x7f41c81a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8102ef0 0x7f41c81a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41106/0 (socket says 172.21.15.37:41106) 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c8102530 0x7f41c8199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c7fff640 1 -- 172.21.15.37:0/1385027085 learned_addr learned my addr 172.21.15.37:0/1385027085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c77fe640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101340 0x7f41c8199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c7fff640 1 -- 172.21.15.37:0/1385027085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101340 msgr2=0x7f41c8199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c7fff640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101340 0x7f41c8199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c7fff640 1 -- 172.21.15.37:0/1385027085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c8102530 msgr2=0x7f41c8199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c7fff640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c8102530 0x7f41c8199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c7fff640 1 -- 172.21.15.37:0/1385027085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c81a13b0 con 0x7f41c8102ef0 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c6ffd640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c8102530 0x7f41c8199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.093+0000 7f41c7fff640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8102ef0 0x7f41c81a0d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009fd0 tx=0x7f41c000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.094+0000 7f41c4ff9640 1 -- 172.21.15.37:0/1385027085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c0017070 con 0x7f41c8102ef0 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.094+0000 7f41c4ff9640 1 -- 172.21.15.37:0/1385027085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41c0003e50 con 0x7f41c8102ef0 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.094+0000 7f41cdf0a640 1 -- 172.21.15.37:0/1385027085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c81a16a0 con 0x7f41c8102ef0 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.094+0000 7f41c4ff9640 1 -- 172.21.15.37:0/1385027085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c00106c0 con 0x7f41c8102ef0 2024-03-26T06:22:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.094+0000 7f41cdf0a640 1 -- 172.21.15.37:0/1385027085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c81a1bc0 con 0x7f41c8102ef0 2024-03-26T06:22:52.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:51 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2624227887' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-26T06:22:52.098 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:51 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: ignoring --setuser ceph since I am not root 2024-03-26T06:22:52.099 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:51 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: ignoring --setgroup ceph since I am not root 2024-03-26T06:22:52.099 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:51 smithi066 ceph-mgr[37933]: -- 172.21.15.66:0/284491462 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555e342d25a0 con 0x555e342a6800 2024-03-26T06:22:52.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.095+0000 7f41cdf0a640 1 -- 172.21.15.37:0/1385027085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c81a1e20 con 0x7f41c8102ef0 2024-03-26T06:22:52.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.096+0000 7f41c4ff9640 1 -- 172.21.15.37:0/1385027085 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 137307+0+0 (secure 0 0 0) 0x7f41c0003970 con 0x7f41c8102ef0 2024-03-26T06:22:52.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.097+0000 7f41c4ff9640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f41b00a29f0 0x7f41b00a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:52.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.097+0000 7f41c77fe640 1 -- 172.21.15.37:0/1385027085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f41b00a29f0 msgr2=0x7f41b00a4eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/641545212 2024-03-26T06:22:52.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.097+0000 7f41c4ff9640 1 -- 172.21.15.37:0/1385027085 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41c00ce6d0 con 0x7f41c8102ef0 2024-03-26T06:22:52.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.097+0000 7f41c77fe640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f41b00a29f0 0x7f41b00a4eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T06:22:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.100+0000 7f41c4ff9640 1 -- 172.21.15.37:0/1385027085 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f41c009b150 con 0x7f41c8102ef0 2024-03-26T06:22:52.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.285+0000 7f41cdf0a640 1 -- 172.21.15.37:0/1385027085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f41c81a1ff0 con 0x7f41c8102ef0 2024-03-26T06:22:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.289+0000 7f41c4ff9640 1 -- 172.21.15.37:0/1385027085 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f41c009f1f0 con 0x7f41c8102ef0 2024-03-26T06:22:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 -- 172.21.15.37:0/1385027085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f41b00a29f0 msgr2=0x7f41b00a4eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T06:22:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f41b00a29f0 0x7f41b00a4eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 -- 172.21.15.37:0/1385027085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8102ef0 msgr2=0x7f41c81a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8102ef0 0x7f41c81a0d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009fd0 tx=0x7f41c000b260 comp rx=0 tx=0).stop 2024-03-26T06:22:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 -- 172.21.15.37:0/1385027085 shutdown_connections 2024-03-26T06:22:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f41b00a29f0 0x7f41b00a4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41c8102ef0 0x7f41c81a0d40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41c8102530 0x7f41c8199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 --2- 172.21.15.37:0/1385027085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101340 0x7f41c8199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 -- 172.21.15.37:0/1385027085 >> 172.21.15.37:0/1385027085 conn(0x7f41c80fc8b0 msgr2=0x7f41c8073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 -- 172.21.15.37:0/1385027085 shutdown_connections 2024-03-26T06:22:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.291+0000 7f41cdf0a640 1 -- 172.21.15.37:0/1385027085 wait complete. 2024-03-26T06:22:52.303 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-26T06:22:52.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.410+0000 7f52e950a640 1 -- 172.21.15.37:0/2400242815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52e4102ed0 msgr2=0x7f52e4109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.410+0000 7f52e950a640 1 --2- 172.21.15.37:0/2400242815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52e4102ed0 0x7f52e4109760 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009ca0 tx=0x7f52dc02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.410+0000 7f52e950a640 1 -- 172.21.15.37:0/2400242815 shutdown_connections 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.410+0000 7f52e950a640 1 --2- 172.21.15.37:0/2400242815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52e4102ed0 0x7f52e4109760 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.410+0000 7f52e950a640 1 --2- 172.21.15.37:0/2400242815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e4102510 0x7f52e4102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.410+0000 7f52e950a640 1 --2- 172.21.15.37:0/2400242815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e4101320 0x7f52e4101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.410+0000 7f52e950a640 1 -- 172.21.15.37:0/2400242815 >> 172.21.15.37:0/2400242815 conn(0x7f52e40fc8b0 msgr2=0x7f52e40fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.411+0000 7f52e950a640 1 -- 172.21.15.37:0/2400242815 shutdown_connections 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.411+0000 7f52e950a640 1 -- 172.21.15.37:0/2400242815 wait complete. 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.411+0000 7f52e950a640 1 Processor -- start 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.411+0000 7f52e950a640 1 -- start start 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.411+0000 7f52e950a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e4101320 0x7f52e40750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.411+0000 7f52e950a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52e4102510 0x7f52e40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.411+0000 7f52e950a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e4102ed0 0x7f52e4073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.411+0000 7f52e950a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e410cfb0 con 0x7f52e4102510 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.411+0000 7f52e950a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e410ce30 con 0x7f52e4101320 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.411+0000 7f52e950a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e410d130 con 0x7f52e4102ed0 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52e4102510 0x7f52e40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e4101320 0x7f52e40750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52e4102510 0x7f52e40736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41118/0 (socket says 172.21.15.37:41118) 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e37fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e4102ed0 0x7f52e4073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e27fc640 1 -- 172.21.15.37:0/2235007788 learned_addr learned my addr 172.21.15.37:0/2235007788 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e4101320 0x7f52e40750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:39188/0 (socket says 172.21.15.37:39188) 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e2ffd640 1 -- 172.21.15.37:0/2235007788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e4102ed0 msgr2=0x7f52e4073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e2ffd640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e4102ed0 0x7f52e4073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e2ffd640 1 -- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52e4102510 msgr2=0x7f52e40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:52.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e2ffd640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52e4102510 0x7f52e40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:52.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e2ffd640 1 -- 172.21.15.37:0/2235007788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e40744b0 con 0x7f52e4101320 2024-03-26T06:22:52.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e37fe640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e4102ed0 0x7f52e4073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:22:52.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e2ffd640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e4101320 0x7f52e40750a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f52d400e9c0 tx=0x7f52d400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:52.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400ccc0 con 0x7f52e4101320 2024-03-26T06:22:52.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e950a640 1 -- 172.21.15.37:0/2235007788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e41a15d0 con 0x7f52e4101320 2024-03-26T06:22:52.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52d40035f0 con 0x7f52e4101320 2024-03-26T06:22:52.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400ce60 con 0x7f52e4101320 2024-03-26T06:22:52.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.412+0000 7f52e950a640 1 -- 172.21.15.37:0/2235007788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e41a1b20 con 0x7f52e4101320 2024-03-26T06:22:52.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.413+0000 7f52e950a640 1 -- 172.21.15.37:0/2235007788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e4108570 con 0x7f52e4101320 2024-03-26T06:22:52.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.414+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 19) v1 ==== 137307+0+0 (secure 0 0 0) 0x7f52d40037a0 con 0x7f52e4101320 2024-03-26T06:22:52.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.415+0000 7f52cbfff640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 0x7f52cc0a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:52.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.415+0000 7f52e27fc640 1 -- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 msgr2=0x7f52cc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/641545212 2024-03-26T06:22:52.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.415+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f52d40cf070 con 0x7f52e4101320 2024-03-26T06:22:52.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.415+0000 7f52e27fc640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 0x7f52cc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T06:22:52.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.418+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f52d409c0c0 con 0x7f52e4101320 2024-03-26T06:22:52.597 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:52 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:52.213+0000 7fe868bce200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T06:22:52.597 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:52 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:52.310+0000 7fe868bce200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T06:22:52.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.616+0000 7f52e27fc640 1 -- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 msgr2=0x7f52cc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/641545212 2024-03-26T06:22:52.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.616+0000 7f52e27fc640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 0x7f52cc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T06:22:52.626 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:52 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:52.245+0000 7fbc941de200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T06:22:52.626 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:52 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:52.345+0000 7fbc941de200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T06:22:52.626 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:52 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:52.188+0000 7f9dc7f34200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T06:22:52.626 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:52 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:52.280+0000 7f9dc7f34200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T06:22:52.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:52.627+0000 7f52e950a640 1 -- 172.21.15.37:0/2235007788 --> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f52e4061730 con 0x7f52cc0a28f0 2024-03-26T06:22:52.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:52 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2624227887' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-26T06:22:52.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:52 smithi149 ceph-mon[36681]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-26T06:22:52.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:52 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2017286357' entity='client.admin' 2024-03-26T06:22:52.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:52 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1385027085' entity='client.admin' 2024-03-26T06:22:52.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:52 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2624227887' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-26T06:22:52.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:52 smithi037 ceph-mon[31974]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-26T06:22:52.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:52 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2017286357' entity='client.admin' 2024-03-26T06:22:52.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:52 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1385027085' entity='client.admin' 2024-03-26T06:22:52.976 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:52 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2624227887' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-26T06:22:52.976 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:52 smithi066 ceph-mon[36504]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-26T06:22:52.976 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:52 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2017286357' entity='client.admin' 2024-03-26T06:22:52.976 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:52 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1385027085' entity='client.admin' 2024-03-26T06:22:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:53.017+0000 7f52e27fc640 1 -- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 msgr2=0x7f52cc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/641545212 2024-03-26T06:22:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:53.017+0000 7f52e27fc640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 0x7f52cc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T06:22:53.245 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:52 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:52.915+0000 7f9dc7f34200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T06:22:53.245 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:53.006+0000 7f9dc7f34200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T06:22:53.245 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:53.149+0000 7f9dc7f34200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T06:22:53.245 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:53.243+0000 7f9dc7f34200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T06:22:53.316 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:52 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:52.974+0000 7fe868bce200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T06:22:53.316 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:53.069+0000 7fe868bce200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T06:22:53.316 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:53.215+0000 7fe868bce200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T06:22:53.317 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:53.314+0000 7fe868bce200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T06:22:53.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:53.015+0000 7fbc941de200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T06:22:53.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:53.110+0000 7fbc941de200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T06:22:53.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:53.254+0000 7fbc941de200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T06:22:53.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:53.354+0000 7fbc941de200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T06:22:53.592 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:53.328+0000 7f9dc7f34200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T06:22:53.592 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:53.591+0000 7f9dc7f34200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T06:22:53.597 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:53.400+0000 7fe868bce200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T06:22:53.698 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:53.439+0000 7fbc941de200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T06:22:53.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:53.818+0000 7f52e27fc640 1 -- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 msgr2=0x7f52cc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/641545212 2024-03-26T06:22:53.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:53.818+0000 7f52e27fc640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 0x7f52cc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T06:22:53.880 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:53.699+0000 7f9dc7f34200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T06:22:53.880 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:53.785+0000 7f9dc7f34200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T06:22:53.880 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:53.879+0000 7f9dc7f34200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T06:22:53.975 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:53.673+0000 7fe868bce200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T06:22:53.976 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:53.787+0000 7fe868bce200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T06:22:53.976 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:53.876+0000 7fe868bce200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T06:22:53.976 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:53.974+0000 7fe868bce200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T06:22:54.011 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:53.710+0000 7fbc941de200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T06:22:54.012 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:53.823+0000 7fbc941de200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T06:22:54.012 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:53.912+0000 7fbc941de200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T06:22:54.012 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:54 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:54.010+0000 7fbc941de200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T06:22:54.160 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:54 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:54.058+0000 7f9dc7f34200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T06:22:54.347 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:54 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:54.156+0000 7fe868bce200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T06:22:54.448 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:54 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:54.190+0000 7fbc941de200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T06:22:55.410 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:55.024+0000 7f9dc7f34200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T06:22:55.410 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:55.153+0000 7f9dc7f34200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T06:22:55.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:55.419+0000 7f52e27fc640 1 -- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 msgr2=0x7f52cc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/641545212 2024-03-26T06:22:55.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:55.419+0000 7f52e27fc640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 0x7f52cc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T06:22:55.448 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:55 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:55.187+0000 7fbc941de200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T06:22:55.448 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:55 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:55.322+0000 7fbc941de200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T06:22:55.597 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:55.171+0000 7fe868bce200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T06:22:55.598 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:55.305+0000 7fe868bce200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T06:22:55.769 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:55.488+0000 7f9dc7f34200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T06:22:55.770 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:55.576+0000 7f9dc7f34200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T06:22:55.770 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:55.768+0000 7f9dc7f34200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:22:55.947 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:55.649+0000 7fe868bce200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T06:22:55.947 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:55.743+0000 7fe868bce200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T06:22:55.948 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:55 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:55.663+0000 7fbc941de200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T06:22:55.949 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:55 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:55.758+0000 7fbc941de200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T06:22:56.160 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:55.950+0000 7f9dc7f34200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T06:22:56.347 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:55.945+0000 7fe868bce200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:22:56.347 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:56 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:56.134+0000 7fe868bce200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T06:22:56.448 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:55 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:55.962+0000 7fbc941de200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:22:56.449 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:56 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:56.154+0000 7fbc941de200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T06:22:56.497 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:56 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:56.230+0000 7f9dc7f34200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:22:56.497 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:56 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:56.495+0000 7f9dc7f34200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T06:22:56.697 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:56 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:56.427+0000 7fe868bce200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:22:56.717 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:56 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:56.452+0000 7fbc941de200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T06:22:56.717 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:56 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:56.716+0000 7fbc941de200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T06:22:56.910 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:56 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:56.732+0000 7f9dc7f34200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T06:22:57.097 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:56 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:56.695+0000 7fe868bce200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T06:22:57.097 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:56 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:56.940+0000 7fe868bce200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T06:22:57.198 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:56 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:56.962+0000 7fbc941de200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T06:22:57.910 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:57 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:57.534+0000 7f9dc7f34200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T06:22:58.097 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:57 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:57.773+0000 7fe868bce200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T06:22:58.198 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:57 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:57.807+0000 7fbc941de200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T06:22:58.409 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:58 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:58.118+0000 7f9dc7f34200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T06:22:58.410 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:58 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:58.217+0000 7f9dc7f34200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T06:22:58.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:58.507+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 20) v1 ==== 137307+0+0 (secure 0 0 0) 0x7f52d4098030 con 0x7f52e4101320 2024-03-26T06:22:58.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:58.623+0000 7f52e27fc640 1 -- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 msgr2=0x7f52cc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/641545212 2024-03-26T06:22:58.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:58.623+0000 7f52e27fc640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 0x7f52cc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-26T06:22:58.697 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:58 smithi066 ceph-mon[36504]: Standby manager daemon z restarted 2024-03-26T06:22:58.697 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:58 smithi066 ceph-mon[36504]: Standby manager daemon z started 2024-03-26T06:22:58.697 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:58 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T06:22:58.697 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:58 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:22:58.697 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:58 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T06:22:58.697 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:58 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:22:58.697 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:58.368+0000 7fe868bce200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T06:22:58.697 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:58.468+0000 7fe868bce200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T06:22:58.697 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:22:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y[37929]: 2024-03-26T06:22:58.695+0000 7fe868bce200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T06:22:58.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-mon[31974]: Standby manager daemon z restarted 2024-03-26T06:22:58.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-mon[31974]: Standby manager daemon z started 2024-03-26T06:22:58.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T06:22:58.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:22:58.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T06:22:58.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:22:58.699 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.395+0000 7fbc941de200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T06:22:58.699 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.500+0000 7fbc941de200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T06:22:58.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:58.759+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 <== mon.2 v2:172.21.15.66:3300/0 8 ==== mgrmap(e 21) v1 ==== 136975+0+0 (secure 0 0 0) 0x7f52d40ccba0 con 0x7f52e4101320 2024-03-26T06:22:58.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:58.760+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 msgr2=0x7f52cc0a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T06:22:58.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:58.760+0000 7f52cbfff640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/641545212,v1:172.21.15.37:6801/641545212] conn(0x7f52cc0a28f0 0x7f52cc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:58.910 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:22:58 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z[38338]: 2024-03-26T06:22:58.432+0000 7f9dc7f34200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T06:22:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:58 smithi149 ceph-mon[36681]: Standby manager daemon z restarted 2024-03-26T06:22:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:58 smithi149 ceph-mon[36681]: Standby manager daemon z started 2024-03-26T06:22:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:58 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T06:22:58.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:58 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:22:58.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:58 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T06:22:58.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:58 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.149:0/1838552445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:22:58.990 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.730+0000 7fbc941de200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T06:22:59.449 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.989+0000 7fbbf6c43700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:22:59.449 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.989+0000 7fbbf6c43700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:22:59.449 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.989+0000 7fbbf6c43700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:22:59.449 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.989+0000 7fbbf6c43700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:22:59.449 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.989+0000 7fbbf6c43700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:22:59.449 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.990+0000 7fbbf3c3d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:22:59.449 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.990+0000 7fbbf3c3d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:22:59.449 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.990+0000 7fbbf3c3d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:22:59.450 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.990+0000 7fbbf3c3d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:22:59.450 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:22:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:22:58.990+0000 7fbbf3c3d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:22:59.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.761+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 <== mon.2 v2:172.21.15.66:3300/0 9 ==== mgrmap(e 22) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f52d4073f70 con 0x7f52e4101320 2024-03-26T06:22:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.762+0000 7f52cbfff640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f52cc034fb0 0x7f52cc035390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.762+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 --> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f52d400e1a0 con 0x7f52cc034fb0 2024-03-26T06:22:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.764+0000 7f52e27fc640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f52cc034fb0 0x7f52cc035390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:59.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.766+0000 7f52e27fc640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f52cc034fb0 0x7f52cc035390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f52e4074e10 tx=0x7f52d8005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:59.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.834+0000 7f52cbfff640 1 -- 172.21.15.37:0/2235007788 <== mgr.15411 v2:172.21.15.37:6800/3894397572 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f52d400e1a0 con 0x7f52cc034fb0 2024-03-26T06:22:59.835 INFO:teuthology.orchestra.run.smithi037.stdout:Schedule set for path / 2024-03-26T06:22:59.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.836+0000 7f52e950a640 1 -- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f52cc034fb0 msgr2=0x7f52cc035390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.836+0000 7f52e950a640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f52cc034fb0 0x7f52cc035390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f52e4074e10 tx=0x7f52d8005f90 comp rx=0 tx=0).stop 2024-03-26T06:22:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.836+0000 7f52e950a640 1 -- 172.21.15.37:0/2235007788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e4101320 msgr2=0x7f52e40750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.836+0000 7f52e950a640 1 --2- 172.21.15.37:0/2235007788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52e4101320 0x7f52e40750a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f52d400e9c0 tx=0x7f52d400ee80 comp rx=0 tx=0).stop 2024-03-26T06:22:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.836+0000 7f52e37fe640 1 -- 172.21.15.37:0/2235007788 reap_dead start 2024-03-26T06:22:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.836+0000 7f52e950a640 1 -- 172.21.15.37:0/2235007788 shutdown_connections 2024-03-26T06:22:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.836+0000 7f52e950a640 1 -- 172.21.15.37:0/2235007788 >> 172.21.15.37:0/2235007788 conn(0x7f52e40fc8b0 msgr2=0x7f52e4105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.836+0000 7f52e950a640 1 -- 172.21.15.37:0/2235007788 shutdown_connections 2024-03-26T06:22:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.837+0000 7f52e950a640 1 -- 172.21.15.37:0/2235007788 wait complete. 2024-03-26T06:22:59.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-03-26T06:22:59.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: Standby manager daemon y restarted 2024-03-26T06:22:59.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: Standby manager daemon y started 2024-03-26T06:22:59.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T06:22:59.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:22:59.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T06:22:59.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:22:59.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: Active manager daemon x restarted 2024-03-26T06:22:59.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: Activating manager daemon x 2024-03-26T06:22:59.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T06:22:59.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: mgrmap e21: x(active, starting, since 0.0109664s), standbys: z, y 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T06:22:59.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T06:22:59.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T06:22:59.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T06:22:59.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T06:22:59.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T06:22:59.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:22:59.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: Manager daemon x is now available 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:22:59 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: Standby manager daemon y restarted 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: Standby manager daemon y started 2024-03-26T06:22:59.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: Active manager daemon x restarted 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: Activating manager daemon x 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: mgrmap e21: x(active, starting, since 0.0109664s), standbys: z, y 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T06:22:59.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T06:22:59.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T06:22:59.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T06:22:59.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T06:22:59.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T06:22:59.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:22:59.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T06:22:59.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T06:22:59.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T06:22:59.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T06:22:59.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T06:22:59.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T06:22:59.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T06:22:59.856 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T06:22:59.856 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: Manager daemon x is now available 2024-03-26T06:22:59.856 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:22:59.856 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:22:59.856 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T06:22:59.856 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:22:59 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T06:22:59.857 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-26T06:22:59.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-03-26T06:22:59.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: Standby manager daemon y restarted 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: Standby manager daemon y started 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.? 172.21.15.66:0/172841724' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: Active manager daemon x restarted 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: Activating manager daemon x 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: mgrmap e21: x(active, starting, since 0.0109664s), standbys: z, y 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T06:22:59.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T06:22:59.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: Manager daemon x is now available 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:22:59.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:22:59.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T06:22:59.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:22:59 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T06:22:59.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.963+0000 7f33e4389640 1 -- 172.21.15.37:0/1923696958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc101400 msgr2=0x7f33dc101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.963+0000 7f33e4389640 1 --2- 172.21.15.37:0/1923696958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc101400 0x7f33dc101880 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009900 tx=0x7f33cc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:22:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.963+0000 7f33e4389640 1 -- 172.21.15.37:0/1923696958 shutdown_connections 2024-03-26T06:22:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.963+0000 7f33e4389640 1 --2- 172.21.15.37:0/1923696958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33dc101dc0 0x7f33dc108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.963+0000 7f33e4389640 1 --2- 172.21.15.37:0/1923696958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc101400 0x7f33dc101880 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.963+0000 7f33e4389640 1 --2- 172.21.15.37:0/1923696958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33dc100210 0x7f33dc100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.963+0000 7f33e4389640 1 -- 172.21.15.37:0/1923696958 >> 172.21.15.37:0/1923696958 conn(0x7f33dc0fb780 msgr2=0x7f33dc0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:22:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.963+0000 7f33e4389640 1 -- 172.21.15.37:0/1923696958 shutdown_connections 2024-03-26T06:22:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.963+0000 7f33e4389640 1 -- 172.21.15.37:0/1923696958 wait complete. 2024-03-26T06:22:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.963+0000 7f33e4389640 1 Processor -- start 2024-03-26T06:22:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.963+0000 7f33e4389640 1 -- start start 2024-03-26T06:22:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e4389640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33dc100210 0x7f33dc0694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e4389640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33dc101400 0x7f33dc19c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e4389640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc101dc0 0x7f33dc19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e4389640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc10bef0 con 0x7f33dc101dc0 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e4389640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc10bd70 con 0x7f33dc100210 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e4389640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc10c070 con 0x7f33dc101400 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e2926640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc101dc0 0x7f33dc19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e2926640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc101dc0 0x7f33dc19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44956/0 (socket says 172.21.15.37:44956) 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e2926640 1 -- 172.21.15.37:0/1255405322 learned_addr learned my addr 172.21.15.37:0/1255405322 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e2125640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33dc100210 0x7f33dc0694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e1924640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33dc101400 0x7f33dc19c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e2926640 1 -- 172.21.15.37:0/1255405322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33dc101400 msgr2=0x7f33dc19c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e2926640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33dc101400 0x7f33dc19c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e2926640 1 -- 172.21.15.37:0/1255405322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33dc100210 msgr2=0x7f33dc0694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:22:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e2926640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33dc100210 0x7f33dc0694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:22:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e2926640 1 -- 172.21.15.37:0/1255405322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc19f290 con 0x7f33dc101dc0 2024-03-26T06:22:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e2926640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc101dc0 0x7f33dc19e980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f33d400eaa0 tx=0x7f33d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33d37fe640 1 -- 172.21.15.37:0/1255405322 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d400cdb0 con 0x7f33dc101dc0 2024-03-26T06:22:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33d37fe640 1 -- 172.21.15.37:0/1255405322 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33d4005020 con 0x7f33dc101dc0 2024-03-26T06:22:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e4389640 1 -- 172.21.15.37:0/1255405322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc19f520 con 0x7f33dc101dc0 2024-03-26T06:22:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33d37fe640 1 -- 172.21.15.37:0/1255405322 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d40116b0 con 0x7f33dc101dc0 2024-03-26T06:22:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.964+0000 7f33e4389640 1 -- 172.21.15.37:0/1255405322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc19fa40 con 0x7f33dc101dc0 2024-03-26T06:22:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.965+0000 7f33e4389640 1 -- 172.21.15.37:0/1255405322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc1074d0 con 0x7f33dc101dc0 2024-03-26T06:22:59.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.966+0000 7f33d37fe640 1 -- 172.21.15.37:0/1255405322 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f33d4005560 con 0x7f33dc101dc0 2024-03-26T06:22:59.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.967+0000 7f33d37fe640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f33c80a2730 0x7f33c80a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:22:59.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.967+0000 7f33e2125640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f33c80a2730 0x7f33c80a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:22:59.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.967+0000 7f33d37fe640 1 -- 172.21.15.37:0/1255405322 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f33d40cea30 con 0x7f33dc101dc0 2024-03-26T06:22:59.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.968+0000 7f33e2125640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f33c80a2730 0x7f33c80a4bf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f33d8005fd0 tx=0x7f33d8005850 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:22:59.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:22:59.972+0000 7f33d37fe640 1 -- 172.21.15.37:0/1255405322 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f33d4016030 con 0x7f33dc101dc0 2024-03-26T06:23:00.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.187+0000 7f33e4389640 1 -- 172.21.15.37:0/1255405322 --> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f33dc061890 con 0x7f33c80a2730 2024-03-26T06:23:00.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.243+0000 7f33d37fe640 1 -- 172.21.15.37:0/1255405322 <== mgr.15411 v2:172.21.15.37:6800/3894397572 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f33dc061890 con 0x7f33c80a2730 2024-03-26T06:23:00.245 INFO:teuthology.orchestra.run.smithi037.stdout:Retention added to path / 2024-03-26T06:23:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.245+0000 7f33e4389640 1 -- 172.21.15.37:0/1255405322 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f33c80a2730 msgr2=0x7f33c80a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.245+0000 7f33e4389640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f33c80a2730 0x7f33c80a4bf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f33d8005fd0 tx=0x7f33d8005850 comp rx=0 tx=0).stop 2024-03-26T06:23:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.245+0000 7f33e4389640 1 -- 172.21.15.37:0/1255405322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc101dc0 msgr2=0x7f33dc19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.245+0000 7f33e4389640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc101dc0 0x7f33dc19e980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f33d400eaa0 tx=0x7f33d400ef60 comp rx=0 tx=0).stop 2024-03-26T06:23:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.245+0000 7f33e4389640 1 -- 172.21.15.37:0/1255405322 shutdown_connections 2024-03-26T06:23:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.245+0000 7f33e4389640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f33c80a2730 0x7f33c80a4bf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.245+0000 7f33e4389640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33dc101dc0 0x7f33dc19e980 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.245+0000 7f33e4389640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33dc101400 0x7f33dc19c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.245+0000 7f33e4389640 1 --2- 172.21.15.37:0/1255405322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33dc100210 0x7f33dc0694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.245+0000 7f33e4389640 1 -- 172.21.15.37:0/1255405322 >> 172.21.15.37:0/1255405322 conn(0x7f33dc0fb780 msgr2=0x7f33dc1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:00.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.246+0000 7f33e4389640 1 -- 172.21.15.37:0/1255405322 shutdown_connections 2024-03-26T06:23:00.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.246+0000 7f33e4389640 1 -- 172.21.15.37:0/1255405322 wait complete. 2024-03-26T06:23:00.258 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-26T06:23:00.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.361+0000 7f103de52640 1 -- 172.21.15.37:0/3044328333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1038100ee0 msgr2=0x7f1038101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.361+0000 7f103de52640 1 --2- 172.21.15.37:0/3044328333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1038100ee0 0x7f1038101360 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1030009ca0 tx=0x7f103002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 -- 172.21.15.37:0/3044328333 shutdown_connections 2024-03-26T06:23:00.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 --2- 172.21.15.37:0/3044328333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1038100ee0 0x7f1038101360 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 --2- 172.21.15.37:0/3044328333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1038100540 0x7f10381009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 --2- 172.21.15.37:0/3044328333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10380ffbe0 0x7f1038100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 -- 172.21.15.37:0/3044328333 >> 172.21.15.37:0/3044328333 conn(0x7f10380fb780 msgr2=0x7f10380fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:00.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 -- 172.21.15.37:0/3044328333 shutdown_connections 2024-03-26T06:23:00.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 -- 172.21.15.37:0/3044328333 wait complete. 2024-03-26T06:23:00.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 Processor -- start 2024-03-26T06:23:00.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 -- start start 2024-03-26T06:23:00.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10380ffbe0 0x7f1038199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1038100540 0x7f1038199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1038100ee0 0x7f10381a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103810f1b0 con 0x7f1038100ee0 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103810f030 con 0x7f1038100540 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.362+0000 7f103de52640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103810f330 con 0x7f10380ffbe0 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1037fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1038100ee0 0x7f10381a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1037fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1038100ee0 0x7f10381a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44976/0 (socket says 172.21.15.37:44976) 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1037fff640 1 -- 172.21.15.37:0/2067558800 learned_addr learned my addr 172.21.15.37:0/2067558800 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1036ffd640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1038100540 0x7f1038199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f10377fe640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10380ffbe0 0x7f1038199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1037fff640 1 -- 172.21.15.37:0/2067558800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10380ffbe0 msgr2=0x7f1038199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1037fff640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10380ffbe0 0x7f1038199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1037fff640 1 -- 172.21.15.37:0/2067558800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1038100540 msgr2=0x7f1038199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1037fff640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1038100540 0x7f1038199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1037fff640 1 -- 172.21.15.37:0/2067558800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10381a1300 con 0x7f1038100ee0 2024-03-26T06:23:00.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1037fff640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1038100ee0 0x7f10381a0c90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1030037040 tx=0x7f1030032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:00.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1034ff9640 1 -- 172.21.15.37:0/2067558800 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030041020 con 0x7f1038100ee0 2024-03-26T06:23:00.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1034ff9640 1 -- 172.21.15.37:0/2067558800 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1030034cd0 con 0x7f1038100ee0 2024-03-26T06:23:00.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f103de52640 1 -- 172.21.15.37:0/2067558800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10381a1590 con 0x7f1038100ee0 2024-03-26T06:23:00.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f1034ff9640 1 -- 172.21.15.37:0/2067558800 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030039040 con 0x7f1038100ee0 2024-03-26T06:23:00.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.363+0000 7f103de52640 1 -- 172.21.15.37:0/2067558800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10381a1ad0 con 0x7f1038100ee0 2024-03-26T06:23:00.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.364+0000 7f103de52640 1 -- 172.21.15.37:0/2067558800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103810a930 con 0x7f1038100ee0 2024-03-26T06:23:00.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.365+0000 7f1034ff9640 1 -- 172.21.15.37:0/2067558800 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f1030044020 con 0x7f1038100ee0 2024-03-26T06:23:00.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.366+0000 7f1034ff9640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f10200a2730 0x7f10200a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:00.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.366+0000 7f10377fe640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f10200a2730 0x7f10200a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:00.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.366+0000 7f1034ff9640 1 -- 172.21.15.37:0/2067558800 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f10300f0520 con 0x7f1038100ee0 2024-03-26T06:23:00.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.366+0000 7f10377fe640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f10200a2730 0x7f10200a4bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1028009bd0 tx=0x7f1028005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.370+0000 7f1034ff9640 1 -- 172.21.15.37:0/2067558800 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f103004b020 con 0x7f1038100ee0 2024-03-26T06:23:00.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:00 smithi066 ceph-mon[36504]: [26/Mar/2024:06:22:59] ENGINE Bus STARTING 2024-03-26T06:23:00.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:00 smithi066 ceph-mon[36504]: [26/Mar/2024:06:22:59] ENGINE Serving on http://172.21.15.37:8765 2024-03-26T06:23:00.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:00 smithi066 ceph-mon[36504]: mgrmap e22: x(active, since 1.0192s), standbys: z, y 2024-03-26T06:23:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.607+0000 7f103de52640 1 -- 172.21.15.37:0/2067558800 --> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f1038061730 con 0x7f10200a2730 2024-03-26T06:23:00.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.615+0000 7f1034ff9640 1 -- 172.21.15.37:0/2067558800 <== mgr.15411 v2:172.21.15.37:6800/3894397572 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f1038061730 con 0x7f10200a2730 2024-03-26T06:23:00.616 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-26T00:00:00", "created": "2024-03-26T06:22:59", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-26T06:23:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 -- 172.21.15.37:0/2067558800 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f10200a2730 msgr2=0x7f10200a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f10200a2730 0x7f10200a4bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1028009bd0 tx=0x7f1028005f90 comp rx=0 tx=0).stop 2024-03-26T06:23:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 -- 172.21.15.37:0/2067558800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1038100ee0 msgr2=0x7f10381a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1038100ee0 0x7f10381a0c90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1030037040 tx=0x7f1030032f40 comp rx=0 tx=0).stop 2024-03-26T06:23:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 -- 172.21.15.37:0/2067558800 shutdown_connections 2024-03-26T06:23:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f10200a2730 0x7f10200a4bf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1038100ee0 0x7f10381a0c90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1038100540 0x7f1038199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 --2- 172.21.15.37:0/2067558800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10380ffbe0 0x7f1038199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 -- 172.21.15.37:0/2067558800 >> 172.21.15.37:0/2067558800 conn(0x7f10380fb780 msgr2=0x7f10380fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 -- 172.21.15.37:0/2067558800 shutdown_connections 2024-03-26T06:23:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.617+0000 7f103de52640 1 -- 172.21.15.37:0/2067558800 wait complete. 2024-03-26T06:23:00.630 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-26T06:23:00.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.742+0000 7f31648ab640 1 -- 172.21.15.37:0/3497634136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3160100540 msgr2=0x7f31601009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.742+0000 7f31648ab640 1 --2- 172.21.15.37:0/3497634136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3160100540 0x7f31601009a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3154009900 tx=0x7f315402bc30 comp rx=0 tx=0).stop 2024-03-26T06:23:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.742+0000 7f31648ab640 1 -- 172.21.15.37:0/3497634136 shutdown_connections 2024-03-26T06:23:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.742+0000 7f31648ab640 1 --2- 172.21.15.37:0/3497634136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3160100ee0 0x7f3160101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.742+0000 7f31648ab640 1 --2- 172.21.15.37:0/3497634136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3160100540 0x7f31601009a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.742+0000 7f31648ab640 1 --2- 172.21.15.37:0/3497634136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31600ffbe0 0x7f3160100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.742+0000 7f31648ab640 1 -- 172.21.15.37:0/3497634136 >> 172.21.15.37:0/3497634136 conn(0x7f31600fb780 msgr2=0x7f31600fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.742+0000 7f31648ab640 1 -- 172.21.15.37:0/3497634136 shutdown_connections 2024-03-26T06:23:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.742+0000 7f31648ab640 1 -- 172.21.15.37:0/3497634136 wait complete. 2024-03-26T06:23:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f31648ab640 1 Processor -- start 2024-03-26T06:23:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f31648ab640 1 -- start start 2024-03-26T06:23:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f31648ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31600ffbe0 0x7f31601997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f31648ab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3160100540 0x7f3160199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f31648ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3160100ee0 0x7f31601a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f31648ab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316010f200 con 0x7f3160100ee0 2024-03-26T06:23:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f31648ab640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316010f080 con 0x7f31600ffbe0 2024-03-26T06:23:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f31648ab640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316010f380 con 0x7f3160100540 2024-03-26T06:23:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f315ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3160100ee0 0x7f31601a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f315ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3160100ee0 0x7f31601a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45000/0 (socket says 172.21.15.37:45000) 2024-03-26T06:23:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f315ed9d640 1 -- 172.21.15.37:0/3739662442 learned_addr learned my addr 172.21.15.37:0/3739662442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f315e59c640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31600ffbe0 0x7f31601997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.743+0000 7f315dd9b640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3160100540 0x7f3160199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f315e59c640 1 -- 172.21.15.37:0/3739662442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3160100540 msgr2=0x7f3160199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f315e59c640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3160100540 0x7f3160199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f315e59c640 1 -- 172.21.15.37:0/3739662442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3160100ee0 msgr2=0x7f31601a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f315e59c640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3160100ee0 0x7f31601a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f315e59c640 1 -- 172.21.15.37:0/3739662442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31601a13c0 con 0x7f31600ffbe0 2024-03-26T06:23:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f315ed9d640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3160100ee0 0x7f31601a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:23:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f315e59c640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31600ffbe0 0x7f31601997a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f315000e9f0 tx=0x7f315000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f314f7fe640 1 -- 172.21.15.37:0/3739662442 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315000ccc0 con 0x7f31600ffbe0 2024-03-26T06:23:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f314f7fe640 1 -- 172.21.15.37:0/3739662442 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3150010910 con 0x7f31600ffbe0 2024-03-26T06:23:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f31648ab640 1 -- 172.21.15.37:0/3739662442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31601a16b0 con 0x7f31600ffbe0 2024-03-26T06:23:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f314f7fe640 1 -- 172.21.15.37:0/3739662442 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150018960 con 0x7f31600ffbe0 2024-03-26T06:23:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.744+0000 7f31648ab640 1 -- 172.21.15.37:0/3739662442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31601a1c20 con 0x7f31600ffbe0 2024-03-26T06:23:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.745+0000 7f31648ab640 1 -- 172.21.15.37:0/3739662442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316010a930 con 0x7f31600ffbe0 2024-03-26T06:23:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.747+0000 7f314f7fe640 1 -- 172.21.15.37:0/3739662442 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 22) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f3150010430 con 0x7f31600ffbe0 2024-03-26T06:23:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.748+0000 7f314f7fe640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31480a2730 0x7f31480a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:00.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.748+0000 7f315dd9b640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31480a2730 0x7f31480a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:00.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.748+0000 7f314f7fe640 1 -- 172.21.15.37:0/3739662442 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f31500cea70 con 0x7f31600ffbe0 2024-03-26T06:23:00.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.748+0000 7f315dd9b640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31480a2730 0x7f31480a4bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f315402c240 tx=0x7f315402df90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:00.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.756+0000 7f314f7fe640 1 -- 172.21.15.37:0/3739662442 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f315009b6c0 con 0x7f31600ffbe0 2024-03-26T06:23:00.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:00 smithi149 ceph-mon[36681]: [26/Mar/2024:06:22:59] ENGINE Bus STARTING 2024-03-26T06:23:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:00 smithi149 ceph-mon[36681]: [26/Mar/2024:06:22:59] ENGINE Serving on http://172.21.15.37:8765 2024-03-26T06:23:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:00 smithi149 ceph-mon[36681]: mgrmap e22: x(active, since 1.0192s), standbys: z, y 2024-03-26T06:23:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:00 smithi037 ceph-mon[31974]: [26/Mar/2024:06:22:59] ENGINE Bus STARTING 2024-03-26T06:23:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:00 smithi037 ceph-mon[31974]: [26/Mar/2024:06:22:59] ENGINE Serving on http://172.21.15.37:8765 2024-03-26T06:23:00.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:00 smithi037 ceph-mon[31974]: mgrmap e22: x(active, since 1.0192s), standbys: z, y 2024-03-26T06:23:00.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.975+0000 7f31648ab640 1 -- 172.21.15.37:0/3739662442 --> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3160061730 con 0x7f31480a2730 2024-03-26T06:23:00.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.982+0000 7f314f7fe640 1 -- 172.21.15.37:0/3739662442 <== mgr.15411 v2:172.21.15.37:6800/3894397572 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f3160061730 con 0x7f31480a2730 2024-03-26T06:23:00.983 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-03-26T06:23:00.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 -- 172.21.15.37:0/3739662442 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31480a2730 msgr2=0x7f31480a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31480a2730 0x7f31480a4bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f315402c240 tx=0x7f315402df90 comp rx=0 tx=0).stop 2024-03-26T06:23:00.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 -- 172.21.15.37:0/3739662442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31600ffbe0 msgr2=0x7f31601997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:00.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31600ffbe0 0x7f31601997a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f315000e9f0 tx=0x7f315000eeb0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 -- 172.21.15.37:0/3739662442 shutdown_connections 2024-03-26T06:23:00.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31480a2730 0x7f31480a4bf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3160100ee0 0x7f31601a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3160100540 0x7f3160199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 --2- 172.21.15.37:0/3739662442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31600ffbe0 0x7f31601997a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:00.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 -- 172.21.15.37:0/3739662442 >> 172.21.15.37:0/3739662442 conn(0x7f31600fb780 msgr2=0x7f31600fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:00.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 -- 172.21.15.37:0/3739662442 shutdown_connections 2024-03-26T06:23:00.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:23:00.984+0000 7f31648ab640 1 -- 172.21.15.37:0/3739662442 wait complete. 2024-03-26T06:23:00.997 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-26T06:23:01.027 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-26T06:23:01.059 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-26T06:23:01.147 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-26T06:23:01.147 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-26T06:23:01.150 INFO:teuthology.run_tasks:Running task workunit... 2024-03-26T06:23:01.177 INFO:tasks.workunit:Pulling workunits from ref 7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:23:01.177 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-26T06:23:01.177 INFO:tasks.workunit:timeout=3h 2024-03-26T06:23:01.178 INFO:tasks.workunit:cleanup=True 2024-03-26T06:23:01.178 DEBUG:teuthology.orchestra.run.smithi037:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-26T06:23:01.193 INFO:teuthology.orchestra.run.smithi037.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-26T06:23:01.194 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-26T06:23:01.194 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-26T06:23:01.194 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T06:23:01.194 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-26T06:23:01.194 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-26 06:21:00.051369758 +0000 2024-03-26T06:23:01.194 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-26 06:21:00.051369758 +0000 2024-03-26T06:23:01.194 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-26 06:22:48.792771092 +0000 2024-03-26T06:23:01.194 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-26 06:21:00.051369758 +0000 2024-03-26T06:23:01.194 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-26T06:23:01.195 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-26T06:23:01.272 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7e98f5f6b535defc7fa715b456904ffc0500aa9f 2024-03-26T06:23:01.328 INFO:tasks.workunit.client.0.smithi037.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-26T06:23:01.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:01.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:01.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: [26/Mar/2024:06:22:59] ENGINE Serving on https://172.21.15.37:7150 2024-03-26T06:23:01.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: [26/Mar/2024:06:22:59] ENGINE Bus STARTED 2024-03-26T06:23:01.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:01.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:01.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:01 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:01.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:01.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: [26/Mar/2024:06:22:59] ENGINE Serving on https://172.21.15.37:7150 2024-03-26T06:23:01.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: [26/Mar/2024:06:22:59] ENGINE Bus STARTED 2024-03-26T06:23:01.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:01.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:01.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:01 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:01.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:01.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: [26/Mar/2024:06:22:59] ENGINE Serving on https://172.21.15.37:7150 2024-03-26T06:23:01.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: [26/Mar/2024:06:22:59] ENGINE Bus STARTED 2024-03-26T06:23:01.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:01.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:01.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:01.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:01 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-03-26T06:23:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-03-26T06:23:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:23:02.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:02 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:23:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:02 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:23:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='client.25088 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:23:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-03-26T06:23:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-03-26T06:23:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:23:02.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:02 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:23:03.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Unable to set osd_memory_target on smithi037 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi066 to 174.1M 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Unable to set osd_memory_target on smithi066 to 182611251: error parsing value: Value '182611251' is below minimum 939524096 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Updating smithi066:/etc/ceph/ceph.conf 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Updating smithi037:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Updating smithi066:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Updating smithi149:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:23:03.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:23:03.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:23:03.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:23:03.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:23:03.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-03-26T06:23:03.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Unable to set osd_memory_target on smithi037 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-03-26T06:23:03.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi066 to 174.1M 2024-03-26T06:23:03.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Unable to set osd_memory_target on smithi066 to 182611251: error parsing value: Value '182611251' is below minimum 939524096 2024-03-26T06:23:03.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-26T06:23:03.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Updating smithi066:/etc/ceph/ceph.conf 2024-03-26T06:23:03.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Updating smithi037:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Updating smithi066:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Updating smithi149:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:23:03.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:23:03.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:23:03.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-03-26T06:23:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Unable to set osd_memory_target on smithi037 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-03-26T06:23:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi066 to 174.1M 2024-03-26T06:23:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Unable to set osd_memory_target on smithi066 to 182611251: error parsing value: Value '182611251' is below minimum 939524096 2024-03-26T06:23:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-26T06:23:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Updating smithi066:/etc/ceph/ceph.conf 2024-03-26T06:23:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-26T06:23:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Updating smithi037:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:23:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Updating smithi066:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:23:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Updating smithi149:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T06:23:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:23:03.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:23:04.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:04 smithi066 ceph-mon[36504]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:04.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:04 smithi066 ceph-mon[36504]: Updating smithi066:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:23:04.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:04 smithi066 ceph-mon[36504]: Updating smithi037:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:23:04.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:04 smithi066 ceph-mon[36504]: Updating smithi149:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:23:04.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:04 smithi066 ceph-mon[36504]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-26T06:23:04.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:04 smithi149 ceph-mon[36681]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:04.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:04 smithi149 ceph-mon[36681]: Updating smithi066:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:23:04.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:04 smithi149 ceph-mon[36681]: Updating smithi037:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:23:04.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:04 smithi149 ceph-mon[36681]: Updating smithi149:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:23:04.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:04 smithi149 ceph-mon[36681]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-26T06:23:04.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:04 smithi037 ceph-mon[31974]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:04.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:04 smithi037 ceph-mon[31974]: Updating smithi066:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:23:04.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:04 smithi037 ceph-mon[31974]: Updating smithi037:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:23:04.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:04 smithi037 ceph-mon[31974]: Updating smithi149:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T06:23:04.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:04 smithi037 ceph-mon[31974]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-26T06:23:06.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:06 smithi066 ceph-mon[36504]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:06.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:06 smithi149 ceph-mon[36681]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:06.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:06 smithi037 ceph-mon[31974]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T06:23:08.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:08 smithi066 ceph-mon[36504]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-26T06:23:08.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:08 smithi149 ceph-mon[36681]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-26T06:23:08.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:08 smithi037 ceph-mon[31974]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-26T06:23:10.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:10 smithi066 ceph-mon[36504]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-26T06:23:10.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:10 smithi149 ceph-mon[36681]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-26T06:23:10.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:10 smithi037 ceph-mon[31974]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-26T06:23:12.597 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:12 smithi066 ceph-mon[36504]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-26T06:23:12.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:12 smithi149 ceph-mon[36681]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-26T06:23:12.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:12 smithi037 ceph-mon[31974]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-26T06:23:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.285+0000 7f808f0b2640 1 -- 172.21.15.37:0/388056654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f808810b940 msgr2=0x7f808810ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.285+0000 7f808f0b2640 1 --2- 172.21.15.37:0/388056654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f808810b940 0x7f808810ddb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8080009ca0 tx=0x7f808002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.285+0000 7f808f0b2640 1 -- 172.21.15.37:0/388056654 shutdown_connections 2024-03-26T06:23:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.285+0000 7f808f0b2640 1 --2- 172.21.15.37:0/388056654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f808810b940 0x7f808810ddb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.285+0000 7f808f0b2640 1 --2- 172.21.15.37:0/388056654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8088102df0 0x7f808810b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.285+0000 7f808f0b2640 1 --2- 172.21.15.37:0/388056654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80881024d0 0x7f80881028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.285+0000 7f808f0b2640 1 -- 172.21.15.37:0/388056654 >> 172.21.15.37:0/388056654 conn(0x7f8088077320 msgr2=0x7f8088077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.285+0000 7f808f0b2640 1 -- 172.21.15.37:0/388056654 shutdown_connections 2024-03-26T06:23:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.285+0000 7f808f0b2640 1 -- 172.21.15.37:0/388056654 wait complete. 2024-03-26T06:23:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808f0b2640 1 Processor -- start 2024-03-26T06:23:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808f0b2640 1 -- start start 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808f0b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80881024d0 0x7f8088100120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808f0b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8088102df0 0x7f8088100660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808f0b2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f808810b940 0x7f8088100ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808f0b2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8088111700 con 0x7f80881024d0 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808f0b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8088111580 con 0x7f8088102df0 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808f0b2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8088111880 con 0x7f808810b940 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808ce4e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80881024d0 0x7f8088100120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808ce4e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80881024d0 0x7f8088100120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44544/0 (socket says 172.21.15.37:44544) 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808ce4e640 1 -- 172.21.15.37:0/1592394987 learned_addr learned my addr 172.21.15.37:0/1592394987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808d64f640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f808810b940 0x7f8088100ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f8087fff640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8088102df0 0x7f8088100660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808ce4e640 1 -- 172.21.15.37:0/1592394987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f808810b940 msgr2=0x7f8088100ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808ce4e640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f808810b940 0x7f8088100ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808ce4e640 1 -- 172.21.15.37:0/1592394987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8088102df0 msgr2=0x7f8088100660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808ce4e640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8088102df0 0x7f8088100660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.286+0000 7f808ce4e640 1 -- 172.21.15.37:0/1592394987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80881a5f20 con 0x7f80881024d0 2024-03-26T06:23:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.287+0000 7f808ce4e640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80881024d0 0x7f8088100120 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f807c00c970 tx=0x7f807c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.287+0000 7f8085ffb640 1 -- 172.21.15.37:0/1592394987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c007bb0 con 0x7f80881024d0 2024-03-26T06:23:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.287+0000 7f8085ffb640 1 -- 172.21.15.37:0/1592394987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f807c007d50 con 0x7f80881024d0 2024-03-26T06:23:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.287+0000 7f808f0b2640 1 -- 172.21.15.37:0/1592394987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80881a6210 con 0x7f80881024d0 2024-03-26T06:23:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.287+0000 7f8085ffb640 1 -- 172.21.15.37:0/1592394987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00f450 con 0x7f80881024d0 2024-03-26T06:23:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.287+0000 7f808f0b2640 1 -- 172.21.15.37:0/1592394987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80881a6760 con 0x7f80881024d0 2024-03-26T06:23:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.289+0000 7f8085ffb640 1 -- 172.21.15.37:0/1592394987 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f807c004f40 con 0x7f80881024d0 2024-03-26T06:23:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.289+0000 7f8091d75740 1 Processor -- start 2024-03-26T06:23:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.289+0000 7f8085ffb640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f80780a2b30 0x7f80780a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f8091d75740 1 -- start start 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808d64f640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f80780a2b30 0x7f80780a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f8085ffb640 1 -- 172.21.15.37:0/1592394987 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f807c0cec00 con 0x7f80881024d0 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f8091d75740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c48a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f8091d75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c48fd0 0x563a84c49450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f8091d75740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c52550 0x563a84c54940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f8091d75740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a84bad5f0 con 0x563a84c49d40 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f8091d75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a84b56930 con 0x563a84c48fd0 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f8091d75740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a84ba9690 con 0x563a84c52550 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808ce4e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c48a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808ce4e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c48a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44550/0 (socket says 172.21.15.37:44550) 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808ce4e640 1 -- 172.21.15.37:0/2509545167 learned_addr learned my addr 172.21.15.37:0/2509545167 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808d64f640 1 --2- 172.21.15.37:0/2509545167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c52550 0x563a84c54940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f8087fff640 1 --2- 172.21.15.37:0/2509545167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c48fd0 0x563a84c49450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808ce4e640 1 -- 172.21.15.37:0/2509545167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c52550 msgr2=0x563a84c54940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808ce4e640 1 --2- 172.21.15.37:0/2509545167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c52550 0x563a84c54940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808ce4e640 1 -- 172.21.15.37:0/2509545167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c48fd0 msgr2=0x563a84c49450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808ce4e640 1 --2- 172.21.15.37:0/2509545167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c48fd0 0x563a84c49450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808ce4e640 1 -- 172.21.15.37:0/2509545167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a848c2210 con 0x563a84c49d40 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f8087fff640 1 --2- 172.21.15.37:0/2509545167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c48fd0 0x563a84c49450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808d64f640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f80780a2b30 0x7f80780a4ff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f80800098d0 tx=0x7f8080032480 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.290+0000 7f808ce4e640 1 --2- 172.21.15.37:0/2509545167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c48a90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f807c0cc350 tx=0x7f807c09aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be98f6255e4a152 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8072ffd640 1 -- 172.21.15.37:0/2509545167 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c01d030 con 0x563a84c49d40 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8072ffd640 1 -- 172.21.15.37:0/2509545167 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f807c0a38c0 con 0x563a84c49d40 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8072ffd640 1 -- 172.21.15.37:0/2509545167 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c005230 con 0x563a84c49d40 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8091d75740 1 -- 172.21.15.37:0/2509545167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 msgr2=0x563a84c48a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8091d75740 1 --2- 172.21.15.37:0/2509545167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c48a90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f807c0cc350 tx=0x7f807c09aba0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8091d75740 1 -- 172.21.15.37:0/2509545167 shutdown_connections 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8091d75740 1 --2- 172.21.15.37:0/2509545167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c52550 0x563a84c54940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8091d75740 1 --2- 172.21.15.37:0/2509545167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c48fd0 0x563a84c49450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8091d75740 1 --2- 172.21.15.37:0/2509545167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c48a90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8091d75740 1 -- 172.21.15.37:0/2509545167 >> 172.21.15.37:0/2509545167 conn(0x563a84c105b0 msgr2=0x563a84c43cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8091d75740 1 -- 172.21.15.37:0/2509545167 shutdown_connections 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.291+0000 7f8091d75740 1 -- 172.21.15.37:0/2509545167 wait complete. 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8091d75740 1 -- start start 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8091d75740 1 Processor -- start 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8091d75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c48fd0 0x563a84c4f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8091d75740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c4fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8091d75740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c52550 0x563a84c500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8091d75740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a84c55220 con 0x563a84c49d40 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8091d75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a84b69d90 con 0x563a84c48fd0 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8091d75740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a84c553a0 con 0x563a84c52550 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8091d75740 10 client.0 _ll_register_callbacks cb 0x563a848c1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8087fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c4fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8087fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c4fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44560/0 (socket says 172.21.15.37:44560) 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f8087fff640 1 -- 172.21.15.37:0/2448010327 learned_addr learned my addr 172.21.15.37:0/2448010327 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f808ce4e640 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c48fd0 0x563a84c4f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.292+0000 7f808d64f640 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c52550 0x563a84c500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8087fff640 1 -- 172.21.15.37:0/2448010327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c52550 msgr2=0x563a84c500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8087fff640 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c52550 0x563a84c500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8087fff640 1 -- 172.21.15.37:0/2448010327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c48fd0 msgr2=0x563a84c4f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8087fff640 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c48fd0 0x563a84c4f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8087fff640 1 -- 172.21.15.37:0/2448010327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a8498e620 con 0x563a84c49d40 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8087fff640 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c4fb60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8074011720 tx=0x7f8074011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a45f8d3468f38c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8067fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8067fff640 1 -- 172.21.15.37:0/2448010327 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807401f060 con 0x563a84c49d40 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8067fff640 1 -- 172.21.15.37:0/2448010327 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f807400fb40 con 0x563a84c49d40 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8091d75740 1 -- 172.21.15.37:0/2448010327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f807c0a38c0 con 0x563a84c49d40 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8067fff640 1 -- 172.21.15.37:0/2448010327 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f807400fd40 con 0x563a84c49d40 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8091d75740 10 client.15501 fetch_fsmap learned FSMap version 27 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8091d75740 1 -- 172.21.15.37:0/2448010327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563a84c61560 con 0x563a84c49d40 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8067fff640 1 -- 172.21.15.37:0/2448010327 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f807c0a38c0 con 0x563a84c49d40 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8067fff640 1 -- 172.21.15.37:0/2448010327 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f80740125f0 con 0x563a84c49d40 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8091d75740 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:23:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8091d75740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8091d75740 20 client.15501 populate_metadata read hostname 'smithi037' 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8091d75740 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563a84c62a20 0x563a84c64e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8091d75740 4 client.15501 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.293+0000 7f8091d75740 1 -- 172.21.15.37:0/2448010327 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a84bbab30 con 0x563a84c62a20 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.294+0000 7f808ce4e640 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563a84c62a20 0x563a84c64e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.294+0000 7f8067fff640 10 client.15501 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.294+0000 7f808ce4e640 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563a84c62a20 0x563a84c64e10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f613c2fcbe81a2af server_cookie=bd0010ccaaa09151 in_seq=0 out_seq=0 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8067fff640 1 -- 172.21.15.37:0/2448010327 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a84bbab30 con 0x563a84c62a20 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8067fff640 10 client.15501 handle_command_reply: tid=0 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 1 client.15501 shutdown 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 1 -- 172.21.15.37:0/2448010327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 msgr2=0x563a84c4fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c4fb60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8074011720 tx=0x7f8074011be0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 1 -- 172.21.15.37:0/2448010327 shutdown_connections 2024-03-26T06:23:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563a84c62a20 0x563a84c64e10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c52550 0x563a84c500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c4fb60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8067fff640 0 client.15501 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 1 --2- 172.21.15.37:0/2448010327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c48fd0 0x563a84c4f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 1 -- 172.21.15.37:0/2448010327 >> 172.21.15.37:0/2448010327 conn(0x563a84c105b0 msgr2=0x563a84c45a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 1 -- 172.21.15.37:0/2448010327 shutdown_connections 2024-03-26T06:23:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 1 -- 172.21.15.37:0/2448010327 wait complete. 2024-03-26T06:23:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.295+0000 7f8091d75740 20 client.15501 trim_cache size 0 max 16384 2024-03-26T06:23:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.308+0000 7f8091d75740 1 Processor -- start 2024-03-26T06:23:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.308+0000 7f8091d75740 1 -- start start 2024-03-26T06:23:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.308+0000 7f8091d75740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c48fd0 0x563a84c51d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.308+0000 7f8091d75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c49d40 0x563a84c59610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.308+0000 7f8091d75740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c52550 0x563a84c59b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.308+0000 7f8091d75740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80881aedd0 con 0x563a84c52550 2024-03-26T06:23:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.308+0000 7f8091d75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80881b0750 con 0x563a84c49d40 2024-03-26T06:23:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.308+0000 7f8091d75740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80881af430 con 0x563a84c48fd0 2024-03-26T06:23:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f808d64f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c52550 0x563a84c59b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f808d64f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c52550 0x563a84c59b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44576/0 (socket says 172.21.15.37:44576) 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f808d64f640 1 -- 172.21.15.37:0/1150028642 learned_addr learned my addr 172.21.15.37:0/1150028642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f808ce4e640 1 --2- 172.21.15.37:0/1150028642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c48fd0 0x563a84c51d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f8087fff640 1 --2- 172.21.15.37:0/1150028642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c49d40 0x563a84c59610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f808d64f640 1 -- 172.21.15.37:0/1150028642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c48fd0 msgr2=0x563a84c51d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f808d64f640 1 --2- 172.21.15.37:0/1150028642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c48fd0 0x563a84c51d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f808d64f640 1 -- 172.21.15.37:0/1150028642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c49d40 msgr2=0x563a84c59610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f808d64f640 1 --2- 172.21.15.37:0/1150028642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c49d40 0x563a84c59610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f808d64f640 1 -- 172.21.15.37:0/1150028642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a84c65180 con 0x563a84c52550 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f8087fff640 1 --2- 172.21.15.37:0/1150028642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c49d40 0x563a84c59610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f808ce4e640 1 --2- 172.21.15.37:0/1150028642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c48fd0 0x563a84c51d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f808d64f640 1 --2- 172.21.15.37:0/1150028642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c52550 0x563a84c59b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8080009ca0 tx=0x7f8080040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e322336da837a145 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f80727fc640 1 -- 172.21.15.37:0/1150028642 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080050070 con 0x563a84c52550 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f80727fc640 1 -- 172.21.15.37:0/1150028642 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8080003b70 con 0x563a84c52550 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.309+0000 7f80727fc640 1 -- 172.21.15.37:0/1150028642 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080039040 con 0x563a84c52550 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.310+0000 7f8091d75740 1 -- 172.21.15.37:0/1150028642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c52550 msgr2=0x563a84c59b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.310+0000 7f8091d75740 1 --2- 172.21.15.37:0/1150028642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c52550 0x563a84c59b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8080009ca0 tx=0x7f8080040000 comp rx=0 tx=0).stop 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.310+0000 7f8091d75740 1 -- 172.21.15.37:0/1150028642 shutdown_connections 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.310+0000 7f8091d75740 1 --2- 172.21.15.37:0/1150028642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c52550 0x563a84c59b50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.310+0000 7f8091d75740 1 --2- 172.21.15.37:0/1150028642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c49d40 0x563a84c59610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.310+0000 7f8091d75740 1 --2- 172.21.15.37:0/1150028642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c48fd0 0x563a84c51d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.310+0000 7f8091d75740 1 -- 172.21.15.37:0/1150028642 >> 172.21.15.37:0/1150028642 conn(0x563a84c105b0 msgr2=0x563a84c4def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.310+0000 7f8091d75740 1 -- 172.21.15.37:0/1150028642 shutdown_connections 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.310+0000 7f8091d75740 1 -- 172.21.15.37:0/1150028642 wait complete. 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.310+0000 7f8091d75740 1 -- start start 2024-03-26T06:23:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.310+0000 7f8091d75740 1 Processor -- start 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8091d75740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c48fd0 0x563a84c56400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8091d75740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c56940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8091d75740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c52550 0x563a84cb8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8091d75740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80881b00f0 con 0x563a84c49d40 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8091d75740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80881afa90 con 0x563a84c52550 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8091d75740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80881ad070 con 0x563a84c48fd0 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8091d75740 10 client.0 _ll_register_callbacks cb 0x7f807c0cc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8087fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c56940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8087fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c56940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44578/0 (socket says 172.21.15.37:44578) 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8087fff640 1 -- 172.21.15.37:0/2229720656 learned_addr learned my addr 172.21.15.37:0/2229720656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8087fff640 1 -- 172.21.15.37:0/2229720656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c48fd0 msgr2=0x563a84c56400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8087fff640 1 --2- 172.21.15.37:0/2229720656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c48fd0 0x563a84c56400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8087fff640 1 -- 172.21.15.37:0/2229720656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c52550 msgr2=0x563a84cb8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8087fff640 1 --2- 172.21.15.37:0/2229720656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c52550 0x563a84cb8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8087fff640 1 -- 172.21.15.37:0/2229720656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a84cb93c0 con 0x563a84c49d40 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8087fff640 1 --2- 172.21.15.37:0/2229720656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c56940 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8074012a40 tx=0x7f807400fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd1110efa2af1c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8067fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8067fff640 1 -- 172.21.15.37:0/2229720656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074012660 con 0x563a84c49d40 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8067fff640 1 -- 172.21.15.37:0/2229720656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8074005040 con 0x563a84c49d40 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8091d75740 1 -- 172.21.15.37:0/2229720656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8080003b70 con 0x563a84c49d40 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8067fff640 1 -- 172.21.15.37:0/2229720656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f807403a050 con 0x563a84c49d40 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8091d75740 10 client.15507 fetch_fsmap learned FSMap version 27 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.311+0000 7f8091d75740 1 -- 172.21.15.37:0/2229720656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563a84cba500 con 0x563a84c49d40 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.312+0000 7f8067fff640 1 -- 172.21.15.37:0/2229720656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8080003b70 con 0x563a84c49d40 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.312+0000 7f8067fff640 1 -- 172.21.15.37:0/2229720656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8074004b90 con 0x563a84c49d40 2024-03-26T06:23:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.312+0000 7f8091d75740 10 client.15507 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:23:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.312+0000 7f8091d75740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:23:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.312+0000 7f8091d75740 20 client.15507 populate_metadata read hostname 'smithi037' 2024-03-26T06:23:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.312+0000 7f8091d75740 1 --2- 172.21.15.37:0/2229720656 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563a84cbac50 0x563a84cbd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.312+0000 7f8091d75740 4 client.15507 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:23:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.312+0000 7f8091d75740 1 -- 172.21.15.37:0/2229720656 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563a84bb4e80 con 0x563a84cbac50 2024-03-26T06:23:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.312+0000 7f808ce4e640 1 --2- 172.21.15.37:0/2229720656 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563a84cbac50 0x563a84cbd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.313+0000 7f8067fff640 10 client.15507 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.313+0000 7f808ce4e640 1 --2- 172.21.15.37:0/2229720656 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563a84cbac50 0x563a84cbd070 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7f69e72409988c server_cookie=f490ef0fdc3e17a0 in_seq=0 out_seq=0 2024-03-26T06:23:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8067fff640 1 -- 172.21.15.37:0/2229720656 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563a84bb4e80 con 0x563a84cbac50 2024-03-26T06:23:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8067fff640 10 client.15507 handle_command_reply: tid=0 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 1 client.15507 shutdown 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 1 -- 172.21.15.37:0/2229720656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 msgr2=0x563a84c56940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 1 --2- 172.21.15.37:0/2229720656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c56940 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8074012a40 tx=0x7f807400fbf0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 1 -- 172.21.15.37:0/2229720656 shutdown_connections 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 1 --2- 172.21.15.37:0/2229720656 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563a84cbac50 0x563a84cbd070 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 1 --2- 172.21.15.37:0/2229720656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563a84c52550 0x563a84cb8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 1 --2- 172.21.15.37:0/2229720656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a84c49d40 0x563a84c56940 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8067fff640 0 client.15507 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 1 --2- 172.21.15.37:0/2229720656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a84c48fd0 0x563a84c56400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 1 -- 172.21.15.37:0/2229720656 >> 172.21.15.37:0/2229720656 conn(0x563a84c105b0 msgr2=0x563a84c425b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 1 -- 172.21.15.37:0/2229720656 shutdown_connections 2024-03-26T06:23:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 1 -- 172.21.15.37:0/2229720656 wait complete. 2024-03-26T06:23:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.325+0000 7f8091d75740 20 client.15507 trim_cache size 0 max 16384 2024-03-26T06:23:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:23:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:23:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:23:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:23:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 -- 172.21.15.37:0/1592394987 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f80780a2b30 msgr2=0x7f80780a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f80780a2b30 0x7f80780a4ff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f80800098d0 tx=0x7f8080032480 comp rx=0 tx=0).stop 2024-03-26T06:23:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 -- 172.21.15.37:0/1592394987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80881024d0 msgr2=0x7f8088100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80881024d0 0x7f8088100120 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f807c00c970 tx=0x7f807c00cd70 comp rx=0 tx=0).stop 2024-03-26T06:23:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 -- 172.21.15.37:0/1592394987 shutdown_connections 2024-03-26T06:23:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f80780a2b30 0x7f80780a4ff0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f808810b940 0x7f8088100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8088102df0 0x7f8088100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 --2- 172.21.15.37:0/1592394987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80881024d0 0x7f8088100120 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 -- 172.21.15.37:0/1592394987 >> 172.21.15.37:0/1592394987 conn(0x7f8088077320 msgr2=0x7f80880fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 -- 172.21.15.37:0/1592394987 shutdown_connections 2024-03-26T06:23:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:14.326+0000 7f808f0b2640 1 -- 172.21.15.37:0/1592394987 wait complete. 2024-03-26T06:23:14.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:14 smithi066 ceph-mon[36504]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-26T06:23:14.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:14 smithi149 ceph-mon[36681]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-26T06:23:14.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:14 smithi037 ceph-mon[31974]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-26T06:23:16.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:16 smithi066 ceph-mon[36504]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-03-26T06:23:16.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:16 smithi149 ceph-mon[36681]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-03-26T06:23:16.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:16 smithi037 ceph-mon[31974]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-03-26T06:23:18.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:18 smithi066 ceph-mon[36504]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-03-26T06:23:18.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:18 smithi149 ceph-mon[36681]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-03-26T06:23:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:18 smithi037 ceph-mon[31974]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-03-26T06:23:20.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:20 smithi066 ceph-mon[36504]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T06:23:20.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:20 smithi149 ceph-mon[36681]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T06:23:20.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:20 smithi037 ceph-mon[31974]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T06:23:22.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:22 smithi066 ceph-mon[36504]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-03-26T06:23:22.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:22 smithi149 ceph-mon[36681]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-03-26T06:23:22.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:22 smithi037 ceph-mon[31974]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-03-26T06:23:24.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:24 smithi066 ceph-mon[36504]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T06:23:24.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:24 smithi149 ceph-mon[36681]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T06:23:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:24 smithi037 ceph-mon[31974]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T06:23:26.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:26 smithi066 ceph-mon[36504]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-26T06:23:26.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:26 smithi149 ceph-mon[36681]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-26T06:23:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:26 smithi037 ceph-mon[31974]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-26T06:23:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.423+0000 7f2a13220640 1 -- 172.21.15.37:0/3420346046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c06a140 msgr2=0x7f2a0c06a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.423+0000 7f2a13220640 1 --2- 172.21.15.37:0/3420346046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c06a140 0x7f2a0c06a5a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009900 tx=0x7f29f802bc30 comp rx=0 tx=0).stop 2024-03-26T06:23:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.423+0000 7f2a13220640 1 -- 172.21.15.37:0/3420346046 shutdown_connections 2024-03-26T06:23:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.423+0000 7f2a13220640 1 --2- 172.21.15.37:0/3420346046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0c10db40 0x7f2a0c10ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.423+0000 7f2a13220640 1 --2- 172.21.15.37:0/3420346046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c06a140 0x7f2a0c06a5a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.423+0000 7f2a13220640 1 --2- 172.21.15.37:0/3420346046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c102200 0x7f2a0c069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.423+0000 7f2a13220640 1 -- 172.21.15.37:0/3420346046 >> 172.21.15.37:0/3420346046 conn(0x7f2a0c077040 msgr2=0x7f2a0c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.424+0000 7f2a13220640 1 -- 172.21.15.37:0/3420346046 shutdown_connections 2024-03-26T06:23:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.424+0000 7f2a13220640 1 -- 172.21.15.37:0/3420346046 wait complete. 2024-03-26T06:23:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.424+0000 7f2a13220640 1 Processor -- start 2024-03-26T06:23:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.424+0000 7f2a13220640 1 -- start start 2024-03-26T06:23:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.424+0000 7f2a13220640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c06a140 0x7f2a0c199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.424+0000 7f2a13220640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c102200 0x7f2a0c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.424+0000 7f2a13220640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0c10db40 0x7f2a0c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.424+0000 7f2a13220640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c113890 con 0x7f2a0c102200 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.424+0000 7f2a13220640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c113710 con 0x7f2a0c10db40 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.424+0000 7f2a13220640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c113a10 con 0x7f2a0c06a140 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a0bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c102200 0x7f2a0c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a0bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c102200 0x7f2a0c19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49130/0 (socket says 172.21.15.37:49130) 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a0bfff640 1 -- 172.21.15.37:0/2279543939 learned_addr learned my addr 172.21.15.37:0/2279543939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a10fbc640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c06a140 0x7f2a0c199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a117bd640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0c10db40 0x7f2a0c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a0bfff640 1 -- 172.21.15.37:0/2279543939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c06a140 msgr2=0x7f2a0c199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a0bfff640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c06a140 0x7f2a0c199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a0bfff640 1 -- 172.21.15.37:0/2279543939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0c10db40 msgr2=0x7f2a0c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a0bfff640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0c10db40 0x7f2a0c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a0bfff640 1 -- 172.21.15.37:0/2279543939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c19ed80 con 0x7f2a0c102200 2024-03-26T06:23:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a0bfff640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c102200 0x7f2a0c19a2d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f29f8036040 tx=0x7f29f802cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a09ffb640 1 -- 172.21.15.37:0/2279543939 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8041070 con 0x7f2a0c102200 2024-03-26T06:23:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a09ffb640 1 -- 172.21.15.37:0/2279543939 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29f8004450 con 0x7f2a0c102200 2024-03-26T06:23:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a13220640 1 -- 172.21.15.37:0/2279543939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c19f010 con 0x7f2a0c102200 2024-03-26T06:23:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.425+0000 7f2a09ffb640 1 -- 172.21.15.37:0/2279543939 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8003dc0 con 0x7f2a0c102200 2024-03-26T06:23:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.426+0000 7f2a13220640 1 -- 172.21.15.37:0/2279543939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c1a6900 con 0x7f2a0c102200 2024-03-26T06:23:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.428+0000 7f2a09ffb640 1 -- 172.21.15.37:0/2279543939 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f29f8044070 con 0x7f2a0c102200 2024-03-26T06:23:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a15e03740 1 Processor -- start 2024-03-26T06:23:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a15e03740 1 -- start start 2024-03-26T06:23:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a526d40 0x556e6a525a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a525fd0 0x556e6a526450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a531940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a15e03740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e6a48a5f0 con 0x556e6a526d40 2024-03-26T06:23:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a15e03740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e6a433930 con 0x556e6a525fd0 2024-03-26T06:23:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a15e03740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e6a486690 con 0x556e6a52f550 2024-03-26T06:23:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a117bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a526d40 0x556e6a525a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a09ffb640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29fc0a2ab0 0x7f29fc0a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a117bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a526d40 0x556e6a525a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49144/0 (socket says 172.21.15.37:49144) 2024-03-26T06:23:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a117bd640 1 -- 172.21.15.37:0/2798940208 learned_addr learned my addr 172.21.15.37:0/2798940208 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a0bfff640 1 --2- 172.21.15.37:0/2798940208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a531940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a117bd640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29fc0a2ab0 0x7f29fc0a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a10fbc640 1 --2- 172.21.15.37:0/2798940208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a525fd0 0x556e6a526450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a117bd640 1 -- 172.21.15.37:0/2798940208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 msgr2=0x556e6a531940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a117bd640 1 --2- 172.21.15.37:0/2798940208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a531940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a117bd640 1 -- 172.21.15.37:0/2798940208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a525fd0 msgr2=0x556e6a526450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a117bd640 1 --2- 172.21.15.37:0/2798940208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a525fd0 0x556e6a526450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a117bd640 1 -- 172.21.15.37:0/2798940208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e6a19f210 con 0x556e6a526d40 2024-03-26T06:23:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a09ffb640 1 -- 172.21.15.37:0/2279543939 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f29f80f09d0 con 0x7f2a0c102200 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.429+0000 7f2a0bfff640 1 --2- 172.21.15.37:0/2798940208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a531940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.430+0000 7f2a117bd640 1 --2- 172.21.15.37:0/2798940208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a526d40 0x556e6a525a90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2a04013a60 tx=0x7f2a04013f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64920609cb78590 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.430+0000 7f29f6ffd640 1 -- 172.21.15.37:0/2798940208 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04023030 con 0x556e6a526d40 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.430+0000 7f29f6ffd640 1 -- 172.21.15.37:0/2798940208 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a0400fe40 con 0x556e6a526d40 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.430+0000 7f29f6ffd640 1 -- 172.21.15.37:0/2798940208 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a040107b0 con 0x556e6a526d40 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.430+0000 7f2a15e03740 1 -- 172.21.15.37:0/2798940208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a526d40 msgr2=0x556e6a525a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.430+0000 7f2a117bd640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29fc0a2ab0 0x7f29fc0a4f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2a04023000 tx=0x7f2a04020000 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.430+0000 7f2a15e03740 1 --2- 172.21.15.37:0/2798940208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a526d40 0x556e6a525a90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2a04013a60 tx=0x7f2a04013f20 comp rx=0 tx=0).stop 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.430+0000 7f2a15e03740 1 -- 172.21.15.37:0/2798940208 shutdown_connections 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 --2- 172.21.15.37:0/2798940208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a531940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 --2- 172.21.15.37:0/2798940208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a525fd0 0x556e6a526450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 --2- 172.21.15.37:0/2798940208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a526d40 0x556e6a525a90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 -- 172.21.15.37:0/2798940208 >> 172.21.15.37:0/2798940208 conn(0x556e6a4ed5b0 msgr2=0x556e6a520cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 -- 172.21.15.37:0/2798940208 shutdown_connections 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 -- 172.21.15.37:0/2798940208 wait complete. 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 -- start start 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 Processor -- start 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a52c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 0x556e6a52cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a52cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e6a532280 con 0x556e6a525fd0 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e6a446d90 con 0x556e6a526d40 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e6a532400 con 0x556e6a52f550 2024-03-26T06:23:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.431+0000 7f2a15e03740 10 client.0 _ll_register_callbacks cb 0x556e6a19ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a10fbc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a52c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a10fbc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a52c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49160/0 (socket says 172.21.15.37:49160) 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a117bd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a52cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a10fbc640 1 -- 172.21.15.37:0/293864649 learned_addr learned my addr 172.21.15.37:0/293864649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a0bfff640 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 0x556e6a52cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a117bd640 1 -- 172.21.15.37:0/293864649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 msgr2=0x556e6a52cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a117bd640 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 0x556e6a52cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a117bd640 1 -- 172.21.15.37:0/293864649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 msgr2=0x556e6a52c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a117bd640 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a52c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a117bd640 1 -- 172.21.15.37:0/293864649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e6a26b620 con 0x556e6a52f550 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a10fbc640 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a52c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:23:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a117bd640 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a52cff0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2a04005fd0 tx=0x7f2a04009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96fba5946a7b40d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f29ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:23:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f29ebfff640 1 -- 172.21.15.37:0/293864649 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04010d40 con 0x556e6a52f550 2024-03-26T06:23:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a15e03740 1 -- 172.21.15.37:0/293864649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e6a497b30 con 0x556e6a52f550 2024-03-26T06:23:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f29ebfff640 1 -- 172.21.15.37:0/293864649 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2a0401f850 con 0x556e6a52f550 2024-03-26T06:23:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f29ebfff640 1 -- 172.21.15.37:0/293864649 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a0401fa50 con 0x556e6a52f550 2024-03-26T06:23:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.432+0000 7f2a15e03740 10 client.24994 fetch_fsmap learned FSMap version 27 2024-03-26T06:23:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.433+0000 7f2a15e03740 1 -- 172.21.15.37:0/293864649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556e6a53e540 con 0x556e6a52f550 2024-03-26T06:23:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.433+0000 7f29ebfff640 1 -- 172.21.15.37:0/293864649 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556e6a497b30 con 0x556e6a52f550 2024-03-26T06:23:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.433+0000 7f29ebfff640 1 -- 172.21.15.37:0/293864649 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2a0402f020 con 0x556e6a52f550 2024-03-26T06:23:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.433+0000 7f2a15e03740 10 client.24994 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:23:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.433+0000 7f2a15e03740 10 client.24994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.433+0000 7f2a15e03740 20 client.24994 populate_metadata read hostname 'smithi037' 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.433+0000 7f2a15e03740 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556e6a53f890 0x556e6a541c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.433+0000 7f2a15e03740 4 client.24994 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.433+0000 7f2a15e03740 1 -- 172.21.15.37:0/293864649 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e6a491e80 con 0x556e6a53f890 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.433+0000 7f2a10fbc640 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556e6a53f890 0x556e6a541c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.434+0000 7f2a10fbc640 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556e6a53f890 0x556e6a541c80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0de6325cd078b8a server_cookie=e0a73ffb0619a821 in_seq=0 out_seq=0 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.434+0000 7f29ebfff640 10 client.24994 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.434+0000 7f29ebfff640 1 -- 172.21.15.37:0/293864649 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556e6a491e80 con 0x556e6a53f890 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.434+0000 7f29ebfff640 10 client.24994 handle_command_reply: tid=0 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.434+0000 7f2a15e03740 1 client.24994 shutdown 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f2a15e03740 1 -- 172.21.15.37:0/293864649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 msgr2=0x556e6a52cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f2a15e03740 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a52cff0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2a04005fd0 tx=0x7f2a04009630 comp rx=0 tx=0).stop 2024-03-26T06:23:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f2a15e03740 1 -- 172.21.15.37:0/293864649 shutdown_connections 2024-03-26T06:23:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f2a15e03740 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556e6a53f890 0x556e6a541c80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f2a15e03740 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a52cff0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f2a15e03740 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 0x556e6a52cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f2a15e03740 1 --2- 172.21.15.37:0/293864649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a52c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f29ebfff640 0 client.24994 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f2a15e03740 1 -- 172.21.15.37:0/293864649 >> 172.21.15.37:0/293864649 conn(0x556e6a4ed5b0 msgr2=0x556e6a522a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f2a15e03740 1 -- 172.21.15.37:0/293864649 shutdown_connections 2024-03-26T06:23:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f2a15e03740 1 -- 172.21.15.37:0/293864649 wait complete. 2024-03-26T06:23:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.435+0000 7f2a15e03740 20 client.24994 trim_cache size 0 max 16384 2024-03-26T06:23:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a15e03740 1 Processor -- start 2024-03-26T06:23:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a15e03740 1 -- start start 2024-03-26T06:23:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a525fd0 0x556e6a52ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 0x556e6a52b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a52f550 0x556e6a5297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a15e03740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c1aee60 con 0x556e6a52f550 2024-03-26T06:23:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a15e03740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c1b07e0 con 0x556e6a526d40 2024-03-26T06:23:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a15e03740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c1af4c0 con 0x556e6a525fd0 2024-03-26T06:23:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a117bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a52f550 0x556e6a5297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a117bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a52f550 0x556e6a5297d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49174/0 (socket says 172.21.15.37:49174) 2024-03-26T06:23:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a117bd640 1 -- 172.21.15.37:0/2600389755 learned_addr learned my addr 172.21.15.37:0/2600389755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.448+0000 7f2a10fbc640 1 --2- 172.21.15.37:0/2600389755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a525fd0 0x556e6a52ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.449+0000 7f2a117bd640 1 -- 172.21.15.37:0/2600389755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a525fd0 msgr2=0x556e6a52ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.449+0000 7f2a117bd640 1 --2- 172.21.15.37:0/2600389755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a525fd0 0x556e6a52ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.449+0000 7f2a117bd640 1 -- 172.21.15.37:0/2600389755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 msgr2=0x556e6a52b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.449+0000 7f2a117bd640 1 --2- 172.21.15.37:0/2600389755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 0x556e6a52b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.449+0000 7f2a117bd640 1 -- 172.21.15.37:0/2600389755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e6a528140 con 0x556e6a52f550 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.449+0000 7f2a117bd640 1 --2- 172.21.15.37:0/2600389755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a52f550 0x556e6a5297d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2a04013f60 tx=0x7f2a04009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23bf0ce91910163 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.449+0000 7f29f67fc640 1 -- 172.21.15.37:0/2600389755 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0403f070 con 0x556e6a52f550 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.449+0000 7f29f67fc640 1 -- 172.21.15.37:0/2600389755 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a04003e50 con 0x556e6a52f550 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.449+0000 7f29f67fc640 1 -- 172.21.15.37:0/2600389755 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04010b40 con 0x556e6a52f550 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 -- 172.21.15.37:0/2600389755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a52f550 msgr2=0x556e6a5297d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 --2- 172.21.15.37:0/2600389755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a52f550 0x556e6a5297d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2a04013f60 tx=0x7f2a04009fc0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 -- 172.21.15.37:0/2600389755 shutdown_connections 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 --2- 172.21.15.37:0/2600389755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a52f550 0x556e6a5297d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 --2- 172.21.15.37:0/2600389755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 0x556e6a52b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 --2- 172.21.15.37:0/2600389755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a525fd0 0x556e6a52ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 -- 172.21.15.37:0/2600389755 >> 172.21.15.37:0/2600389755 conn(0x556e6a4ed5b0 msgr2=0x556e6a5464a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 -- 172.21.15.37:0/2600389755 shutdown_connections 2024-03-26T06:23:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 -- 172.21.15.37:0/2600389755 wait complete. 2024-03-26T06:23:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 -- start start 2024-03-26T06:23:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 Processor -- start 2024-03-26T06:23:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a536700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 0x556e6a534aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.450+0000 7f2a15e03740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a534fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a15e03740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c1b0180 con 0x556e6a525fd0 2024-03-26T06:23:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a15e03740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c1afb20 con 0x556e6a526d40 2024-03-26T06:23:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a15e03740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c1ad100 con 0x556e6a52f550 2024-03-26T06:23:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a15e03740 10 client.0 _ll_register_callbacks cb 0x7f2a04009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:23:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a10fbc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a536700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a10fbc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a536700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49186/0 (socket says 172.21.15.37:49186) 2024-03-26T06:23:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a10fbc640 1 -- 172.21.15.37:0/795001156 learned_addr learned my addr 172.21.15.37:0/795001156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a117bd640 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a534fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a0bfff640 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 0x556e6a534aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a10fbc640 1 -- 172.21.15.37:0/795001156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 msgr2=0x556e6a534fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a10fbc640 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a534fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a10fbc640 1 -- 172.21.15.37:0/795001156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 msgr2=0x556e6a534aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a10fbc640 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 0x556e6a534aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a10fbc640 1 -- 172.21.15.37:0/795001156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e6a535520 con 0x556e6a525fd0 2024-03-26T06:23:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a117bd640 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a534fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:23:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a10fbc640 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a536700 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x556e6a542450 tx=0x7f2a00012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d141e91d3763dbe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f29ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f29ebfff640 1 -- 172.21.15.37:0/795001156 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a00021640 con 0x556e6a525fd0 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f29ebfff640 1 -- 172.21.15.37:0/795001156 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2a000217e0 con 0x556e6a525fd0 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a15e03740 1 -- 172.21.15.37:0/795001156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a04003e50 con 0x556e6a525fd0 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f29ebfff640 1 -- 172.21.15.37:0/795001156 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a00005020 con 0x556e6a525fd0 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a15e03740 10 client.15525 fetch_fsmap learned FSMap version 27 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.451+0000 7f2a15e03740 1 -- 172.21.15.37:0/795001156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556e6a5969b0 con 0x556e6a525fd0 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.452+0000 7f29ebfff640 1 -- 172.21.15.37:0/795001156 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a04003e50 con 0x556e6a525fd0 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.452+0000 7f29ebfff640 1 -- 172.21.15.37:0/795001156 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2a00028070 con 0x556e6a525fd0 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.452+0000 7f2a15e03740 10 client.15525 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.452+0000 7f2a15e03740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.452+0000 7f2a15e03740 20 client.15525 populate_metadata read hostname 'smithi037' 2024-03-26T06:23:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.452+0000 7f2a15e03740 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556e6a597110 0x556e6a599530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.452+0000 7f2a15e03740 4 client.15525 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:23:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.452+0000 7f2a15e03740 1 -- 172.21.15.37:0/795001156 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556e6a410710 con 0x556e6a597110 2024-03-26T06:23:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.452+0000 7f2a0bfff640 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556e6a597110 0x556e6a599530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.452+0000 7f2a0bfff640 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556e6a597110 0x556e6a599530 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b76ac454a75708e7 server_cookie=26e0e4e852258e3d in_seq=0 out_seq=0 2024-03-26T06:23:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.452+0000 7f29ebfff640 10 client.15525 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f29ebfff640 1 -- 172.21.15.37:0/795001156 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556e6a410710 con 0x556e6a597110 2024-03-26T06:23:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f29ebfff640 10 client.15525 handle_command_reply: tid=0 2024-03-26T06:23:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 1 client.15525 shutdown 2024-03-26T06:23:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 1 -- 172.21.15.37:0/795001156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 msgr2=0x556e6a536700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a536700 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x556e6a542450 tx=0x7f2a00012c30 comp rx=0 tx=0).stop 2024-03-26T06:23:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 1 -- 172.21.15.37:0/795001156 shutdown_connections 2024-03-26T06:23:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556e6a597110 0x556e6a599530 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e6a52f550 0x556e6a534fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556e6a526d40 0x556e6a534aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f29ebfff640 0 client.15525 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 1 --2- 172.21.15.37:0/795001156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e6a525fd0 0x556e6a536700 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 1 -- 172.21.15.37:0/795001156 >> 172.21.15.37:0/795001156 conn(0x556e6a4ed5b0 msgr2=0x556e6a52acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 1 -- 172.21.15.37:0/795001156 shutdown_connections 2024-03-26T06:23:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 1 -- 172.21.15.37:0/795001156 wait complete. 2024-03-26T06:23:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.459+0000 7f2a15e03740 20 client.15525 trim_cache size 0 max 16384 2024-03-26T06:23:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:23:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:23:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:23:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:23:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 -- 172.21.15.37:0/2279543939 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29fc0a2ab0 msgr2=0x7f29fc0a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29fc0a2ab0 0x7f29fc0a4f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2a04023000 tx=0x7f2a04020000 comp rx=0 tx=0).stop 2024-03-26T06:23:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 -- 172.21.15.37:0/2279543939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c102200 msgr2=0x7f2a0c19a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c102200 0x7f2a0c19a2d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f29f8036040 tx=0x7f29f802cb20 comp rx=0 tx=0).stop 2024-03-26T06:23:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 -- 172.21.15.37:0/2279543939 shutdown_connections 2024-03-26T06:23:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29fc0a2ab0 0x7f29fc0a4f70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a0c10db40 0x7f2a0c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c102200 0x7f2a0c19a2d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 --2- 172.21.15.37:0/2279543939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c06a140 0x7f2a0c199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 -- 172.21.15.37:0/2279543939 >> 172.21.15.37:0/2279543939 conn(0x7f2a0c077040 msgr2=0x7f2a0c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 -- 172.21.15.37:0/2279543939 shutdown_connections 2024-03-26T06:23:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:27.460+0000 7f2a13220640 1 -- 172.21.15.37:0/2279543939 wait complete. 2024-03-26T06:23:28.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:28 smithi066 ceph-mon[36504]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-03-26T06:23:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:28 smithi149 ceph-mon[36681]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-03-26T06:23:28.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:28 smithi037 ceph-mon[31974]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-03-26T06:23:30.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:30 smithi066 ceph-mon[36504]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-26T06:23:30.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:30 smithi149 ceph-mon[36681]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-26T06:23:30.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:30 smithi037 ceph-mon[31974]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-26T06:23:32.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:32 smithi066 ceph-mon[36504]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 13 op/s 2024-03-26T06:23:32.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:32 smithi149 ceph-mon[36681]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 13 op/s 2024-03-26T06:23:32.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:32 smithi037 ceph-mon[31974]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 13 op/s 2024-03-26T06:23:34.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:34 smithi066 ceph-mon[36504]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-26T06:23:34.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:34 smithi149 ceph-mon[36681]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-26T06:23:34.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:34 smithi037 ceph-mon[31974]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-26T06:23:36.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:36 smithi066 ceph-mon[36504]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T06:23:36.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:36 smithi149 ceph-mon[36681]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T06:23:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:36 smithi037 ceph-mon[31974]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T06:23:38.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:38 smithi066 ceph-mon[36504]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-26T06:23:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:38 smithi149 ceph-mon[36681]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-26T06:23:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:38 smithi037 ceph-mon[31974]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-26T06:23:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.557+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2343124587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5a8101510 msgr2=0x7fc5a81018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.557+0000 7fc5af0fc640 1 --2- 172.21.15.37:0/2343124587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5a8101510 0x7fc5a81018f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc598009900 tx=0x7fc59802bc30 comp rx=0 tx=0).stop 2024-03-26T06:23:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2343124587 shutdown_connections 2024-03-26T06:23:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 --2- 172.21.15.37:0/2343124587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a81065e0 0x7fc5a810e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 --2- 172.21.15.37:0/2343124587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5a8101ec0 0x7fc5a8105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 --2- 172.21.15.37:0/2343124587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5a8101510 0x7fc5a81018f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2343124587 >> 172.21.15.37:0/2343124587 conn(0x7fc5a80fcdb0 msgr2=0x7fc5a80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2343124587 shutdown_connections 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2343124587 wait complete. 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 Processor -- start 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 -- start start 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5a8101510 0x7fc5a819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a81065e0 0x7fc5a819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5a81979f0 0x7fc5a8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8111670 con 0x7fc5a81979f0 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81114f0 con 0x7fc5a81065e0 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81117f0 con 0x7fc5a8101510 2024-03-26T06:23:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5ad699640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5a81979f0 0x7fc5a8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5ad699640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5a81979f0 0x7fc5a8197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48648/0 (socket says 172.21.15.37:48648) 2024-03-26T06:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5ad699640 1 -- 172.21.15.37:0/2558899910 learned_addr learned my addr 172.21.15.37:0/2558899910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5ace98640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5a8101510 0x7fc5a819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a81065e0 0x7fc5a819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5ad699640 1 -- 172.21.15.37:0/2558899910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5a8101510 msgr2=0x7fc5a819d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5ad699640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5a8101510 0x7fc5a819d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5ad699640 1 -- 172.21.15.37:0/2558899910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a81065e0 msgr2=0x7fc5a819dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5ad699640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a81065e0 0x7fc5a819dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5ad699640 1 -- 172.21.15.37:0/2558899910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a8198510 con 0x7fc5a81979f0 2024-03-26T06:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5ad699640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5a81979f0 0x7fc5a8197ea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000eaa0 tx=0x7fc5a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5a5ffb640 1 -- 172.21.15.37:0/2558899910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000cdb0 con 0x7fc5a81979f0 2024-03-26T06:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5a5ffb640 1 -- 172.21.15.37:0/2558899910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5a0005020 con 0x7fc5a81979f0 2024-03-26T06:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2558899910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a8198800 con 0x7fc5a81979f0 2024-03-26T06:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.559+0000 7fc5a5ffb640 1 -- 172.21.15.37:0/2558899910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00116b0 con 0x7fc5a81979f0 2024-03-26T06:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.560+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2558899910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a81a49b0 con 0x7fc5a81979f0 2024-03-26T06:23:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.562+0000 7fc5a5ffb640 1 -- 172.21.15.37:0/2558899910 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7fc5a0005560 con 0x7fc5a81979f0 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5b1db4740 1 Processor -- start 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5b1db4740 1 -- start start 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf9d40 0x565010cf8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010cf8fd0 0x565010cf9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010d02550 0x565010d04940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x565010c5d5f0 con 0x565010cf8fd0 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x565010c06930 con 0x565010d02550 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x565010c59690 con 0x565010cf9d40 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5a5ffb640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc5900a2ab0 0x7fc5900a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5ace98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010cf8fd0 0x565010cf9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5ace98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010cf8fd0 0x565010cf9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48650/0 (socket says 172.21.15.37:48650) 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5ace98640 1 -- 172.21.15.37:0/2420758006 learned_addr learned my addr 172.21.15.37:0/2420758006 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/2420758006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf9d40 0x565010cf8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5ad699640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc5900a2ab0 0x7fc5900a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/2420758006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010d02550 0x565010d04940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.563+0000 7fc5a5ffb640 1 -- 172.21.15.37:0/2558899910 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc5a00cf250 con 0x7fc5a81979f0 2024-03-26T06:23:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.564+0000 7fc5ace98640 1 -- 172.21.15.37:0/2420758006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf9d40 msgr2=0x565010cf8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.564+0000 7fc5ace98640 1 --2- 172.21.15.37:0/2420758006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf9d40 0x565010cf8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.564+0000 7fc5ace98640 1 -- 172.21.15.37:0/2420758006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010d02550 msgr2=0x565010d04940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.564+0000 7fc5ace98640 1 --2- 172.21.15.37:0/2420758006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010d02550 0x565010d04940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.564+0000 7fc5ace98640 1 -- 172.21.15.37:0/2420758006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565010972210 con 0x565010cf8fd0 2024-03-26T06:23:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.564+0000 7fc5ace98640 1 --2- 172.21.15.37:0/2420758006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010cf8fd0 0x565010cf9450 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc59802c170 tx=0x7fc598005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36566940ecd76ad3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.564+0000 7fc5ad699640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc5900a2ab0 0x7fc5900a4f70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00cc910 tx=0x7fc5a006d340 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.564+0000 7fc58effd640 1 -- 172.21.15.37:0/2420758006 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598042070 con 0x565010cf8fd0 2024-03-26T06:23:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.564+0000 7fc58effd640 1 -- 172.21.15.37:0/2420758006 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc59802ca80 con 0x565010cf8fd0 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.564+0000 7fc58effd640 1 -- 172.21.15.37:0/2420758006 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598004ec0 con 0x565010cf8fd0 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.565+0000 7fc5b1db4740 1 -- 172.21.15.37:0/2420758006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010cf8fd0 msgr2=0x565010cf9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.565+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/2420758006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010cf8fd0 0x565010cf9450 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc59802c170 tx=0x7fc598005e80 comp rx=0 tx=0).stop 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.565+0000 7fc5b1db4740 1 -- 172.21.15.37:0/2420758006 shutdown_connections 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.565+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/2420758006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010d02550 0x565010d04940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.565+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/2420758006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010cf8fd0 0x565010cf9450 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.565+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/2420758006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf9d40 0x565010cf8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.565+0000 7fc5b1db4740 1 -- 172.21.15.37:0/2420758006 >> 172.21.15.37:0/2420758006 conn(0x565010cc05b0 msgr2=0x565010cf3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.565+0000 7fc5b1db4740 1 -- 172.21.15.37:0/2420758006 shutdown_connections 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.565+0000 7fc5b1db4740 1 -- 172.21.15.37:0/2420758006 wait complete. 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.565+0000 7fc5b1db4740 1 -- start start 2024-03-26T06:23:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.565+0000 7fc5b1db4740 1 Processor -- start 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf8fd0 0x565010cff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf9d40 0x565010cffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d00070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x565010d05220 con 0x565010d02550 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x565010c19d90 con 0x565010cf8fd0 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x565010d053a0 con 0x565010cf9d40 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5b1db4740 10 client.0 _ll_register_callbacks cb 0x565010971a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d00070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d00070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48664/0 (socket says 172.21.15.37:48664) 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3368597688 learned_addr learned my addr 172.21.15.37:0/3368597688 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf8fd0 0x565010cff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5ace98640 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf9d40 0x565010cffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3368597688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf9d40 msgr2=0x565010cffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf9d40 0x565010cffb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3368597688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf8fd0 msgr2=0x565010cff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf8fd0 0x565010cff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3368597688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565010a3e620 con 0x565010d02550 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d00070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8199190 tx=0x7fc59c014dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283e41de7db49265 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc587fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc587fff640 1 -- 172.21.15.37:0/3368597688 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c005020 con 0x565010d02550 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.566+0000 7fc587fff640 1 -- 172.21.15.37:0/3368597688 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc59c0051c0 con 0x565010d02550 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3368597688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc59802ca80 con 0x565010d02550 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc587fff640 1 -- 172.21.15.37:0/3368597688 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc59c0053c0 con 0x565010d02550 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc5b1db4740 10 client.15537 fetch_fsmap learned FSMap version 27 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3368597688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565010d11560 con 0x565010d02550 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc587fff640 1 -- 172.21.15.37:0/3368597688 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc59802ca80 con 0x565010d02550 2024-03-26T06:23:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc587fff640 1 -- 172.21.15.37:0/3368597688 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc59c00f040 con 0x565010d02550 2024-03-26T06:23:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc5b1db4740 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:23:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc5b1db4740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:23:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc5b1db4740 20 client.15537 populate_metadata read hostname 'smithi037' 2024-03-26T06:23:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x565010d12a20 0x565010d14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc5b1db4740 4 client.15537 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:23:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3368597688 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565010c6ab30 con 0x565010d12a20 2024-03-26T06:23:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.567+0000 7fc5ace98640 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x565010d12a20 0x565010d14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.568+0000 7fc5ace98640 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x565010d12a20 0x565010d14e10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853458ca4a935c91 server_cookie=8b2660bddc9b8b33 in_seq=0 out_seq=0 2024-03-26T06:23:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.568+0000 7fc587fff640 10 client.15537 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc587fff640 1 -- 172.21.15.37:0/3368597688 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565010c6ab30 con 0x565010d12a20 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc587fff640 10 client.15537 handle_command_reply: tid=0 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 1 client.15537 shutdown 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3368597688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 msgr2=0x565010d00070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d00070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8199190 tx=0x7fc59c014dd0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3368597688 shutdown_connections 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x565010d12a20 0x565010d14e10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d00070 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc587fff640 0 client.15537 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf9d40 0x565010cffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3368597688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf8fd0 0x565010cff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3368597688 >> 172.21.15.37:0/3368597688 conn(0x565010cc05b0 msgr2=0x565010cf5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3368597688 shutdown_connections 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3368597688 wait complete. 2024-03-26T06:23:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.569+0000 7fc5b1db4740 20 client.15537 trim_cache size 0 max 16384 2024-03-26T06:23:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.582+0000 7fc5b1db4740 1 Processor -- start 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.582+0000 7fc5b1db4740 1 -- start start 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf8fd0 0x565010d01c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf9d40 0x565010d09610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d09b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81aaa10 con 0x565010d02550 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81ac390 con 0x565010cf9d40 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81ab070 con 0x565010cf8fd0 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d09b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d09b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48674/0 (socket says 172.21.15.37:48674) 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3979399049 learned_addr learned my addr 172.21.15.37:0/3979399049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3979399049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf8fd0 0x565010d01c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5ace98640 1 --2- 172.21.15.37:0/3979399049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf9d40 0x565010d09610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3979399049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf8fd0 msgr2=0x565010d01c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3979399049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf8fd0 0x565010d01c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3979399049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf9d40 msgr2=0x565010d09610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3979399049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf9d40 0x565010d09610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3979399049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565010cf82a0 con 0x565010d02550 2024-03-26T06:23:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.583+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3979399049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d09b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc59c014880 tx=0x7fc59c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c37c095afabd28d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc58e7fc640 1 -- 172.21.15.37:0/3979399049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00e750 con 0x565010d02550 2024-03-26T06:23:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc58e7fc640 1 -- 172.21.15.37:0/3979399049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc59c00f040 con 0x565010d02550 2024-03-26T06:23:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc58e7fc640 1 -- 172.21.15.37:0/3979399049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c009de0 con 0x565010d02550 2024-03-26T06:23:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3979399049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 msgr2=0x565010d09b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3979399049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d09b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc59c014880 tx=0x7fc59c00ed10 comp rx=0 tx=0).stop 2024-03-26T06:23:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3979399049 shutdown_connections 2024-03-26T06:23:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3979399049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010d09b50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3979399049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf9d40 0x565010d09610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3979399049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf8fd0 0x565010d01c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3979399049 >> 172.21.15.37:0/3979399049 conn(0x565010cc05b0 msgr2=0x565010cf37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3979399049 shutdown_connections 2024-03-26T06:23:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3979399049 wait complete. 2024-03-26T06:23:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.587+0000 7fc5b1db4740 1 -- start start 2024-03-26T06:23:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5b1db4740 1 Processor -- start 2024-03-26T06:23:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf8fd0 0x565010cfc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf9d40 0x565010cfc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5b1db4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010cfcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81abd30 con 0x565010d02550 2024-03-26T06:23:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81ab6d0 con 0x565010cf9d40 2024-03-26T06:23:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5b1db4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81a8cb0 con 0x565010cf8fd0 2024-03-26T06:23:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5b1db4740 10 client.0 _ll_register_callbacks cb 0x7fc598005dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:23:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010cfcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010cfcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48680/0 (socket says 172.21.15.37:48680) 2024-03-26T06:23:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3258615832 learned_addr learned my addr 172.21.15.37:0/3258615832 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5ace98640 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf9d40 0x565010cfc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf8fd0 0x565010cfc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3258615832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf8fd0 msgr2=0x565010cfc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.588+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf8fd0 0x565010cfc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3258615832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf9d40 msgr2=0x565010cfc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf9d40 0x565010cfc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5a7fff640 1 -- 172.21.15.37:0/3258615832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565010d69070 con 0x565010d02550 2024-03-26T06:23:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5ace98640 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf9d40 0x565010cfc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:23:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5a7fff640 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010cfcea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x565010cfd630 tx=0x7fc59c00e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de13c7ac50f8164 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc587fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:23:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc587fff640 1 -- 172.21.15.37:0/3258615832 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c009d80 con 0x565010d02550 2024-03-26T06:23:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc587fff640 1 -- 172.21.15.37:0/3258615832 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc59c02b430 con 0x565010d02550 2024-03-26T06:23:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3258615832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc59c00f040 con 0x565010d02550 2024-03-26T06:23:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc587fff640 1 -- 172.21.15.37:0/3258615832 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc59c0149d0 con 0x565010d02550 2024-03-26T06:23:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5b1db4740 10 client.15543 fetch_fsmap learned FSMap version 27 2024-03-26T06:23:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3258615832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565010d6a340 con 0x565010d02550 2024-03-26T06:23:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc587fff640 1 -- 172.21.15.37:0/3258615832 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc59c00f040 con 0x565010d02550 2024-03-26T06:23:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc587fff640 1 -- 172.21.15.37:0/3258615832 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc59c036520 con 0x565010d02550 2024-03-26T06:23:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5b1db4740 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:23:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5b1db4740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:23:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5b1db4740 20 client.15543 populate_metadata read hostname 'smithi037' 2024-03-26T06:23:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x565010d6a8a0 0x565010d6cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5b1db4740 4 client.15543 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:23:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.589+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3258615832 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565010c64e80 con 0x565010d6a8a0 2024-03-26T06:23:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.590+0000 7fc5ace98640 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x565010d6a8a0 0x565010d6cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.590+0000 7fc587fff640 10 client.15543 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.590+0000 7fc5ace98640 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x565010d6a8a0 0x565010d6cc90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=678db67f735bb1d2 server_cookie=8dd364552b74dc97 in_seq=0 out_seq=0 2024-03-26T06:23:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.597+0000 7fc587fff640 1 -- 172.21.15.37:0/3258615832 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565010c64e80 con 0x565010d6a8a0 2024-03-26T06:23:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.597+0000 7fc587fff640 10 client.15543 handle_command_reply: tid=0 2024-03-26T06:23:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.597+0000 7fc5b1db4740 1 client.15543 shutdown 2024-03-26T06:23:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3258615832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 msgr2=0x565010cfcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010cfcea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x565010cfd630 tx=0x7fc59c00e670 comp rx=0 tx=0).stop 2024-03-26T06:23:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3258615832 shutdown_connections 2024-03-26T06:23:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x565010d6a8a0 0x565010d6cc90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565010d02550 0x565010cfcea0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565010cf9d40 0x565010cfc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc587fff640 0 client.15543 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5b1db4740 1 --2- 172.21.15.37:0/3258615832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565010cf8fd0 0x565010cfc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3258615832 >> 172.21.15.37:0/3258615832 conn(0x565010cc05b0 msgr2=0x565010cfde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3258615832 shutdown_connections 2024-03-26T06:23:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5b1db4740 1 -- 172.21.15.37:0/3258615832 wait complete. 2024-03-26T06:23:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5b1db4740 20 client.15543 trim_cache size 0 max 16384 2024-03-26T06:23:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:23:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:23:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:23:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:23:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2558899910 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc5900a2ab0 msgr2=0x7fc5900a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5af0fc640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc5900a2ab0 0x7fc5900a4f70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00cc910 tx=0x7fc5a006d340 comp rx=0 tx=0).stop 2024-03-26T06:23:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2558899910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5a81979f0 msgr2=0x7fc5a8197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5af0fc640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5a81979f0 0x7fc5a8197ea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000eaa0 tx=0x7fc5a000ef60 comp rx=0 tx=0).stop 2024-03-26T06:23:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2558899910 shutdown_connections 2024-03-26T06:23:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5af0fc640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc5900a2ab0 0x7fc5900a4f70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5af0fc640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5a81979f0 0x7fc5a8197ea0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5af0fc640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc5a81065e0 0x7fc5a819dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5af0fc640 1 --2- 172.21.15.37:0/2558899910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5a8101510 0x7fc5a819d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.598+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2558899910 >> 172.21.15.37:0/2558899910 conn(0x7fc5a80fcdb0 msgr2=0x7fc5a80ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.599+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2558899910 shutdown_connections 2024-03-26T06:23:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:40.599+0000 7fc5af0fc640 1 -- 172.21.15.37:0/2558899910 wait complete. 2024-03-26T06:23:40.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:40 smithi066 ceph-mon[36504]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-26T06:23:40.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:40 smithi149 ceph-mon[36681]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-26T06:23:40.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:40 smithi037 ceph-mon[31974]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-26T06:23:42.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:42 smithi066 ceph-mon[36504]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-03-26T06:23:42.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:42 smithi149 ceph-mon[36681]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-03-26T06:23:42.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:42 smithi037 ceph-mon[31974]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-03-26T06:23:44.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:44 smithi066 ceph-mon[36504]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-26T06:23:44.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:44 smithi149 ceph-mon[36681]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-26T06:23:44.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:44 smithi037 ceph-mon[31974]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-26T06:23:46.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:46 smithi066 ceph-mon[36504]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-03-26T06:23:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:46 smithi149 ceph-mon[36681]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-03-26T06:23:46.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:46 smithi037 ceph-mon[31974]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-03-26T06:23:48.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:48 smithi066 ceph-mon[36504]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-26T06:23:48.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:48 smithi149 ceph-mon[36681]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-26T06:23:48.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:48 smithi037 ceph-mon[31974]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-26T06:23:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:50 smithi149 ceph-mon[36681]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-26T06:23:50.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:50 smithi037 ceph-mon[31974]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-26T06:23:51.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:50 smithi066 ceph-mon[36504]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-26T06:23:52.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:52 smithi149 ceph-mon[36681]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-26T06:23:52.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:52 smithi037 ceph-mon[31974]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-26T06:23:53.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:52 smithi066 ceph-mon[36504]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-26T06:23:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.687+0000 7f202e3d4640 1 -- 172.21.15.37:0/2571468809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2028100e00 msgr2=0x7f202810d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.687+0000 7f202e3d4640 1 --2- 172.21.15.37:0/2571468809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2028100e00 0x7f202810d020 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2020009ca0 tx=0x7f202002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.687+0000 7f202e3d4640 1 -- 172.21.15.37:0/2571468809 shutdown_connections 2024-03-26T06:23:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.687+0000 7f202e3d4640 1 --2- 172.21.15.37:0/2571468809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2028100e00 0x7f202810d020 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.687+0000 7f202e3d4640 1 --2- 172.21.15.37:0/2571468809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2028100440 0x7f20281008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.687+0000 7f202e3d4640 1 --2- 172.21.15.37:0/2571468809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2028105190 0x7f2028105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.687+0000 7f202e3d4640 1 -- 172.21.15.37:0/2571468809 >> 172.21.15.37:0/2571468809 conn(0x7f2028077040 msgr2=0x7f2028077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.687+0000 7f202e3d4640 1 -- 172.21.15.37:0/2571468809 shutdown_connections 2024-03-26T06:23:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.687+0000 7f202e3d4640 1 -- 172.21.15.37:0/2571468809 wait complete. 2024-03-26T06:23:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.688+0000 7f202e3d4640 1 Processor -- start 2024-03-26T06:23:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.688+0000 7f202e3d4640 1 -- start start 2024-03-26T06:23:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202e3d4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2028100440 0x7f202819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202e3d4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2028100e00 0x7f202819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202e3d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2028105190 0x7f20281979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202e3d4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20281117d0 con 0x7f2028105190 2024-03-26T06:23:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202e3d4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2028111650 con 0x7f2028100440 2024-03-26T06:23:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202e3d4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2028111950 con 0x7f2028100e00 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202c971640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2028105190 0x7f20281979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202c971640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2028105190 0x7f20281979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50064/0 (socket says 172.21.15.37:50064) 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202c971640 1 -- 172.21.15.37:0/3494524593 learned_addr learned my addr 172.21.15.37:0/3494524593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f20277fe640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2028100e00 0x7f202819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f2027fff640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2028100440 0x7f202819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202c971640 1 -- 172.21.15.37:0/3494524593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2028100e00 msgr2=0x7f202819dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202c971640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2028100e00 0x7f202819dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202c971640 1 -- 172.21.15.37:0/3494524593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2028100440 msgr2=0x7f202819d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202c971640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2028100440 0x7f202819d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202c971640 1 -- 172.21.15.37:0/3494524593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028198170 con 0x7f2028105190 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f2027fff640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2028100440 0x7f202819d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f20277fe640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2028100e00 0x7f202819dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.689+0000 7f202c971640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2028105190 0x7f20281979f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f202002c700 tx=0x7f2020032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.690+0000 7f20257fa640 1 -- 172.21.15.37:0/3494524593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020042020 con 0x7f2028105190 2024-03-26T06:23:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.690+0000 7f202e3d4640 1 -- 172.21.15.37:0/3494524593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2028198400 con 0x7f2028105190 2024-03-26T06:23:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.690+0000 7f20257fa640 1 -- 172.21.15.37:0/3494524593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2020034640 con 0x7f2028105190 2024-03-26T06:23:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.690+0000 7f20257fa640 1 -- 172.21.15.37:0/3494524593 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020039040 con 0x7f2028105190 2024-03-26T06:23:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.690+0000 7f202e3d4640 1 -- 172.21.15.37:0/3494524593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20281a46b0 con 0x7f2028105190 2024-03-26T06:23:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.692+0000 7f2030fb8740 1 Processor -- start 2024-03-26T06:23:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.692+0000 7f20257fa640 1 -- 172.21.15.37:0/3494524593 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) v1 ==== 137466+0+0 (secure 0 0 0) 0x7f2020045020 con 0x7f2028105190 2024-03-26T06:23:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.692+0000 7f2030fb8740 1 -- start start 2024-03-26T06:23:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b6d40 0x557f110b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 0x557f110b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110c1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f2030fb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1101a5f0 con 0x557f110bf550 2024-03-26T06:23:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f2030fb8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f10fc3930 con 0x557f110b5fd0 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f2030fb8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f11016690 con 0x557f110b6d40 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f20277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f20277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110c1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50078/0 (socket says 172.21.15.37:50078) 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f20277fe640 1 -- 172.21.15.37:0/3086627551 learned_addr learned my addr 172.21.15.37:0/3086627551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f20277fe640 1 --2- 172.21.15.37:0/3086627551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b6d40 0x557f110b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f20257fa640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f20100a2ab0 0x7f20100a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f2027fff640 1 --2- 172.21.15.37:0/3086627551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 0x557f110b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f20277fe640 1 -- 172.21.15.37:0/3086627551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b6d40 msgr2=0x557f110b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f20277fe640 1 --2- 172.21.15.37:0/3086627551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b6d40 0x557f110b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f20277fe640 1 -- 172.21.15.37:0/3086627551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 msgr2=0x557f110b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f20277fe640 1 --2- 172.21.15.37:0/3086627551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 0x557f110b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f20277fe640 1 -- 172.21.15.37:0/3086627551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f10d2f210 con 0x557f110bf550 2024-03-26T06:23:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f20257fa640 1 -- 172.21.15.37:0/3494524593 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f20200f0f70 con 0x7f2028105190 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f202c971640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f20100a2ab0 0x7f20100a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.693+0000 7f2027fff640 1 --2- 172.21.15.37:0/3086627551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 0x557f110b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.694+0000 7f20277fe640 1 --2- 172.21.15.37:0/3086627551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110c1940 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f201c014a20 tx=0x7f201c00c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4a973c96f60b83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.694+0000 7f202c971640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f20100a2ab0 0x7f20100a4f70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20200ee7a0 tx=0x7f202002c0f0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.694+0000 7f200e7fc640 1 -- 172.21.15.37:0/3086627551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c0142d0 con 0x557f110bf550 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.694+0000 7f200e7fc640 1 -- 172.21.15.37:0/3086627551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f201c005500 con 0x557f110bf550 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.694+0000 7f200e7fc640 1 -- 172.21.15.37:0/3086627551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00f760 con 0x557f110bf550 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.695+0000 7f2030fb8740 1 -- 172.21.15.37:0/3086627551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 msgr2=0x557f110c1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.695+0000 7f2030fb8740 1 --2- 172.21.15.37:0/3086627551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110c1940 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f201c014a20 tx=0x7f201c00c540 comp rx=0 tx=0).stop 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.695+0000 7f2030fb8740 1 -- 172.21.15.37:0/3086627551 shutdown_connections 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.695+0000 7f2030fb8740 1 --2- 172.21.15.37:0/3086627551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110c1940 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.695+0000 7f2030fb8740 1 --2- 172.21.15.37:0/3086627551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 0x557f110b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.695+0000 7f2030fb8740 1 --2- 172.21.15.37:0/3086627551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b6d40 0x557f110b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.695+0000 7f2030fb8740 1 -- 172.21.15.37:0/3086627551 >> 172.21.15.37:0/3086627551 conn(0x557f1107d5b0 msgr2=0x557f110b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.695+0000 7f2030fb8740 1 -- 172.21.15.37:0/3086627551 shutdown_connections 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.695+0000 7f2030fb8740 1 -- 172.21.15.37:0/3086627551 wait complete. 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.695+0000 7f2030fb8740 1 -- start start 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f2030fb8740 1 Processor -- start 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b5fd0 0x557f110bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b6d40 0x557f110bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110bd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f2030fb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f110c6540 con 0x557f110bf550 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f2030fb8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f10fd6d90 con 0x557f110b6d40 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f2030fb8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f110c66c0 con 0x557f110b5fd0 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f2030fb8740 10 client.0 _ll_register_callbacks cb 0x557f10d2ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:23:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f20277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110bd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f20277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110bd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50084/0 (socket says 172.21.15.37:50084) 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f20277fe640 1 -- 172.21.15.37:0/1793882271 learned_addr learned my addr 172.21.15.37:0/1793882271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f2027fff640 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b6d40 0x557f110bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f20277fe640 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b5fd0 0x557f110bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f20277fe640 1 -- 172.21.15.37:0/1793882271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b5fd0 msgr2=0x557f110bc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f20277fe640 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b5fd0 0x557f110bc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f20277fe640 1 -- 172.21.15.37:0/1793882271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b6d40 msgr2=0x557f110bcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f20277fe640 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b6d40 0x557f110bcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.696+0000 7f20277fe640 1 -- 172.21.15.37:0/1793882271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f10dfb620 con 0x557f110bf550 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.697+0000 7f20277fe640 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110bd040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f201c00c2f0 tx=0x7f201c00c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f2a77cfbdb0dd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.697+0000 7f20077fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:23:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.697+0000 7f20077fe640 1 -- 172.21.15.37:0/1793882271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c005200 con 0x557f110bf550 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.697+0000 7f20077fe640 1 -- 172.21.15.37:0/1793882271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f201c00f730 con 0x557f110bf550 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.697+0000 7f2030fb8740 1 -- 172.21.15.37:0/1793882271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f201c005500 con 0x557f110bf550 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.697+0000 7f20077fe640 1 -- 172.21.15.37:0/1793882271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f201c00f930 con 0x557f110bf550 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.697+0000 7f2030fb8740 10 client.15555 fetch_fsmap learned FSMap version 27 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.697+0000 7f2030fb8740 1 -- 172.21.15.37:0/1793882271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f110d29b0 con 0x557f110bf550 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.697+0000 7f20077fe640 1 -- 172.21.15.37:0/1793882271 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f201c005500 con 0x557f110bf550 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.698+0000 7f20077fe640 1 -- 172.21.15.37:0/1793882271 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f201c02d8e0 con 0x557f110bf550 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.698+0000 7f2030fb8740 10 client.15555 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.698+0000 7f2030fb8740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.698+0000 7f2030fb8740 20 client.15555 populate_metadata read hostname 'smithi037' 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.698+0000 7f2030fb8740 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f110d3d60 0x557f110d6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.698+0000 7f2030fb8740 4 client.15555 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:23:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.698+0000 7f2030fb8740 1 -- 172.21.15.37:0/1793882271 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f11027b30 con 0x557f110d3d60 2024-03-26T06:23:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.698+0000 7f2027fff640 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f110d3d60 0x557f110d6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.698+0000 7f20077fe640 10 client.15555 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.698+0000 7f2027fff640 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f110d3d60 0x557f110d6150 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35046fdb18861ffc server_cookie=9609140103deb0bb in_seq=0 out_seq=0 2024-03-26T06:23:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.699+0000 7f20077fe640 1 -- 172.21.15.37:0/1793882271 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f11027b30 con 0x557f110d3d60 2024-03-26T06:23:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.699+0000 7f20077fe640 10 client.15555 handle_command_reply: tid=0 2024-03-26T06:23:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.699+0000 7f2030fb8740 1 client.15555 shutdown 2024-03-26T06:23:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.699+0000 7f2030fb8740 1 -- 172.21.15.37:0/1793882271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 msgr2=0x557f110bd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.699+0000 7f2030fb8740 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110bd040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f201c00c2f0 tx=0x7f201c00c480 comp rx=0 tx=0).stop 2024-03-26T06:23:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.700+0000 7f2030fb8740 1 -- 172.21.15.37:0/1793882271 shutdown_connections 2024-03-26T06:23:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.700+0000 7f2030fb8740 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f110d3d60 0x557f110d6150 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.700+0000 7f20077fe640 0 client.15555 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.700+0000 7f2030fb8740 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f110bf550 0x557f110bd040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.700+0000 7f2030fb8740 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b6d40 0x557f110bcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.700+0000 7f2030fb8740 1 --2- 172.21.15.37:0/1793882271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b5fd0 0x557f110bc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.700+0000 7f2030fb8740 1 -- 172.21.15.37:0/1793882271 >> 172.21.15.37:0/1793882271 conn(0x557f1107d5b0 msgr2=0x557f110b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.700+0000 7f2030fb8740 1 -- 172.21.15.37:0/1793882271 shutdown_connections 2024-03-26T06:23:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.700+0000 7f2030fb8740 1 -- 172.21.15.37:0/1793882271 wait complete. 2024-03-26T06:23:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.700+0000 7f2030fb8740 20 client.15555 trim_cache size 0 max 16384 2024-03-26T06:23:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.723+0000 7f2030fb8740 1 Processor -- start 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.723+0000 7f2030fb8740 1 -- start start 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.723+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110becf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.723+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 0x557f110baeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.723+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b6d40 0x557f110bb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.723+0000 7f2030fb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20281aa990 con 0x557f1107d5b0 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.723+0000 7f2030fb8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20281ac310 con 0x557f110b5fd0 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.723+0000 7f2030fb8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20281aaff0 con 0x557f110b6d40 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.723+0000 7f20277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110becf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.723+0000 7f20277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110becf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50096/0 (socket says 172.21.15.37:50096) 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f20277fe640 1 -- 172.21.15.37:0/3823462354 learned_addr learned my addr 172.21.15.37:0/3823462354 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f2027fff640 1 --2- 172.21.15.37:0/3823462354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 0x557f110baeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f20277fe640 1 --2- 172.21.15.37:0/3823462354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b6d40 0x557f110bb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f20277fe640 1 -- 172.21.15.37:0/3823462354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b6d40 msgr2=0x557f110bb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f20277fe640 1 --2- 172.21.15.37:0/3823462354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b6d40 0x557f110bb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f20277fe640 1 -- 172.21.15.37:0/3823462354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 msgr2=0x557f110baeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f20277fe640 1 --2- 172.21.15.37:0/3823462354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 0x557f110baeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f20277fe640 1 -- 172.21.15.37:0/3823462354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f110c6480 con 0x557f1107d5b0 2024-03-26T06:23:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f2027fff640 1 --2- 172.21.15.37:0/3823462354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 0x557f110baeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:23:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f20277fe640 1 --2- 172.21.15.37:0/3823462354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110becf0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f201c02dcb0 tx=0x7f201c012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2331784efcdddb3c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f200dffb640 1 -- 172.21.15.37:0/3823462354 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c014690 con 0x557f1107d5b0 2024-03-26T06:23:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f200dffb640 1 -- 172.21.15.37:0/3823462354 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f201c0142e0 con 0x557f1107d5b0 2024-03-26T06:23:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.724+0000 7f200dffb640 1 -- 172.21.15.37:0/3823462354 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c02d460 con 0x557f1107d5b0 2024-03-26T06:23:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.725+0000 7f2030fb8740 1 -- 172.21.15.37:0/3823462354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 msgr2=0x557f110becf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.725+0000 7f2030fb8740 1 --2- 172.21.15.37:0/3823462354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110becf0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f201c02dcb0 tx=0x7f201c012450 comp rx=0 tx=0).stop 2024-03-26T06:23:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.725+0000 7f2030fb8740 1 -- 172.21.15.37:0/3823462354 shutdown_connections 2024-03-26T06:23:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.725+0000 7f2030fb8740 1 --2- 172.21.15.37:0/3823462354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b6d40 0x557f110bb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.725+0000 7f2030fb8740 1 --2- 172.21.15.37:0/3823462354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b5fd0 0x557f110baeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.725+0000 7f2030fb8740 1 --2- 172.21.15.37:0/3823462354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110becf0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.725+0000 7f2030fb8740 1 -- 172.21.15.37:0/3823462354 >> 172.21.15.37:0/3823462354 conn(0x557f110d64c0 msgr2=0x557f110b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.725+0000 7f2030fb8740 1 -- 172.21.15.37:0/3823462354 shutdown_connections 2024-03-26T06:23:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.725+0000 7f2030fb8740 1 -- 172.21.15.37:0/3823462354 wait complete. 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f2030fb8740 1 -- start start 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f2030fb8740 1 Processor -- start 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b5fd0 0x557f110c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f2030fb8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b6d40 0x557f1112a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f2030fb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20281abcb0 con 0x557f1107d5b0 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f2030fb8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20281ab650 con 0x557f110b6d40 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f2030fb8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20281a8c30 con 0x557f110b5fd0 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f2030fb8740 10 client.0 _ll_register_callbacks cb 0x7f201c00cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f20277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f20277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50108/0 (socket says 172.21.15.37:50108) 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f20277fe640 1 -- 172.21.15.37:0/2922979209 learned_addr learned my addr 172.21.15.37:0/2922979209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.726+0000 7f2027fff640 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b5fd0 0x557f110c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f20277fe640 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b6d40 0x557f1112a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f20277fe640 1 -- 172.21.15.37:0/2922979209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b5fd0 msgr2=0x557f110c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f20277fe640 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b5fd0 0x557f110c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f20277fe640 1 -- 172.21.15.37:0/2922979209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b6d40 msgr2=0x557f1112a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f20277fe640 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b6d40 0x557f1112a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f20277fe640 1 -- 172.21.15.37:0/2922979209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f110b52a0 con 0x557f1107d5b0 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f2027fff640 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b5fd0 0x557f110c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f20277fe640 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110c9880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f201c006010 tx=0x7f201c014490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78aed346d8a82b8f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f20077fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f20077fe640 1 -- 172.21.15.37:0/2922979209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c014690 con 0x557f1107d5b0 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f20077fe640 1 -- 172.21.15.37:0/2922979209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f201c00f730 con 0x557f1107d5b0 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f2030fb8740 1 -- 172.21.15.37:0/2922979209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f201c0142e0 con 0x557f1107d5b0 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f20077fe640 1 -- 172.21.15.37:0/2922979209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f201c004c00 con 0x557f1107d5b0 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f2030fb8740 10 client.15561 fetch_fsmap learned FSMap version 27 2024-03-26T06:23:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.727+0000 7f2030fb8740 1 -- 172.21.15.37:0/2922979209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f1112b670 con 0x557f1107d5b0 2024-03-26T06:23:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.728+0000 7f20077fe640 1 -- 172.21.15.37:0/2922979209 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f201c02d460 con 0x557f1107d5b0 2024-03-26T06:23:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.728+0000 7f20077fe640 1 -- 172.21.15.37:0/2922979209 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f201c012600 con 0x557f1107d5b0 2024-03-26T06:23:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.728+0000 7f2030fb8740 10 client.15561 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:23:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.728+0000 7f2030fb8740 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:23:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.728+0000 7f2030fb8740 20 client.15561 populate_metadata read hostname 'smithi037' 2024-03-26T06:23:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.728+0000 7f2030fb8740 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f110bf550 0x557f1112da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:23:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.728+0000 7f2030fb8740 4 client.15561 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:23:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.728+0000 7f2030fb8740 1 -- 172.21.15.37:0/2922979209 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f201c0142e0 con 0x557f110bf550 2024-03-26T06:23:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.728+0000 7f2027fff640 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f110bf550 0x557f1112da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:23:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.729+0000 7f2027fff640 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f110bf550 0x557f1112da50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1f9182dc96796b server_cookie=b7ba986497ee4433 in_seq=0 out_seq=0 2024-03-26T06:23:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.729+0000 7f20077fe640 10 client.15561 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.732+0000 7f20077fe640 1 -- 172.21.15.37:0/2922979209 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f201c0142e0 con 0x557f110bf550 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.732+0000 7f20077fe640 10 client.15561 handle_command_reply: tid=0 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.732+0000 7f2030fb8740 1 client.15561 shutdown 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f2030fb8740 1 -- 172.21.15.37:0/2922979209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 msgr2=0x557f110c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f2030fb8740 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110c9880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f201c006010 tx=0x7f201c014490 comp rx=0 tx=0).stop 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f2030fb8740 1 -- 172.21.15.37:0/2922979209 shutdown_connections 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f2030fb8740 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f110bf550 0x557f1112da50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f2030fb8740 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f110b6d40 0x557f1112a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f2030fb8740 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f110b5fd0 0x557f110c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f2030fb8740 1 --2- 172.21.15.37:0/2922979209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f1107d5b0 0x557f110c9880 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f2030fb8740 1 -- 172.21.15.37:0/2922979209 >> 172.21.15.37:0/2922979209 conn(0x557f110d64c0 msgr2=0x557f110bae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f20077fe640 0 client.15561 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:23:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f2030fb8740 1 -- 172.21.15.37:0/2922979209 shutdown_connections 2024-03-26T06:23:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f2030fb8740 1 -- 172.21.15.37:0/2922979209 wait complete. 2024-03-26T06:23:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f2030fb8740 20 client.15561 trim_cache size 0 max 16384 2024-03-26T06:23:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:23:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:23:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:23:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:23:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f202e3d4640 1 -- 172.21.15.37:0/3494524593 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f20100a2ab0 msgr2=0x7f20100a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f202e3d4640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f20100a2ab0 0x7f20100a4f70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20200ee7a0 tx=0x7f202002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f202e3d4640 1 -- 172.21.15.37:0/3494524593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2028105190 msgr2=0x7f20281979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:23:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.733+0000 7f202e3d4640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2028105190 0x7f20281979f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f202002c700 tx=0x7f2020032f20 comp rx=0 tx=0).stop 2024-03-26T06:23:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.734+0000 7f202e3d4640 1 -- 172.21.15.37:0/3494524593 shutdown_connections 2024-03-26T06:23:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.734+0000 7f202e3d4640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f20100a2ab0 0x7f20100a4f70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.734+0000 7f202e3d4640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2028105190 0x7f20281979f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.734+0000 7f202e3d4640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2028100e00 0x7f202819dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.734+0000 7f202e3d4640 1 --2- 172.21.15.37:0/3494524593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2028100440 0x7f202819d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:23:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.734+0000 7f202e3d4640 1 -- 172.21.15.37:0/3494524593 >> 172.21.15.37:0/3494524593 conn(0x7f2028077040 msgr2=0x7f20280fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:23:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.734+0000 7f202e3d4640 1 -- 172.21.15.37:0/3494524593 shutdown_connections 2024-03-26T06:23:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:23:53.734+0000 7f202e3d4640 1 -- 172.21.15.37:0/3494524593 wait complete. 2024-03-26T06:23:54.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:54 smithi149 ceph-mon[36681]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-26T06:23:54.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:54 smithi037 ceph-mon[31974]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-26T06:23:55.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:54 smithi066 ceph-mon[36504]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-26T06:23:56.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:56 smithi149 ceph-mon[36681]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-03-26T06:23:56.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:56 smithi037 ceph-mon[31974]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-03-26T06:23:57.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:56 smithi066 ceph-mon[36504]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-03-26T06:23:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:23:58 smithi149 ceph-mon[36681]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-26T06:23:58.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:23:58 smithi037 ceph-mon[31974]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-26T06:23:59.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:23:58 smithi066 ceph-mon[36504]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-26T06:24:00.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:00 smithi037 ceph-mon[31974]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:00.610 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:24:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:24:00.254+0000 7fbbd47bf700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:24:00.610 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:24:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:24:00.254+0000 7fbbd47bf700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:24:00.610 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:24:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:24:00.254+0000 7fbbd47bf700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:24:00.610 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:24:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:24:00.254+0000 7fbbd47bf700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:24:00.610 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:24:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:24:00.254+0000 7fbbd47bf700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T06:24:00.900 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:00 smithi066 ceph-mon[36504]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:00 smithi149 ceph-mon[36681]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:02 smithi149 ceph-mon[36681]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-26T06:24:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:02 smithi149 ceph-mon[36681]: mgrmap e25: x(active, since 62s), standbys: z, y 2024-03-26T06:24:02.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:02 smithi037 ceph-mon[31974]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-26T06:24:02.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:02 smithi037 ceph-mon[31974]: mgrmap e25: x(active, since 62s), standbys: z, y 2024-03-26T06:24:03.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:02 smithi066 ceph-mon[36504]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-26T06:24:03.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:02 smithi066 ceph-mon[36504]: mgrmap e25: x(active, since 62s), standbys: z, y 2024-03-26T06:24:03.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:03 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:24:03.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:03 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:24:04.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:03 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:24:04.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:04 smithi149 ceph-mon[36681]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:04.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:04 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:24:04.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:04 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:24:04.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:04 smithi037 ceph-mon[31974]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:04.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:04 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:24:04.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:04 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:24:05.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:04 smithi066 ceph-mon[36504]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:05.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:04 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:24:05.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:04 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:24:05.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:05 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:24:05.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:05 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:24:05.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:05 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:24:05.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:05 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:24:05.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:05 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:24:05.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:05 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:24:06.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:05 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:24:06.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:05 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:24:06.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:05 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:24:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.823+0000 7f92e92ea640 1 -- 172.21.15.37:0/1610475482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e410dff0 msgr2=0x7f92e41103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.823+0000 7f92e92ea640 1 --2- 172.21.15.37:0/1610475482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e410dff0 0x7f92e41103e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009ca0 tx=0x7f92dc02c180 comp rx=0 tx=0).stop 2024-03-26T06:24:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.823+0000 7f92e92ea640 1 -- 172.21.15.37:0/1610475482 shutdown_connections 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.823+0000 7f92e92ea640 1 --2- 172.21.15.37:0/1610475482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e410dff0 0x7f92e41103e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.823+0000 7f92e92ea640 1 --2- 172.21.15.37:0/1610475482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92e4069290 0x7f92e410dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.823+0000 7f92e92ea640 1 --2- 172.21.15.37:0/1610475482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92e4068970 0x7f92e4068d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.823+0000 7f92e92ea640 1 -- 172.21.15.37:0/1610475482 >> 172.21.15.37:0/1610475482 conn(0x7f92e4077040 msgr2=0x7f92e4077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.823+0000 7f92e92ea640 1 -- 172.21.15.37:0/1610475482 shutdown_connections 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.823+0000 7f92e92ea640 1 -- 172.21.15.37:0/1610475482 wait complete. 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.823+0000 7f92e92ea640 1 Processor -- start 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e92ea640 1 -- start start 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e92ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e4068970 0x7f92e419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e92ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92e4069290 0x7f92e419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e92ea640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92e410dff0 0x7f92e4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e92ea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4113890 con 0x7f92e4068970 2024-03-26T06:24:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e92ea640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4113710 con 0x7f92e4069290 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e92ea640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4113a10 con 0x7f92e410dff0 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e4068970 0x7f92e419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e4068970 0x7f92e419f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51806/0 (socket says 172.21.15.37:51806) 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e2ffd640 1 -- 172.21.15.37:0/1901246580 learned_addr learned my addr 172.21.15.37:0/1901246580 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e27fc640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92e4069290 0x7f92e419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.824+0000 7f92e37fe640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92e410dff0 0x7f92e4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92e2ffd640 1 -- 172.21.15.37:0/1901246580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92e410dff0 msgr2=0x7f92e4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92e410dff0 0x7f92e4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92e2ffd640 1 -- 172.21.15.37:0/1901246580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92e4069290 msgr2=0x7f92e419fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92e4069290 0x7f92e419fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92e2ffd640 1 -- 172.21.15.37:0/1901246580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e419a230 con 0x7f92e4068970 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92e27fc640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92e4069290 0x7f92e419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:24:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92e37fe640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92e410dff0 0x7f92e4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:24:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e4068970 0x7f92e419f8b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f92d400d770 tx=0x7f92d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92cbfff640 1 -- 172.21.15.37:0/1901246580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4017070 con 0x7f92e4068970 2024-03-26T06:24:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92e92ea640 1 -- 172.21.15.37:0/1901246580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e419a520 con 0x7f92e4068970 2024-03-26T06:24:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92cbfff640 1 -- 172.21.15.37:0/1901246580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92d4005500 con 0x7f92e4068970 2024-03-26T06:24:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92e92ea640 1 -- 172.21.15.37:0/1901246580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e41a6770 con 0x7f92e4068970 2024-03-26T06:24:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.825+0000 7f92cbfff640 1 -- 172.21.15.37:0/1901246580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4004e00 con 0x7f92e4068970 2024-03-26T06:24:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.828+0000 7f92ebfa4740 1 Processor -- start 2024-03-26T06:24:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.828+0000 7f92ebfa4740 1 -- start start 2024-03-26T06:24:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.828+0000 7f92cbfff640 1 -- 172.21.15.37:0/1901246580 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f92d4010460 con 0x7f92e4068970 2024-03-26T06:24:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.828+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4697d40 0x558bd4696a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.828+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4696fd0 0x558bd4697450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.828+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd46a2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.828+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd45fb5f0 con 0x558bd46a0550 2024-03-26T06:24:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.828+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd45a4930 con 0x558bd4697d40 2024-03-26T06:24:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.828+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd45f7690 con 0x558bd4696fd0 2024-03-26T06:24:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.828+0000 7f92e27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd46a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92e27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd46a2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51822/0 (socket says 172.21.15.37:51822) 2024-03-26T06:24:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92e27fc640 1 -- 172.21.15.37:0/1579869500 learned_addr learned my addr 172.21.15.37:0/1579869500 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/1579869500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4696fd0 0x558bd4697450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92e27fc640 1 -- 172.21.15.37:0/1579869500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4696fd0 msgr2=0x558bd4697450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92e27fc640 1 --2- 172.21.15.37:0/1579869500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4696fd0 0x558bd4697450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92e27fc640 1 -- 172.21.15.37:0/1579869500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4697d40 msgr2=0x558bd4696a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92e27fc640 1 --2- 172.21.15.37:0/1579869500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4697d40 0x558bd4696a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92e27fc640 1 -- 172.21.15.37:0/1579869500 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bd4310210 con 0x558bd46a0550 2024-03-26T06:24:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/1579869500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4696fd0 0x558bd4697450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:24:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92e27fc640 1 --2- 172.21.15.37:0/1579869500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd46a2940 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f92d80114f0 tx=0x7f92d80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4970d71ffa4d22 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92c97fa640 1 -- 172.21.15.37:0/1579869500 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8005020 con 0x558bd46a0550 2024-03-26T06:24:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92cbfff640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f92cc0a2b30 0x7f92cc0a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92c97fa640 1 -- 172.21.15.37:0/1579869500 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92d80047a0 con 0x558bd46a0550 2024-03-26T06:24:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.829+0000 7f92e37fe640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f92cc0a2b30 0x7f92cc0a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92cbfff640 1 -- 172.21.15.37:0/1901246580 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f92d40ceba0 con 0x7f92e4068970 2024-03-26T06:24:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92c97fa640 1 -- 172.21.15.37:0/1579869500 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d800f970 con 0x558bd46a0550 2024-03-26T06:24:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92ebfa4740 1 -- 172.21.15.37:0/1579869500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 msgr2=0x558bd46a2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92e37fe640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f92cc0a2b30 0x7f92cc0a4ff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f92dc006010 tx=0x7f92dc004a40 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/1579869500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd46a2940 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f92d80114f0 tx=0x7f92d80119b0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92ebfa4740 1 -- 172.21.15.37:0/1579869500 shutdown_connections 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/1579869500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd46a2940 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/1579869500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4696fd0 0x558bd4697450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/1579869500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4697d40 0x558bd4696a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92ebfa4740 1 -- 172.21.15.37:0/1579869500 >> 172.21.15.37:0/1579869500 conn(0x558bd465e5b0 msgr2=0x558bd4691cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92ebfa4740 1 -- 172.21.15.37:0/1579869500 shutdown_connections 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.830+0000 7f92ebfa4740 1 -- 172.21.15.37:0/1579869500 wait complete. 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92ebfa4740 1 -- start start 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92ebfa4740 1 Processor -- start 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4697d40 0x558bd469d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd46a0550 0x558bd469df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd46a3320 con 0x558bd4696fd0 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd45b7d90 con 0x558bd46a0550 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd46a34a0 con 0x558bd4697d40 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92e2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92ebfa4740 10 client.0 _ll_register_callbacks cb 0x558bd430fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92e2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51838/0 (socket says 172.21.15.37:51838) 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92e2ffd640 1 -- 172.21.15.37:0/3759678781 learned_addr learned my addr 172.21.15.37:0/3759678781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92e37fe640 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd46a0550 0x558bd469df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92e27fc640 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4697d40 0x558bd469d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92e2ffd640 1 -- 172.21.15.37:0/3759678781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4697d40 msgr2=0x558bd469d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4697d40 0x558bd469d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92e2ffd640 1 -- 172.21.15.37:0/3759678781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd46a0550 msgr2=0x558bd469df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd46a0550 0x558bd469df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.831+0000 7f92e2ffd640 1 -- 172.21.15.37:0/3759678781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bd43dc620 con 0x558bd4696fd0 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.832+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469d4a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f92d400d740 tx=0x7f92d40994f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cd6b2e993b727b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.832+0000 7f92c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.832+0000 7f92c6ff5640 1 -- 172.21.15.37:0/3759678781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4020070 con 0x558bd4696fd0 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.832+0000 7f92c6ff5640 1 -- 172.21.15.37:0/3759678781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f92d40cc480 con 0x558bd4696fd0 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.832+0000 7f92ebfa4740 1 -- 172.21.15.37:0/3759678781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92d80047a0 con 0x558bd4696fd0 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.832+0000 7f92c6ff5640 1 -- 172.21.15.37:0/3759678781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92d40046c0 con 0x558bd4696fd0 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.832+0000 7f92ebfa4740 10 client.15579 fetch_fsmap learned FSMap version 27 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.832+0000 7f92ebfa4740 1 -- 172.21.15.37:0/3759678781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558bd46af5e0 con 0x558bd4696fd0 2024-03-26T06:24:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.832+0000 7f92c6ff5640 1 -- 172.21.15.37:0/3759678781 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92d80047a0 con 0x558bd4696fd0 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.833+0000 7f92c6ff5640 1 -- 172.21.15.37:0/3759678781 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f92d4095b50 con 0x558bd4696fd0 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.833+0000 7f92ebfa4740 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.833+0000 7f92ebfa4740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.833+0000 7f92ebfa4740 20 client.15579 populate_metadata read hostname 'smithi037' 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.833+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558bd46b0aa0 0x558bd46b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.833+0000 7f92ebfa4740 4 client.15579 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.833+0000 7f92ebfa4740 1 -- 172.21.15.37:0/3759678781 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bd4608b30 con 0x558bd46b0aa0 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.833+0000 7f92e27fc640 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558bd46b0aa0 0x558bd46b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.834+0000 7f92c6ff5640 10 client.15579 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.834+0000 7f92e27fc640 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558bd46b0aa0 0x558bd46b2e90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=608d1aa99d701cc7 server_cookie=304d970aaab8bd61 in_seq=0 out_seq=0 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.834+0000 7f92c6ff5640 1 -- 172.21.15.37:0/3759678781 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558bd4608b30 con 0x558bd46b0aa0 2024-03-26T06:24:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.834+0000 7f92c6ff5640 10 client.15579 handle_command_reply: tid=0 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.834+0000 7f92ebfa4740 1 client.15579 shutdown 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.834+0000 7f92ebfa4740 1 -- 172.21.15.37:0/3759678781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 msgr2=0x558bd469d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.834+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469d4a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f92d400d740 tx=0x7f92d40994f0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.834+0000 7f92ebfa4740 1 -- 172.21.15.37:0/3759678781 shutdown_connections 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.834+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558bd46b0aa0 0x558bd46b2e90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.835+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd46a0550 0x558bd469df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.835+0000 7f92c6ff5640 0 client.15579 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.835+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4697d40 0x558bd469d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.835+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/3759678781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469d4a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.835+0000 7f92ebfa4740 1 -- 172.21.15.37:0/3759678781 >> 172.21.15.37:0/3759678781 conn(0x558bd465e5b0 msgr2=0x558bd4693a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.835+0000 7f92ebfa4740 1 -- 172.21.15.37:0/3759678781 shutdown_connections 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.835+0000 7f92ebfa4740 1 -- 172.21.15.37:0/3759678781 wait complete. 2024-03-26T06:24:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.835+0000 7f92ebfa4740 20 client.15579 trim_cache size 0 max 16384 2024-03-26T06:24:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.848+0000 7f92ebfa4740 1 Processor -- start 2024-03-26T06:24:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.848+0000 7f92ebfa4740 1 -- start start 2024-03-26T06:24:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4697d40 0x558bd469c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd46a0550 0x558bd469a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41aed50 con 0x558bd4696fd0 2024-03-26T06:24:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41b06d0 con 0x558bd4697d40 2024-03-26T06:24:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41af3b0 con 0x558bd46a0550 2024-03-26T06:24:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92e2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92e2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51840/0 (socket says 172.21.15.37:51840) 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92e2ffd640 1 -- 172.21.15.37:0/811958231 learned_addr learned my addr 172.21.15.37:0/811958231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92e37fe640 1 --2- 172.21.15.37:0/811958231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd46a0550 0x558bd469a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92e2ffd640 1 -- 172.21.15.37:0/811958231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd46a0550 msgr2=0x558bd469a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/811958231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd46a0550 0x558bd469a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92e2ffd640 1 -- 172.21.15.37:0/811958231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4697d40 msgr2=0x558bd469c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/811958231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4697d40 0x558bd469c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92e2ffd640 1 -- 172.21.15.37:0/811958231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bd46b3200 con 0x558bd4696fd0 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/811958231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469fbd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f92d40979b0 tx=0x7f92d40048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f214eae8ae6ba5c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.849+0000 7f92c8ff9640 1 -- 172.21.15.37:0/811958231 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4020040 con 0x558bd4696fd0 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.850+0000 7f92c8ff9640 1 -- 172.21.15.37:0/811958231 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92d407d020 con 0x558bd4696fd0 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.850+0000 7f92c8ff9640 1 -- 172.21.15.37:0/811958231 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40a3780 con 0x558bd4696fd0 2024-03-26T06:24:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.850+0000 7f92ebfa4740 1 -- 172.21.15.37:0/811958231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 msgr2=0x558bd469fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.850+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/811958231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469fbd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f92d40979b0 tx=0x7f92d40048a0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.850+0000 7f92ebfa4740 1 -- 172.21.15.37:0/811958231 shutdown_connections 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.850+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/811958231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd46a0550 0x558bd469a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.850+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/811958231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4697d40 0x558bd469c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.850+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/811958231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd4696fd0 0x558bd469fbd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.850+0000 7f92ebfa4740 1 -- 172.21.15.37:0/811958231 >> 172.21.15.37:0/811958231 conn(0x558bd465e5b0 msgr2=0x558bd469bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.850+0000 7f92ebfa4740 1 -- 172.21.15.37:0/811958231 shutdown_connections 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.850+0000 7f92ebfa4740 1 -- 172.21.15.37:0/811958231 wait complete. 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92ebfa4740 1 -- start start 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92ebfa4740 1 Processor -- start 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4696fd0 0x558bd46a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4697d40 0x558bd46a6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92ebfa4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd4706f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41b0070 con 0x558bd46a0550 2024-03-26T06:24:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41afa10 con 0x558bd4696fd0 2024-03-26T06:24:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92ebfa4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41acff0 con 0x558bd4697d40 2024-03-26T06:24:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92ebfa4740 10 client.0 _ll_register_callbacks cb 0x7f92d8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:24:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92e37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd4706f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92e37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd4706f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51852/0 (socket says 172.21.15.37:51852) 2024-03-26T06:24:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92e37fe640 1 -- 172.21.15.37:0/2598524552 learned_addr learned my addr 172.21.15.37:0/2598524552 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4696fd0 0x558bd46a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.851+0000 7f92e27fc640 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4697d40 0x558bd46a6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92e2ffd640 1 -- 172.21.15.37:0/2598524552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4697d40 msgr2=0x558bd46a6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4697d40 0x558bd46a6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92e2ffd640 1 -- 172.21.15.37:0/2598524552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 msgr2=0x558bd4706f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd4706f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92e2ffd640 1 -- 172.21.15.37:0/2598524552 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bd4707490 con 0x558bd4696fd0 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92e37fe640 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd4706f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92e2ffd640 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4696fd0 0x558bd46a67d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92d40cc920 tx=0x7f92d40994f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0104b313b33978b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92c6ff5640 1 -- 172.21.15.37:0/2598524552 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4020070 con 0x558bd4696fd0 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92ebfa4740 1 -- 172.21.15.37:0/2598524552 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92d407d020 con 0x558bd4696fd0 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92c6ff5640 1 -- 172.21.15.37:0/2598524552 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f92d4079e00 con 0x558bd4696fd0 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92c6ff5640 1 -- 172.21.15.37:0/2598524552 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92d40aa050 con 0x558bd4696fd0 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92ebfa4740 10 client.25166 fetch_fsmap learned FSMap version 27 2024-03-26T06:24:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.852+0000 7f92ebfa4740 1 -- 172.21.15.37:0/2598524552 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558bd47086a0 con 0x558bd4696fd0 2024-03-26T06:24:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.853+0000 7f92c6ff5640 1 -- 172.21.15.37:0/2598524552 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92d407d020 con 0x558bd4696fd0 2024-03-26T06:24:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.853+0000 7f92c6ff5640 1 -- 172.21.15.37:0/2598524552 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f92d40840b0 con 0x558bd4696fd0 2024-03-26T06:24:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.853+0000 7f92ebfa4740 10 client.25166 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:24:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.853+0000 7f92ebfa4740 10 client.25166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:24:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.853+0000 7f92ebfa4740 20 client.25166 populate_metadata read hostname 'smithi037' 2024-03-26T06:24:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.853+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558bd47089e0 0x558bd470add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.853+0000 7f92ebfa4740 4 client.25166 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:24:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.853+0000 7f92ebfa4740 1 -- 172.21.15.37:0/2598524552 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558bd4602e80 con 0x558bd47089e0 2024-03-26T06:24:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.853+0000 7f92e27fc640 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558bd47089e0 0x558bd470add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.853+0000 7f92c6ff5640 10 client.25166 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.853+0000 7f92e27fc640 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558bd47089e0 0x558bd470add0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234a9c9061af991f server_cookie=40f8c42e144d888a in_seq=0 out_seq=0 2024-03-26T06:24:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92c6ff5640 1 -- 172.21.15.37:0/2598524552 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558bd4602e80 con 0x558bd47089e0 2024-03-26T06:24:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92c6ff5640 10 client.25166 handle_command_reply: tid=0 2024-03-26T06:24:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 1 client.25166 shutdown 2024-03-26T06:24:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 1 -- 172.21.15.37:0/2598524552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4696fd0 msgr2=0x558bd46a67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4696fd0 0x558bd46a67d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92d40cc920 tx=0x7f92d40994f0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 1 -- 172.21.15.37:0/2598524552 shutdown_connections 2024-03-26T06:24:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558bd47089e0 0x558bd470add0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558bd46a0550 0x558bd4706f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92c6ff5640 0 client.25166 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bd4697d40 0x558bd46a6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 1 --2- 172.21.15.37:0/2598524552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bd4696fd0 0x558bd46a67d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 1 -- 172.21.15.37:0/2598524552 >> 172.21.15.37:0/2598524552 conn(0x558bd465e5b0 msgr2=0x558bd469c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 1 -- 172.21.15.37:0/2598524552 shutdown_connections 2024-03-26T06:24:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 1 -- 172.21.15.37:0/2598524552 wait complete. 2024-03-26T06:24:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.860+0000 7f92ebfa4740 20 client.25166 trim_cache size 0 max 16384 2024-03-26T06:24:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:24:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:24:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:24:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:24:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 -- 172.21.15.37:0/1901246580 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f92cc0a2b30 msgr2=0x7f92cc0a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f92cc0a2b30 0x7f92cc0a4ff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f92dc006010 tx=0x7f92dc004a40 comp rx=0 tx=0).stop 2024-03-26T06:24:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 -- 172.21.15.37:0/1901246580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e4068970 msgr2=0x7f92e419f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e4068970 0x7f92e419f8b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f92d400d770 tx=0x7f92d400dc30 comp rx=0 tx=0).stop 2024-03-26T06:24:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 -- 172.21.15.37:0/1901246580 shutdown_connections 2024-03-26T06:24:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f92cc0a2b30 0x7f92cc0a4ff0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92e410dff0 0x7f92e4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92e4069290 0x7f92e419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 --2- 172.21.15.37:0/1901246580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e4068970 0x7f92e419f8b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 -- 172.21.15.37:0/1901246580 >> 172.21.15.37:0/1901246580 conn(0x7f92e4077040 msgr2=0x7f92e410f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 -- 172.21.15.37:0/1901246580 shutdown_connections 2024-03-26T06:24:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:06.861+0000 7f92e92ea640 1 -- 172.21.15.37:0/1901246580 wait complete. 2024-03-26T06:24:06.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:06 smithi149 ceph-mon[36681]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-26T06:24:06.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:06 smithi037 ceph-mon[31974]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-26T06:24:07.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:06 smithi066 ceph-mon[36504]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-26T06:24:08.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:08 smithi149 ceph-mon[36681]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-26T06:24:08.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:08 smithi037 ceph-mon[31974]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-26T06:24:09.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:08 smithi066 ceph-mon[36504]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-26T06:24:10.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:10 smithi149 ceph-mon[36681]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-26T06:24:10.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:10 smithi037 ceph-mon[31974]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-26T06:24:11.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:10 smithi066 ceph-mon[36504]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-26T06:24:12.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:12 smithi149 ceph-mon[36681]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-03-26T06:24:12.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:12 smithi037 ceph-mon[31974]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-03-26T06:24:13.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:12 smithi066 ceph-mon[36504]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-03-26T06:24:14.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:14 smithi149 ceph-mon[36681]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-26T06:24:14.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:14 smithi037 ceph-mon[31974]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-26T06:24:15.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:14 smithi066 ceph-mon[36504]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-26T06:24:16.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:16 smithi149 ceph-mon[36681]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-03-26T06:24:16.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:16 smithi037 ceph-mon[31974]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-03-26T06:24:17.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:16 smithi066 ceph-mon[36504]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-03-26T06:24:18.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:18 smithi149 ceph-mon[36681]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-26T06:24:18.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:18 smithi037 ceph-mon[31974]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-26T06:24:19.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:18 smithi066 ceph-mon[36504]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-26T06:24:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:19.999+0000 7f0732e0b640 1 -- 172.21.15.37:0/2180865906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c100e00 msgr2=0x7f072c10d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:19.999+0000 7f0732e0b640 1 --2- 172.21.15.37:0/2180865906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c100e00 0x7f072c10d020 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0724009ca0 tx=0x7f072402c0f0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.000+0000 7f0732e0b640 1 -- 172.21.15.37:0/2180865906 shutdown_connections 2024-03-26T06:24:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.000+0000 7f0732e0b640 1 --2- 172.21.15.37:0/2180865906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c100e00 0x7f072c10d020 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.000+0000 7f0732e0b640 1 --2- 172.21.15.37:0/2180865906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f072c100440 0x7f072c1008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.000+0000 7f0732e0b640 1 --2- 172.21.15.37:0/2180865906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f072c105190 0x7f072c105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.000+0000 7f0732e0b640 1 -- 172.21.15.37:0/2180865906 >> 172.21.15.37:0/2180865906 conn(0x7f072c077040 msgr2=0x7f072c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.000+0000 7f0732e0b640 1 -- 172.21.15.37:0/2180865906 shutdown_connections 2024-03-26T06:24:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.000+0000 7f0732e0b640 1 -- 172.21.15.37:0/2180865906 wait complete. 2024-03-26T06:24:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.001+0000 7f0732e0b640 1 Processor -- start 2024-03-26T06:24:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.001+0000 7f0732e0b640 1 -- start start 2024-03-26T06:24:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.001+0000 7f0732e0b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f072c100440 0x7f072c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.001+0000 7f0732e0b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c100e00 0x7f072c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.001+0000 7f0732e0b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f072c105190 0x7f072c196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.001+0000 7f072bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c100e00 0x7f072c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f0730ba7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f072c100440 0x7f072c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f072bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c100e00 0x7f072c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56978/0 (socket says 172.21.15.37:56978) 2024-03-26T06:24:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f0730ba7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f072c100440 0x7f072c19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:35304/0 (socket says 172.21.15.37:35304) 2024-03-26T06:24:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f072bfff640 1 -- 172.21.15.37:0/597120277 learned_addr learned my addr 172.21.15.37:0/597120277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f0732e0b640 1 -- 172.21.15.37:0/597120277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c110650 con 0x7f072c100e00 2024-03-26T06:24:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f0732e0b640 1 -- 172.21.15.37:0/597120277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1104d0 con 0x7f072c105190 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f0732e0b640 1 -- 172.21.15.37:0/597120277 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1107d0 con 0x7f072c100440 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f07313a8640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f072c105190 0x7f072c196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f07313a8640 1 -- 172.21.15.37:0/597120277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f072c100440 msgr2=0x7f072c19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f07313a8640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f072c100440 0x7f072c19c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f07313a8640 1 -- 172.21.15.37:0/597120277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c100e00 msgr2=0x7f072c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f07313a8640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c100e00 0x7f072c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f07313a8640 1 -- 172.21.15.37:0/597120277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c197010 con 0x7f072c105190 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f072bfff640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c100e00 0x7f072c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.002+0000 7f07313a8640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f072c105190 0x7f072c196890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0724009bf0 tx=0x7f0724032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.003+0000 7f0729ffb640 1 -- 172.21.15.37:0/597120277 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724039040 con 0x7f072c105190 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.003+0000 7f0729ffb640 1 -- 172.21.15.37:0/597120277 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0724003970 con 0x7f072c105190 2024-03-26T06:24:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.003+0000 7f0732e0b640 1 -- 172.21.15.37:0/597120277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c1972a0 con 0x7f072c105190 2024-03-26T06:24:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.003+0000 7f0729ffb640 1 -- 172.21.15.37:0/597120277 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724004460 con 0x7f072c105190 2024-03-26T06:24:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.003+0000 7f0732e0b640 1 -- 172.21.15.37:0/597120277 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c068970 con 0x7f072c105190 2024-03-26T06:24:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.006+0000 7f07359eb740 1 Processor -- start 2024-03-26T06:24:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.007+0000 7f0729ffb640 1 -- 172.21.15.37:0/597120277 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) v1 ==== 137523+0+0 (secure 0 0 0) 0x7f0724003b20 con 0x7f072c105190 2024-03-26T06:24:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.007+0000 7f07359eb740 1 -- start start 2024-03-26T06:24:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.007+0000 7f07359eb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb22b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f07359eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb13690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f07359eb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13bd0 0x56539fb2a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f07359eb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56539fa7a5f0 con 0x56539fb13210 2024-03-26T06:24:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f07359eb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56539fa23930 con 0x56539fb13bd0 2024-03-26T06:24:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f07359eb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56539fa76690 con 0x56539fb22790 2024-03-26T06:24:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f0730ba7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb13690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f0730ba7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb13690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56986/0 (socket says 172.21.15.37:56986) 2024-03-26T06:24:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f0730ba7640 1 -- 172.21.15.37:0/938801437 learned_addr learned my addr 172.21.15.37:0/938801437 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f072bfff640 1 --2- 172.21.15.37:0/938801437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb22b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f0729ffb640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f071c0a2bb0 0x7f071c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f072bfff640 1 --2- 172.21.15.37:0/938801437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13bd0 0x56539fb2a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f072bfff640 1 -- 172.21.15.37:0/938801437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13bd0 msgr2=0x56539fb2a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f072bfff640 1 --2- 172.21.15.37:0/938801437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13bd0 0x56539fb2a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f072bfff640 1 -- 172.21.15.37:0/938801437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 msgr2=0x56539fb13690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f072bfff640 1 --2- 172.21.15.37:0/938801437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb13690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f072bfff640 1 -- 172.21.15.37:0/938801437 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56539f78f210 con 0x56539fb22790 2024-03-26T06:24:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f0730ba7640 1 --2- 172.21.15.37:0/938801437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb13690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:24:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f0729ffb640 1 -- 172.21.15.37:0/597120277 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f07240f0980 con 0x7f072c105190 2024-03-26T06:24:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.008+0000 7f07313a8640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f071c0a2bb0 0x7f071c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.009+0000 7f072bfff640 1 --2- 172.21.15.37:0/938801437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb22b70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f072c198080 tx=0x7f071800be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe420f09ccce4dd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.009+0000 7f0716ffd640 1 -- 172.21.15.37:0/938801437 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718004230 con 0x56539fb22790 2024-03-26T06:24:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.009+0000 7f0716ffd640 1 -- 172.21.15.37:0/938801437 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07180043d0 con 0x56539fb22790 2024-03-26T06:24:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.009+0000 7f0716ffd640 1 -- 172.21.15.37:0/938801437 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07180047a0 con 0x56539fb22790 2024-03-26T06:24:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07359eb740 1 -- 172.21.15.37:0/938801437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 msgr2=0x56539fb22b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07313a8640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f071c0a2bb0 0x7f071c0a5070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f07240ee0f0 tx=0x7f07240bcd70 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07359eb740 1 --2- 172.21.15.37:0/938801437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb22b70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f072c198080 tx=0x7f071800be60 comp rx=0 tx=0).stop 2024-03-26T06:24:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07359eb740 1 -- 172.21.15.37:0/938801437 shutdown_connections 2024-03-26T06:24:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07359eb740 1 --2- 172.21.15.37:0/938801437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13bd0 0x56539fb2a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07359eb740 1 --2- 172.21.15.37:0/938801437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb13690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07359eb740 1 --2- 172.21.15.37:0/938801437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb22b70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07359eb740 1 -- 172.21.15.37:0/938801437 >> 172.21.15.37:0/938801437 conn(0x56539fadd5b0 msgr2=0x56539fb12100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07359eb740 1 -- 172.21.15.37:0/938801437 shutdown_connections 2024-03-26T06:24:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07359eb740 1 -- 172.21.15.37:0/938801437 wait complete. 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07359eb740 1 -- start start 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.010+0000 7f07359eb740 1 Processor -- start 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.011+0000 7f07359eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb27910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.011+0000 7f072bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb27910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.011+0000 7f072bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb27910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56998/0 (socket says 172.21.15.37:56998) 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.011+0000 7f072bfff640 1 -- 172.21.15.37:0/3705465791 learned_addr learned my addr 172.21.15.37:0/3705465791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.011+0000 7f07359eb740 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13bd0 0x56539fb27e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.011+0000 7f07359eb740 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb28390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.011+0000 7f07359eb740 1 -- 172.21.15.37:0/3705465791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56539fb2aaf0 con 0x56539fb13210 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.011+0000 7f07359eb740 1 -- 172.21.15.37:0/3705465791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56539fa36d90 con 0x56539fb13bd0 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.012+0000 7f07359eb740 1 -- 172.21.15.37:0/3705465791 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56539fb2ac70 con 0x56539fb22790 2024-03-26T06:24:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.012+0000 7f0730ba7640 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13bd0 0x56539fb27e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.012+0000 7f072bfff640 1 -- 172.21.15.37:0/3705465791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 msgr2=0x56539fb28390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.012+0000 7f072bfff640 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb28390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.012+0000 7f072bfff640 1 -- 172.21.15.37:0/3705465791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13bd0 msgr2=0x56539fb27e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.012+0000 7f072bfff640 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13bd0 0x56539fb27e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.012+0000 7f072bfff640 1 -- 172.21.15.37:0/3705465791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56539f85b620 con 0x56539fb13210 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.012+0000 7f07359eb740 10 client.0 _ll_register_callbacks cb 0x56539f78ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.012+0000 7f07359eb740 1 -- 172.21.15.37:0/3705465791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07180043d0 con 0x56539fb13210 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.012+0000 7f072bfff640 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb27910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0718006010 tx=0x7f0718005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87808111ed5c1e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.012+0000 7f070bfff640 10 client.15597 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.013+0000 7f070bfff640 1 -- 172.21.15.37:0/3705465791 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07180108c0 con 0x56539fb13210 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.013+0000 7f070bfff640 1 -- 172.21.15.37:0/3705465791 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0718010a60 con 0x56539fb13210 2024-03-26T06:24:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.013+0000 7f070bfff640 1 -- 172.21.15.37:0/3705465791 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0718010ca0 con 0x56539fb13210 2024-03-26T06:24:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.013+0000 7f07359eb740 10 client.15597 fetch_fsmap learned FSMap version 27 2024-03-26T06:24:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.013+0000 7f07359eb740 1 -- 172.21.15.37:0/3705465791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x56539fb36bc0 con 0x56539fb13210 2024-03-26T06:24:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.013+0000 7f070bfff640 1 -- 172.21.15.37:0/3705465791 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f07180046c0 con 0x56539fb13210 2024-03-26T06:24:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.013+0000 7f07359eb740 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:24:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.013+0000 7f07359eb740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:24:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.013+0000 7f07359eb740 20 client.15597 populate_metadata read hostname 'smithi037' 2024-03-26T06:24:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.014+0000 7f07359eb740 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56539fb37f10 0x56539fb3a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.014+0000 7f07359eb740 4 client.15597 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:24:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.014+0000 7f07359eb740 1 -- 172.21.15.37:0/3705465791 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56539fa87b30 con 0x56539fb37f10 2024-03-26T06:24:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.014+0000 7f0730ba7640 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56539fb37f10 0x56539fb3a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.014+0000 7f070bfff640 10 client.15597 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.014+0000 7f0730ba7640 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56539fb37f10 0x56539fb3a300 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60808a8f2c10e591 server_cookie=66165b35f324b092 in_seq=0 out_seq=0 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f070bfff640 1 -- 172.21.15.37:0/3705465791 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56539fa87b30 con 0x56539fb37f10 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f070bfff640 10 client.15597 handle_command_reply: tid=0 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f07359eb740 1 client.15597 shutdown 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f07359eb740 1 -- 172.21.15.37:0/3705465791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 msgr2=0x56539fb27910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f07359eb740 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb27910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0718006010 tx=0x7f0718005740 comp rx=0 tx=0).stop 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f07359eb740 1 -- 172.21.15.37:0/3705465791 shutdown_connections 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f07359eb740 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56539fb37f10 0x56539fb3a300 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f07359eb740 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb28390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f070bfff640 0 client.15597 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f07359eb740 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13bd0 0x56539fb27e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f07359eb740 1 --2- 172.21.15.37:0/3705465791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13210 0x56539fb27910 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.015+0000 7f07359eb740 1 -- 172.21.15.37:0/3705465791 >> 172.21.15.37:0/3705465791 conn(0x56539fadd5b0 msgr2=0x56539fb0fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.016+0000 7f07359eb740 1 -- 172.21.15.37:0/3705465791 shutdown_connections 2024-03-26T06:24:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.016+0000 7f07359eb740 1 -- 172.21.15.37:0/3705465791 wait complete. 2024-03-26T06:24:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.016+0000 7f07359eb740 20 client.15597 trim_cache size 0 max 16384 2024-03-26T06:24:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.042+0000 7f07359eb740 1 Processor -- start 2024-03-26T06:24:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.042+0000 7f07359eb740 1 -- start start 2024-03-26T06:24:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.042+0000 7f07359eb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13210 0x56539fb2a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.042+0000 7f07359eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f07359eb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb26720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f07359eb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1aa8e0 con 0x56539fb13bd0 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f0730ba7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f0730ba7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb261e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57004/0 (socket says 172.21.15.37:57004) 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f0730ba7640 1 -- 172.21.15.37:0/3392100990 learned_addr learned my addr 172.21.15.37:0/3392100990 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f07359eb740 1 -- 172.21.15.37:0/3392100990 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1ac260 con 0x56539fb13210 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f072bfff640 1 --2- 172.21.15.37:0/3392100990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13210 0x56539fb2a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f07359eb740 1 -- 172.21.15.37:0/3392100990 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1aaf40 con 0x56539fb22790 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f072bfff640 1 --2- 172.21.15.37:0/3392100990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb26720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f0730ba7640 1 -- 172.21.15.37:0/3392100990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 msgr2=0x56539fb26720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f0730ba7640 1 --2- 172.21.15.37:0/3392100990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb26720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f0730ba7640 1 -- 172.21.15.37:0/3392100990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13210 msgr2=0x56539fb2a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f0730ba7640 1 --2- 172.21.15.37:0/3392100990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13210 0x56539fb2a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f0730ba7640 1 -- 172.21.15.37:0/3392100990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56539fb21840 con 0x56539fb13bd0 2024-03-26T06:24:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.043+0000 7f0730ba7640 1 --2- 172.21.15.37:0/3392100990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb261e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0720009be0 tx=0x7f07200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f671f3538ce625 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.044+0000 7f07167fc640 1 -- 172.21.15.37:0/3392100990 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720003d30 con 0x56539fb13bd0 2024-03-26T06:24:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.044+0000 7f07167fc640 1 -- 172.21.15.37:0/3392100990 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f072001f550 con 0x56539fb13bd0 2024-03-26T06:24:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.044+0000 7f07167fc640 1 -- 172.21.15.37:0/3392100990 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720010a10 con 0x56539fb13bd0 2024-03-26T06:24:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.045+0000 7f07359eb740 1 -- 172.21.15.37:0/3392100990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 msgr2=0x56539fb261e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.045+0000 7f07359eb740 1 --2- 172.21.15.37:0/3392100990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb261e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0720009be0 tx=0x7f07200046c0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.045+0000 7f07359eb740 1 -- 172.21.15.37:0/3392100990 shutdown_connections 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.045+0000 7f07359eb740 1 --2- 172.21.15.37:0/3392100990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb22790 0x56539fb26720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.045+0000 7f07359eb740 1 --2- 172.21.15.37:0/3392100990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb261e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.045+0000 7f07359eb740 1 --2- 172.21.15.37:0/3392100990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb13210 0x56539fb2a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.045+0000 7f07359eb740 1 -- 172.21.15.37:0/3392100990 >> 172.21.15.37:0/3392100990 conn(0x56539fadd5b0 msgr2=0x56539fb11fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.045+0000 7f07359eb740 1 -- 172.21.15.37:0/3392100990 shutdown_connections 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.045+0000 7f07359eb740 1 -- 172.21.15.37:0/3392100990 wait complete. 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.045+0000 7f07359eb740 1 -- start start 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.045+0000 7f07359eb740 1 Processor -- start 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f07359eb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb13210 0x56539fb2f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f07359eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb2d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f07359eb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb22790 0x56539fb2d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f07359eb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1abc00 con 0x56539fb13bd0 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f07359eb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1ab5a0 con 0x56539fb22790 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f07359eb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1a8b80 con 0x56539fb13210 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f0730ba7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb2d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f07359eb740 10 client.0 _ll_register_callbacks cb 0x7f071800bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f0730ba7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb2d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57014/0 (socket says 172.21.15.37:57014) 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f0730ba7640 1 -- 172.21.15.37:0/333681677 learned_addr learned my addr 172.21.15.37:0/333681677 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f072bfff640 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb22790 0x56539fb2d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.046+0000 7f072bfff640 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb13210 0x56539fb2f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f0730ba7640 1 -- 172.21.15.37:0/333681677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb13210 msgr2=0x56539fb2f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f0730ba7640 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb13210 0x56539fb2f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f0730ba7640 1 -- 172.21.15.37:0/333681677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb22790 msgr2=0x56539fb2d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f0730ba7640 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb22790 0x56539fb2d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f0730ba7640 1 -- 172.21.15.37:0/333681677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56539fb2dfc0 con 0x56539fb13bd0 2024-03-26T06:24:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f0730ba7640 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb2d490 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0720023000 tx=0x7f07200105b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec20dc3f1240a54 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f070bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:24:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f070bfff640 1 -- 172.21.15.37:0/333681677 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720003d30 con 0x56539fb13bd0 2024-03-26T06:24:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f070bfff640 1 -- 172.21.15.37:0/333681677 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f072001f040 con 0x56539fb13bd0 2024-03-26T06:24:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f07359eb740 1 -- 172.21.15.37:0/333681677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f072001f550 con 0x56539fb13bd0 2024-03-26T06:24:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f070bfff640 1 -- 172.21.15.37:0/333681677 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0720038050 con 0x56539fb13bd0 2024-03-26T06:24:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f07359eb740 10 client.15603 fetch_fsmap learned FSMap version 27 2024-03-26T06:24:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.047+0000 7f07359eb740 1 -- 172.21.15.37:0/333681677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56539fb8f050 con 0x56539fb13bd0 2024-03-26T06:24:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.048+0000 7f070bfff640 1 -- 172.21.15.37:0/333681677 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f072001f550 con 0x56539fb13bd0 2024-03-26T06:24:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.049+0000 7f070bfff640 1 -- 172.21.15.37:0/333681677 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f07200107d0 con 0x56539fb13bd0 2024-03-26T06:24:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.049+0000 7f07359eb740 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:24:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.049+0000 7f07359eb740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:24:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.049+0000 7f07359eb740 20 client.15603 populate_metadata read hostname 'smithi037' 2024-03-26T06:24:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.049+0000 7f07359eb740 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56539fb37f10 0x56539fb91670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.049+0000 7f07359eb740 4 client.15603 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:24:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.049+0000 7f07359eb740 1 -- 172.21.15.37:0/333681677 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56539fa81e80 con 0x56539fb37f10 2024-03-26T06:24:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.049+0000 7f072bfff640 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56539fb37f10 0x56539fb91670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.050+0000 7f070bfff640 10 client.15603 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.050+0000 7f072bfff640 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56539fb37f10 0x56539fb91670 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c343bc999b1a69a7 server_cookie=c40b208a5d1b55 in_seq=0 out_seq=0 2024-03-26T06:24:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f070bfff640 1 -- 172.21.15.37:0/333681677 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56539fa81e80 con 0x56539fb37f10 2024-03-26T06:24:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f070bfff640 10 client.15603 handle_command_reply: tid=0 2024-03-26T06:24:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f07359eb740 1 client.15603 shutdown 2024-03-26T06:24:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f07359eb740 1 -- 172.21.15.37:0/333681677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 msgr2=0x56539fb2d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f07359eb740 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb2d490 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0720023000 tx=0x7f07200105b0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f07359eb740 1 -- 172.21.15.37:0/333681677 shutdown_connections 2024-03-26T06:24:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f07359eb740 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56539fb37f10 0x56539fb91670 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f07359eb740 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56539fb22790 0x56539fb2d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f07359eb740 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56539fb13bd0 0x56539fb2d490 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f07359eb740 1 --2- 172.21.15.37:0/333681677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56539fb13210 0x56539fb2f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f07359eb740 1 -- 172.21.15.37:0/333681677 >> 172.21.15.37:0/333681677 conn(0x56539fadd5b0 msgr2=0x56539fb278b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.054+0000 7f070bfff640 0 client.15603 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.055+0000 7f07359eb740 1 -- 172.21.15.37:0/333681677 shutdown_connections 2024-03-26T06:24:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.055+0000 7f07359eb740 1 -- 172.21.15.37:0/333681677 wait complete. 2024-03-26T06:24:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.055+0000 7f07359eb740 20 client.15603 trim_cache size 0 max 16384 2024-03-26T06:24:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:24:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:24:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:24:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:24:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.055+0000 7f0732e0b640 1 -- 172.21.15.37:0/597120277 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f071c0a2bb0 msgr2=0x7f071c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.055+0000 7f0732e0b640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f071c0a2bb0 0x7f071c0a5070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f07240ee0f0 tx=0x7f07240bcd70 comp rx=0 tx=0).stop 2024-03-26T06:24:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.055+0000 7f0732e0b640 1 -- 172.21.15.37:0/597120277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f072c105190 msgr2=0x7f072c196890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.055+0000 7f0732e0b640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f072c105190 0x7f072c196890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0724009bf0 tx=0x7f0724032ea0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.056+0000 7f0732e0b640 1 -- 172.21.15.37:0/597120277 shutdown_connections 2024-03-26T06:24:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.056+0000 7f0732e0b640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f071c0a2bb0 0x7f071c0a5070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.056+0000 7f0732e0b640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f072c105190 0x7f072c196890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.056+0000 7f0732e0b640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f072c100e00 0x7f072c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.056+0000 7f0732e0b640 1 --2- 172.21.15.37:0/597120277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f072c100440 0x7f072c19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.056+0000 7f0732e0b640 1 -- 172.21.15.37:0/597120277 >> 172.21.15.37:0/597120277 conn(0x7f072c077040 msgr2=0x7f072c0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.056+0000 7f0732e0b640 1 -- 172.21.15.37:0/597120277 shutdown_connections 2024-03-26T06:24:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:20.056+0000 7f0732e0b640 1 -- 172.21.15.37:0/597120277 wait complete. 2024-03-26T06:24:20.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:20 smithi149 ceph-mon[36681]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-26T06:24:20.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:20 smithi037 ceph-mon[31974]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-26T06:24:21.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:20 smithi066 ceph-mon[36504]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-26T06:24:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:22 smithi149 ceph-mon[36681]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-26T06:24:22.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:22 smithi037 ceph-mon[31974]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-26T06:24:23.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:22 smithi066 ceph-mon[36504]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-26T06:24:24.088 INFO:tasks.workunit.client.0.smithi037.stderr:Updating files: 96% (11818/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-26T06:24:24.271 INFO:tasks.workunit.client.0.smithi037.stderr:Note: switching to '7e98f5f6b535defc7fa715b456904ffc0500aa9f'. 2024-03-26T06:24:24.271 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-26T06:24:24.271 INFO:tasks.workunit.client.0.smithi037.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-26T06:24:24.271 INFO:tasks.workunit.client.0.smithi037.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-26T06:24:24.271 INFO:tasks.workunit.client.0.smithi037.stderr:state without impacting any branches by switching back to a branch. 2024-03-26T06:24:24.271 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-26T06:24:24.271 INFO:tasks.workunit.client.0.smithi037.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-26T06:24:24.271 INFO:tasks.workunit.client.0.smithi037.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-26T06:24:24.271 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-26T06:24:24.271 INFO:tasks.workunit.client.0.smithi037.stderr: git switch -c 2024-03-26T06:24:24.272 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-26T06:24:24.272 INFO:tasks.workunit.client.0.smithi037.stderr:Or undo this operation with: 2024-03-26T06:24:24.272 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-26T06:24:24.272 INFO:tasks.workunit.client.0.smithi037.stderr: git switch - 2024-03-26T06:24:24.272 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-26T06:24:24.272 INFO:tasks.workunit.client.0.smithi037.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-26T06:24:24.272 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-26T06:24:24.272 INFO:tasks.workunit.client.0.smithi037.stderr:HEAD is now at 7e98f5f6b53 DNM: just to trigger builds again 2024-03-26T06:24:24.275 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-26T06:24:24.342 INFO:tasks.workunit.client.0.smithi037.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-26T06:24:24.344 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-26T06:24:24.345 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-26T06:24:24.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:24 smithi037 ceph-mon[31974]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:24:25.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:24 smithi066 ceph-mon[36504]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:24:25.111 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-26T06:24:25.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:24 smithi149 ceph-mon[36681]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-26T06:24:25.173 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-26T06:24:25.212 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-26T06:24:25.214 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-26T06:24:25.214 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-26T06:24:25.253 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-26T06:24:25.256 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:24:25.257 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-26T06:24:25.312 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-03-26T06:24:25.314 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-03-26T06:24:25.314 DEBUG:teuthology.orchestra.run.smithi037:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7e98f5f6b535defc7fa715b456904ffc0500aa9f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-03-26T06:24:25.372 INFO:tasks.workunit.client.0.smithi037.stderr:+ dbench 1 2024-03-26T06:24:25.377 INFO:tasks.workunit.client.0.smithi037.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-26T06:24:25.377 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-03-26T06:24:25.377 INFO:tasks.workunit.client.0.smithi037.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-26T06:24:25.382 INFO:tasks.workunit.client.0.smithi037.stdout:0 of 1 processes prepared for launch 0 sec 2024-03-26T06:24:25.483 INFO:tasks.workunit.client.0.smithi037.stdout:1 of 1 processes prepared for launch 0 sec 2024-03-26T06:24:25.483 INFO:tasks.workunit.client.0.smithi037.stdout:releasing clients 2024-03-26T06:24:26.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 3628 38.60 MB/sec warmup 1 sec latency 6.803 ms 2024-03-26T06:24:26.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:26 smithi037 ceph-mon[31974]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-26T06:24:26.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:26 smithi149 ceph-mon[36681]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-26T06:24:27.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:26 smithi066 ceph-mon[36504]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-26T06:24:27.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 6764 26.70 MB/sec warmup 2 sec latency 6.441 ms 2024-03-26T06:24:28.486 INFO:tasks.workunit.client.0.smithi037.stdout: 1 9600 22.32 MB/sec warmup 3 sec latency 12.340 ms 2024-03-26T06:24:28.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:28 smithi037 ceph-mon[31974]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:29.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:28 smithi066 ceph-mon[36504]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:29.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:28 smithi149 ceph-mon[36681]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:29.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 12743 21.00 MB/sec warmup 4 sec latency 6.947 ms 2024-03-26T06:24:30.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 15416 19.91 MB/sec warmup 5 sec latency 12.936 ms 2024-03-26T06:24:30.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:30 smithi037 ceph-mon[31974]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:30.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:30 smithi037 ceph-mon[31974]: mgrmap e26: x(active, since 90s), standbys: z, y 2024-03-26T06:24:31.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:30 smithi066 ceph-mon[36504]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:31.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:30 smithi066 ceph-mon[36504]: mgrmap e26: x(active, since 90s), standbys: z, y 2024-03-26T06:24:31.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:30 smithi149 ceph-mon[36681]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-26T06:24:31.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:30 smithi149 ceph-mon[36681]: mgrmap e26: x(active, since 90s), standbys: z, y 2024-03-26T06:24:31.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 18556 19.51 MB/sec warmup 6 sec latency 6.395 ms 2024-03-26T06:24:32.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 21809 19.39 MB/sec warmup 7 sec latency 6.493 ms 2024-03-26T06:24:32.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:32 smithi037 ceph-mon[31974]: pgmap v49: 97 pgs: 97 active+clean; 8.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 686 KiB/s wr, 94 op/s 2024-03-26T06:24:33.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:32 smithi066 ceph-mon[36504]: pgmap v49: 97 pgs: 97 active+clean; 8.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 686 KiB/s wr, 94 op/s 2024-03-26T06:24:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.147+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2058024957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f58102e20 msgr2=0x7f9f5810b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.147+0000 7f9f5ff29640 1 --2- 172.21.15.37:0/2058024957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f58102e20 0x7f9f5810b300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009900 tx=0x7f9f4802bcd0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.147+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2058024957 shutdown_connections 2024-03-26T06:24:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.147+0000 7f9f5ff29640 1 --2- 172.21.15.37:0/2058024957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f5810b8d0 0x7f9f5810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.147+0000 7f9f5ff29640 1 --2- 172.21.15.37:0/2058024957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f58102e20 0x7f9f5810b300 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.147+0000 7f9f5ff29640 1 --2- 172.21.15.37:0/2058024957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58102500 0x7f9f581028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.147+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2058024957 >> 172.21.15.37:0/2058024957 conn(0x7f9f58077400 msgr2=0x7f9f58077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.148+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2058024957 shutdown_connections 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.148+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2058024957 wait complete. 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.148+0000 7f9f5ff29640 1 Processor -- start 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.148+0000 7f9f5ff29640 1 -- start start 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.148+0000 7f9f5ff29640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f58102500 0x7f9f5819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.148+0000 7f9f5ff29640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58102e20 0x7f9f5819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.148+0000 7f9f5ff29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f5810b8d0 0x7f9f581978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.148+0000 7f9f5ff29640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58111660 con 0x7f9f5810b8d0 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.148+0000 7f9f5ff29640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581114e0 con 0x7f9f58102500 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.148+0000 7f9f5ff29640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581117e0 con 0x7f9f58102e20 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.148+0000 7f9f5e4c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f5810b8d0 0x7f9f581978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5dcc5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f58102500 0x7f9f5819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5e4c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f5810b8d0 0x7f9f581978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33154/0 (socket says 172.21.15.37:33154) 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5dcc5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f58102500 0x7f9f5819d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:41024/0 (socket says 172.21.15.37:41024) 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/2594576349 learned_addr learned my addr 172.21.15.37:0/2594576349 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5d4c4640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58102e20 0x7f9f5819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/2594576349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58102e20 msgr2=0x7f9f5819dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5e4c6640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58102e20 0x7f9f5819dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/2594576349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f58102500 msgr2=0x7f9f5819d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5e4c6640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f58102500 0x7f9f5819d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/2594576349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f58198060 con 0x7f9f5810b8d0 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5dcc5640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f58102500 0x7f9f5819d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5e4c6640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f5810b8d0 0x7f9f581978e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400e970 tx=0x7f9f5400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f4effd640 1 -- 172.21.15.37:0/2594576349 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400ccc0 con 0x7f9f5810b8d0 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f4effd640 1 -- 172.21.15.37:0/2594576349 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f5400ce60 con 0x7f9f5810b8d0 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f4effd640 1 -- 172.21.15.37:0/2594576349 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54010490 con 0x7f9f5810b8d0 2024-03-26T06:24:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2594576349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f58198350 con 0x7f9f5810b8d0 2024-03-26T06:24:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.149+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2594576349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f581a45f0 con 0x7f9f5810b8d0 2024-03-26T06:24:33.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:32 smithi149 ceph-mon[36681]: pgmap v49: 97 pgs: 97 active+clean; 8.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 686 KiB/s wr, 94 op/s 2024-03-26T06:24:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.151+0000 7f9f4effd640 1 -- 172.21.15.37:0/2594576349 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9f54005020 con 0x7f9f5810b8d0 2024-03-26T06:24:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f62be8740 1 Processor -- start 2024-03-26T06:24:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f62be8740 1 -- start start 2024-03-26T06:24:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a3d40 0x55af637a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637a3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f4effd640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9f440a2a10 0x7f9f440a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637ac550 0x55af637ae940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f62be8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af637075f0 con 0x55af637a3d40 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f62be8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af636b0930 con 0x55af637ac550 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f62be8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af63703690 con 0x55af637a2fd0 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f5d4c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a3d40 0x55af637a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f5dcc5640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9f440a2a10 0x7f9f440a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f5e4c6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637ac550 0x55af637ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f5d4c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a3d40 0x55af637a2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33168/0 (socket says 172.21.15.37:33168) 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f5d4c4640 1 -- 172.21.15.37:0/2924808996 learned_addr learned my addr 172.21.15.37:0/2924808996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f4effd640 1 -- 172.21.15.37:0/2594576349 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f9f540cf2b0 con 0x7f9f5810b8d0 2024-03-26T06:24:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f5d4c4640 1 --2- 172.21.15.37:0/2924808996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/2924808996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 msgr2=0x55af637a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.152+0000 7f9f5e4c6640 1 --2- 172.21.15.37:0/2924808996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.153+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/2924808996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a3d40 msgr2=0x55af637a2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.153+0000 7f9f5e4c6640 1 --2- 172.21.15.37:0/2924808996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a3d40 0x55af637a2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.153+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/2924808996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af6341c210 con 0x55af637ac550 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.153+0000 7f9f5d4c4640 1 --2- 172.21.15.37:0/2924808996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.153+0000 7f9f5dcc5640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9f440a2a10 0x7f9f440a4ed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f58198a20 tx=0x7f9f50009500 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.153+0000 7f9f5e4c6640 1 --2- 172.21.15.37:0/2924808996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637ac550 0x55af637ae940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9f540cc9d0 tx=0x7f9f54007850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a0121cdf8c7761 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.153+0000 7f9f43fff640 1 -- 172.21.15.37:0/2924808996 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5409bfe0 con 0x55af637ac550 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.153+0000 7f9f43fff640 1 -- 172.21.15.37:0/2924808996 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f540a4f40 con 0x55af637ac550 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.153+0000 7f9f43fff640 1 -- 172.21.15.37:0/2924808996 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f540a4080 con 0x55af637ac550 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.154+0000 7f9f62be8740 1 -- 172.21.15.37:0/2924808996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637ac550 msgr2=0x55af637ae940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.154+0000 7f9f62be8740 1 --2- 172.21.15.37:0/2924808996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637ac550 0x55af637ae940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9f540cc9d0 tx=0x7f9f54007850 comp rx=0 tx=0).stop 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.154+0000 7f9f62be8740 1 -- 172.21.15.37:0/2924808996 shutdown_connections 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.154+0000 7f9f62be8740 1 --2- 172.21.15.37:0/2924808996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637ac550 0x55af637ae940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.154+0000 7f9f62be8740 1 --2- 172.21.15.37:0/2924808996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.154+0000 7f9f62be8740 1 --2- 172.21.15.37:0/2924808996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a3d40 0x55af637a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.154+0000 7f9f62be8740 1 -- 172.21.15.37:0/2924808996 >> 172.21.15.37:0/2924808996 conn(0x55af6376a5b0 msgr2=0x55af6379dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.154+0000 7f9f62be8740 1 -- 172.21.15.37:0/2924808996 shutdown_connections 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.154+0000 7f9f62be8740 1 -- 172.21.15.37:0/2924808996 wait complete. 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.154+0000 7f9f62be8740 1 -- start start 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.154+0000 7f9f62be8740 1 Processor -- start 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a2fd0 0x55af637a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a3d40 0x55af637a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637ac550 0x55af637aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f62be8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af637af220 con 0x55af637a2fd0 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f62be8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af636c3d90 con 0x55af637ac550 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f62be8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af637af3a0 con 0x55af637a3d40 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f62be8740 10 client.0 _ll_register_callbacks cb 0x55af6341ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5dcc5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a2fd0 0x55af637a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5dcc5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a2fd0 0x55af637a95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33182/0 (socket says 172.21.15.37:33182) 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5dcc5640 1 -- 172.21.15.37:0/436018944 learned_addr learned my addr 172.21.15.37:0/436018944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5e4c6640 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637ac550 0x55af637aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5d4c4640 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a3d40 0x55af637a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5dcc5640 1 -- 172.21.15.37:0/436018944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a3d40 msgr2=0x55af637a9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5dcc5640 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a3d40 0x55af637a9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5dcc5640 1 -- 172.21.15.37:0/436018944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637ac550 msgr2=0x55af637aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5dcc5640 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637ac550 0x55af637aa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5dcc5640 1 -- 172.21.15.37:0/436018944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af634e8620 con 0x55af637a2fd0 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5d4c4640 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a3d40 0x55af637a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f5dcc5640 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a2fd0 0x55af637a95a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9f50003df0 tx=0x7f9f50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2ebce68e30559b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f417fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:24:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f417fa640 1 -- 172.21.15.37:0/436018944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5001e800 con 0x55af637a2fd0 2024-03-26T06:24:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f417fa640 1 -- 172.21.15.37:0/436018944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f9f5001e9a0 con 0x55af637a2fd0 2024-03-26T06:24:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.155+0000 7f9f62be8740 1 -- 172.21.15.37:0/436018944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f540a4f40 con 0x55af637a2fd0 2024-03-26T06:24:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f417fa640 1 -- 172.21.15.37:0/436018944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f5001eba0 con 0x55af637a2fd0 2024-03-26T06:24:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f62be8740 10 client.15615 fetch_fsmap learned FSMap version 27 2024-03-26T06:24:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f62be8740 1 -- 172.21.15.37:0/436018944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55af637bb4f0 con 0x55af637a2fd0 2024-03-26T06:24:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f417fa640 1 -- 172.21.15.37:0/436018944 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f540a4f40 con 0x55af637a2fd0 2024-03-26T06:24:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f417fa640 1 -- 172.21.15.37:0/436018944 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9f5000e3b0 con 0x55af637a2fd0 2024-03-26T06:24:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f62be8740 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:24:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f62be8740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:24:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f62be8740 20 client.15615 populate_metadata read hostname 'smithi037' 2024-03-26T06:24:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f62be8740 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55af637bc9b0 0x55af637beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f62be8740 4 client.15615 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f62be8740 1 -- 172.21.15.37:0/436018944 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af63714b30 con 0x55af637bc9b0 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f5d4c4640 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55af637bc9b0 0x55af637beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f5d4c4640 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55af637bc9b0 0x55af637beda0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c9545c705dc55ad server_cookie=32d2bca10771a0f8 in_seq=0 out_seq=0 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.156+0000 7f9f417fa640 10 client.15615 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f417fa640 1 -- 172.21.15.37:0/436018944 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af63714b30 con 0x55af637bc9b0 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f417fa640 10 client.15615 handle_command_reply: tid=0 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f62be8740 1 client.15615 shutdown 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f62be8740 1 -- 172.21.15.37:0/436018944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a2fd0 msgr2=0x55af637a95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f62be8740 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a2fd0 0x55af637a95a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9f50003df0 tx=0x7f9f50004ab0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f62be8740 1 -- 172.21.15.37:0/436018944 shutdown_connections 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f62be8740 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55af637bc9b0 0x55af637beda0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f62be8740 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637ac550 0x55af637aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f417fa640 0 client.15615 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f62be8740 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a3d40 0x55af637a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f62be8740 1 --2- 172.21.15.37:0/436018944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637a2fd0 0x55af637a95a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.158+0000 7f9f62be8740 1 -- 172.21.15.37:0/436018944 >> 172.21.15.37:0/436018944 conn(0x55af6376a5b0 msgr2=0x55af6379fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.159+0000 7f9f62be8740 1 -- 172.21.15.37:0/436018944 shutdown_connections 2024-03-26T06:24:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.159+0000 7f9f62be8740 1 -- 172.21.15.37:0/436018944 wait complete. 2024-03-26T06:24:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.159+0000 7f9f62be8740 20 client.15615 trim_cache size 0 max 16384 2024-03-26T06:24:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f62be8740 1 Processor -- start 2024-03-26T06:24:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f62be8740 1 -- start start 2024-03-26T06:24:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637a3d40 0x55af637b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f62be8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581acc50 con 0x55af637ac550 2024-03-26T06:24:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f62be8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581ae5d0 con 0x55af637a3d40 2024-03-26T06:24:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f62be8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581ad2b0 con 0x55af637a2fd0 2024-03-26T06:24:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5e4c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5d4c4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637a3d40 0x55af637b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5e4c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33196/0 (socket says 172.21.15.37:33196) 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/1855018389 learned_addr learned my addr 172.21.15.37:0/1855018389 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5dcc5640 1 --2- 172.21.15.37:0/1855018389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5d4c4640 1 -- 172.21.15.37:0/1855018389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 msgr2=0x55af637abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5d4c4640 1 --2- 172.21.15.37:0/1855018389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637abcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5d4c4640 1 -- 172.21.15.37:0/1855018389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 msgr2=0x55af637b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5d4c4640 1 --2- 172.21.15.37:0/1855018389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5d4c4640 1 -- 172.21.15.37:0/1855018389 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af637aedb0 con 0x55af637a3d40 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5e4c6640 1 --2- 172.21.15.37:0/1855018389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.172+0000 7f9f5d4c4640 1 --2- 172.21.15.37:0/1855018389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637a3d40 0x55af637b3610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f48004ec0 tx=0x7f9f48009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2186702ffc45a390 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f437fe640 1 -- 172.21.15.37:0/1855018389 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4804c070 con 0x55af637a3d40 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f437fe640 1 -- 172.21.15.37:0/1855018389 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f4802cd40 con 0x55af637a3d40 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f437fe640 1 -- 172.21.15.37:0/1855018389 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48004a80 con 0x55af637a3d40 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f62be8740 1 -- 172.21.15.37:0/1855018389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637a3d40 msgr2=0x55af637b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f62be8740 1 --2- 172.21.15.37:0/1855018389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637a3d40 0x55af637b3610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f48004ec0 tx=0x7f9f48009260 comp rx=0 tx=0).stop 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f62be8740 1 -- 172.21.15.37:0/1855018389 shutdown_connections 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f62be8740 1 --2- 172.21.15.37:0/1855018389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f62be8740 1 --2- 172.21.15.37:0/1855018389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637a3d40 0x55af637b3610 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f62be8740 1 --2- 172.21.15.37:0/1855018389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f62be8740 1 -- 172.21.15.37:0/1855018389 >> 172.21.15.37:0/1855018389 conn(0x55af6376a5b0 msgr2=0x55af637a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f62be8740 1 -- 172.21.15.37:0/1855018389 shutdown_connections 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.173+0000 7f9f62be8740 1 -- 172.21.15.37:0/1855018389 wait complete. 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f62be8740 1 -- start start 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f62be8740 1 Processor -- start 2024-03-26T06:24:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637a3d40 0x55af637b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f62be8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f62be8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581adf70 con 0x55af637ac550 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f62be8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581ad910 con 0x55af637a3d40 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f62be8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581aaef0 con 0x55af637a2fd0 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f62be8740 10 client.0 _ll_register_callbacks cb 0x7f9f54005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f5e4c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f5e4c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33206/0 (socket says 172.21.15.37:33206) 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/496585233 learned_addr learned my addr 172.21.15.37:0/496585233 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f5dcc5640 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f5d4c4640 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637a3d40 0x55af637b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/496585233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 msgr2=0x55af637b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f5e4c6640 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/496585233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637a3d40 msgr2=0x55af637b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f5e4c6640 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637a3d40 0x55af637b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.174+0000 7f9f5e4c6640 1 -- 172.21.15.37:0/496585233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af637b3d60 con 0x55af637ac550 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f5e4c6640 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b1b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9f54097d40 tx=0x7f9f54004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13566d4250e4f236 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f417fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f417fa640 1 -- 172.21.15.37:0/496585233 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f540b4060 con 0x55af637ac550 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f417fa640 1 -- 172.21.15.37:0/496585233 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f9f540ccc50 con 0x55af637ac550 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f62be8740 1 -- 172.21.15.37:0/496585233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f4802cd40 con 0x55af637ac550 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f417fa640 1 -- 172.21.15.37:0/496585233 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f540cce50 con 0x55af637ac550 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f62be8740 10 client.15621 fetch_fsmap learned FSMap version 27 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f62be8740 1 -- 172.21.15.37:0/496585233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55af63814840 con 0x55af637ac550 2024-03-26T06:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f417fa640 1 -- 172.21.15.37:0/496585233 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f4802cd40 con 0x55af637ac550 2024-03-26T06:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f417fa640 1 -- 172.21.15.37:0/496585233 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9f5409bb40 con 0x55af637ac550 2024-03-26T06:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f62be8740 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T06:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f62be8740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f62be8740 20 client.15621 populate_metadata read hostname 'smithi037' 2024-03-26T06:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f62be8740 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55af63814a70 0x55af63816e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f62be8740 4 client.15621 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.175+0000 7f9f62be8740 1 -- 172.21.15.37:0/496585233 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55af6370ee80 con 0x55af63814a70 2024-03-26T06:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.176+0000 7f9f5dcc5640 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55af63814a70 0x55af63816e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.176+0000 7f9f5dcc5640 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55af63814a70 0x55af63816e60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ee699ecaf6ed1a server_cookie=e993b8af9c38f2f5 in_seq=0 out_seq=0 2024-03-26T06:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.176+0000 7f9f417fa640 10 client.15621 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.285+0000 7f9f417fa640 1 -- 172.21.15.37:0/496585233 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55af6370ee80 con 0x55af63814a70 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.285+0000 7f9f417fa640 10 client.15621 handle_command_reply: tid=0 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.285+0000 7f9f62be8740 1 client.15621 shutdown 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.285+0000 7f9f62be8740 1 -- 172.21.15.37:0/496585233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 msgr2=0x55af637b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.285+0000 7f9f62be8740 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b1b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9f54097d40 tx=0x7f9f54004a50 comp rx=0 tx=0).stop 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.285+0000 7f9f62be8740 1 -- 172.21.15.37:0/496585233 shutdown_connections 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f62be8740 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55af63814a70 0x55af63816e60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f62be8740 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af637ac550 0x55af637b1b40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f62be8740 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55af637a3d40 0x55af637b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f417fa640 0 client.15621 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f62be8740 1 --2- 172.21.15.37:0/496585233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af637a2fd0 0x55af637b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f62be8740 1 -- 172.21.15.37:0/496585233 >> 172.21.15.37:0/496585233 conn(0x55af6376a5b0 msgr2=0x55af637a8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f62be8740 1 -- 172.21.15.37:0/496585233 shutdown_connections 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f62be8740 1 -- 172.21.15.37:0/496585233 wait complete. 2024-03-26T06:24:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f62be8740 20 client.15621 trim_cache size 0 max 16384 2024-03-26T06:24:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:24:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:24:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:24:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:24:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2594576349 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9f440a2a10 msgr2=0x7f9f440a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9f440a2a10 0x7f9f440a4ed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f58198a20 tx=0x7f9f50009500 comp rx=0 tx=0).stop 2024-03-26T06:24:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2594576349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f5810b8d0 msgr2=0x7f9f581978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f5810b8d0 0x7f9f581978e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400e970 tx=0x7f9f5400ed70 comp rx=0 tx=0).stop 2024-03-26T06:24:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2594576349 shutdown_connections 2024-03-26T06:24:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9f440a2a10 0x7f9f440a4ed0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f5810b8d0 0x7f9f581978e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58102e20 0x7f9f5819dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 --2- 172.21.15.37:0/2594576349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f58102500 0x7f9f5819d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2594576349 >> 172.21.15.37:0/2594576349 conn(0x7f9f58077400 msgr2=0x7f9f580fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2594576349 shutdown_connections 2024-03-26T06:24:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:33.286+0000 7f9f5ff29640 1 -- 172.21.15.37:0/2594576349 wait complete. 2024-03-26T06:24:33.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 24955 19.18 MB/sec warmup 8 sec latency 85.318 ms 2024-03-26T06:24:34.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 27871 18.60 MB/sec warmup 9 sec latency 6.424 ms 2024-03-26T06:24:34.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:34 smithi037 ceph-mon[31974]: pgmap v50: 97 pgs: 97 active+clean; 8.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 686 KiB/s wr, 90 op/s 2024-03-26T06:24:35.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:34 smithi066 ceph-mon[36504]: pgmap v50: 97 pgs: 97 active+clean; 8.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 686 KiB/s wr, 90 op/s 2024-03-26T06:24:35.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:34 smithi149 ceph-mon[36681]: pgmap v50: 97 pgs: 97 active+clean; 8.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 686 KiB/s wr, 90 op/s 2024-03-26T06:24:35.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 30526 17.97 MB/sec warmup 10 sec latency 7.462 ms 2024-03-26T06:24:36.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 33068 17.73 MB/sec warmup 11 sec latency 16.049 ms 2024-03-26T06:24:36.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:36 smithi037 ceph-mon[31974]: pgmap v51: 97 pgs: 97 active+clean; 11 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.4 MiB/s wr, 300 op/s 2024-03-26T06:24:37.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:36 smithi066 ceph-mon[36504]: pgmap v51: 97 pgs: 97 active+clean; 11 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.4 MiB/s wr, 300 op/s 2024-03-26T06:24:37.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:36 smithi149 ceph-mon[36681]: pgmap v51: 97 pgs: 97 active+clean; 11 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.4 MiB/s wr, 300 op/s 2024-03-26T06:24:37.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 36175 17.69 MB/sec warmup 12 sec latency 6.534 ms 2024-03-26T06:24:38.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 39363 17.77 MB/sec warmup 13 sec latency 12.336 ms 2024-03-26T06:24:38.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:38 smithi037 ceph-mon[31974]: pgmap v52: 97 pgs: 97 active+clean; 11 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-03-26T06:24:39.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:38 smithi066 ceph-mon[36504]: pgmap v52: 97 pgs: 97 active+clean; 11 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-03-26T06:24:39.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:38 smithi149 ceph-mon[36681]: pgmap v52: 97 pgs: 97 active+clean; 11 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-03-26T06:24:39.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 42069 17.42 MB/sec warmup 14 sec latency 20.694 ms 2024-03-26T06:24:40.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 45015 17.17 MB/sec warmup 15 sec latency 9.352 ms 2024-03-26T06:24:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:40 smithi037 ceph-mon[31974]: pgmap v53: 97 pgs: 97 active+clean; 11 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-03-26T06:24:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:40 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:24:41.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:40 smithi066 ceph-mon[36504]: pgmap v53: 97 pgs: 97 active+clean; 11 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-03-26T06:24:41.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:40 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:24:41.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:40 smithi149 ceph-mon[36681]: pgmap v53: 97 pgs: 97 active+clean; 11 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-03-26T06:24:41.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:40 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:24:41.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46217 16.76 MB/sec warmup 16 sec latency 650.585 ms 2024-03-26T06:24:42.483 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46217 15.77 MB/sec warmup 17 sec latency 1650.639 ms 2024-03-26T06:24:42.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:42 smithi037 ceph-mon[31974]: pgmap v54: 97 pgs: 97 active+clean; 17 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 389 op/s 2024-03-26T06:24:43.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:42 smithi066 ceph-mon[36504]: pgmap v54: 97 pgs: 97 active+clean; 17 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 389 op/s 2024-03-26T06:24:43.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:42 smithi149 ceph-mon[36681]: pgmap v54: 97 pgs: 97 active+clean; 17 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 389 op/s 2024-03-26T06:24:43.484 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46233 14.90 MB/sec warmup 18 sec latency 2354.425 ms 2024-03-26T06:24:44.484 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46233 14.11 MB/sec warmup 19 sec latency 1280.298 ms 2024-03-26T06:24:44.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:44 smithi037 ceph-mon[31974]: pgmap v55: 97 pgs: 97 active+clean; 17 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.5 MiB/s wr, 304 op/s 2024-03-26T06:24:45.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:44 smithi066 ceph-mon[36504]: pgmap v55: 97 pgs: 97 active+clean; 17 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.5 MiB/s wr, 304 op/s 2024-03-26T06:24:45.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:44 smithi149 ceph-mon[36681]: pgmap v55: 97 pgs: 97 active+clean; 17 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.5 MiB/s wr, 304 op/s 2024-03-26T06:24:45.484 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46594 13.48 MB/sec warmup 20 sec latency 1543.807 ms 2024-03-26T06:24:45.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:45 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:24:46.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:45 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:24:46.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:45 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:24:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.384+0000 7f615f59e640 1 -- 172.21.15.37:0/3382246421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61601022b0 msgr2=0x7f6160102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.384+0000 7f615f59e640 1 --2- 172.21.15.37:0/3382246421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61601022b0 0x7f6160102690 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6150009900 tx=0x7f615002bcd0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.385+0000 7f615f59e640 1 -- 172.21.15.37:0/3382246421 shutdown_connections 2024-03-26T06:24:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.385+0000 7f615f59e640 1 --2- 172.21.15.37:0/3382246421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f616010b6d0 0x7f616010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.385+0000 7f615f59e640 1 --2- 172.21.15.37:0/3382246421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6160102bd0 0x7f616010b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.385+0000 7f615f59e640 1 --2- 172.21.15.37:0/3382246421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61601022b0 0x7f6160102690 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.385+0000 7f615f59e640 1 -- 172.21.15.37:0/3382246421 >> 172.21.15.37:0/3382246421 conn(0x7f61600fbaf0 msgr2=0x7f61600fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.385+0000 7f615f59e640 1 -- 172.21.15.37:0/3382246421 shutdown_connections 2024-03-26T06:24:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.385+0000 7f615f59e640 1 -- 172.21.15.37:0/3382246421 wait complete. 2024-03-26T06:24:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.385+0000 7f615f59e640 1 Processor -- start 2024-03-26T06:24:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f615f59e640 1 -- start start 2024-03-26T06:24:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f615f59e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61601022b0 0x7f616019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f615f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6160102bd0 0x7f616019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f615f59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f616010b6d0 0x7f6160197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f615f59e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601114b0 con 0x7f6160102bd0 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f615f59e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6160111330 con 0x7f616010b6d0 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f615f59e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6160111630 con 0x7f61601022b0 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f6155d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6160102bd0 0x7f616019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f6155d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6160102bd0 0x7f616019dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57156/0 (socket says 172.21.15.37:57156) 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f6155d9b640 1 -- 172.21.15.37:0/2515760681 learned_addr learned my addr 172.21.15.37:0/2515760681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f615ed9d640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f616010b6d0 0x7f6160197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f615e59c640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61601022b0 0x7f616019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f6155d9b640 1 -- 172.21.15.37:0/2515760681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61601022b0 msgr2=0x7f616019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f6155d9b640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61601022b0 0x7f616019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f6155d9b640 1 -- 172.21.15.37:0/2515760681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f616010b6d0 msgr2=0x7f6160197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f6155d9b640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f616010b6d0 0x7f6160197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f6155d9b640 1 -- 172.21.15.37:0/2515760681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6160198020 con 0x7f6160102bd0 2024-03-26T06:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.386+0000 7f615e59c640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61601022b0 0x7f616019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.387+0000 7f6155d9b640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6160102bd0 0x7f616019dbd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f614800e9f0 tx=0x7f614800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.387+0000 7f6157fff640 1 -- 172.21.15.37:0/2515760681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614800ccc0 con 0x7f6160102bd0 2024-03-26T06:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.387+0000 7f6157fff640 1 -- 172.21.15.37:0/2515760681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6148010910 con 0x7f6160102bd0 2024-03-26T06:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.387+0000 7f615f59e640 1 -- 172.21.15.37:0/2515760681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6160198310 con 0x7f6160102bd0 2024-03-26T06:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.387+0000 7f6157fff640 1 -- 172.21.15.37:0/2515760681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148018920 con 0x7f6160102bd0 2024-03-26T06:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.387+0000 7f615f59e640 1 -- 172.21.15.37:0/2515760681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61601a44e0 con 0x7f6160102bd0 2024-03-26T06:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.389+0000 7f6157fff640 1 -- 172.21.15.37:0/2515760681 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6148010430 con 0x7f6160102bd0 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f61675e9740 1 Processor -- start 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f61675e9740 1 -- start start 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f6157fff640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f614c0a2960 0x7f614c0a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f61675e9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049ef20 0x5619c049dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f61675e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049e1b0 0x5619c049e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f61675e9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c04a7730 0x5619c04a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f61675e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c04025f0 con 0x5619c049e1b0 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f61675e9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c03ab930 con 0x5619c04a7730 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f61675e9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c03fe690 con 0x5619c049ef20 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f615ed9d640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f614c0a2960 0x7f614c0a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f6155d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049e1b0 0x5619c049e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f6155d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049e1b0 0x5619c049e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57168/0 (socket says 172.21.15.37:57168) 2024-03-26T06:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f6155d9b640 1 -- 172.21.15.37:0/2247990727 learned_addr learned my addr 172.21.15.37:0/2247990727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f615e59c640 1 --2- 172.21.15.37:0/2247990727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049ef20 0x5619c049dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f6157fff640 1 -- 172.21.15.37:0/2515760681 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f61480cf1b0 con 0x7f6160102bd0 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.390+0000 7f615ed9d640 1 --2- 172.21.15.37:0/2247990727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c04a7730 0x5619c04a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f6155d9b640 1 -- 172.21.15.37:0/2247990727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049ef20 msgr2=0x5619c049dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f6155d9b640 1 --2- 172.21.15.37:0/2247990727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049ef20 0x5619c049dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f6155d9b640 1 -- 172.21.15.37:0/2247990727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c04a7730 msgr2=0x5619c04a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f6155d9b640 1 --2- 172.21.15.37:0/2247990727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c04a7730 0x5619c04a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f6155d9b640 1 -- 172.21.15.37:0/2247990727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c0117210 con 0x5619c049e1b0 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f615ed9d640 1 --2- 172.21.15.37:0/2247990727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c04a7730 0x5619c04a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f615e59c640 1 --2- 172.21.15.37:0/2247990727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049ef20 0x5619c049dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f615ed9d640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f614c0a2960 0x7f614c0a4e20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6158005fd0 tx=0x7f6158014b70 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f6155d9b640 1 --2- 172.21.15.37:0/2247990727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049e1b0 0x5619c049e630 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f614800e9c0 tx=0x7f6148009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f9f57770df8bb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f6154d99640 1 -- 172.21.15.37:0/2247990727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61480a3f80 con 0x5619c049e1b0 2024-03-26T06:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f6154d99640 1 -- 172.21.15.37:0/2247990727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6148010720 con 0x5619c049e1b0 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.391+0000 7f6154d99640 1 -- 172.21.15.37:0/2247990727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61480ccb50 con 0x5619c049e1b0 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.392+0000 7f61675e9740 1 -- 172.21.15.37:0/2247990727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049e1b0 msgr2=0x5619c049e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.392+0000 7f61675e9740 1 --2- 172.21.15.37:0/2247990727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049e1b0 0x5619c049e630 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f614800e9c0 tx=0x7f6148009f90 comp rx=0 tx=0).stop 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.392+0000 7f61675e9740 1 -- 172.21.15.37:0/2247990727 shutdown_connections 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.392+0000 7f61675e9740 1 --2- 172.21.15.37:0/2247990727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c04a7730 0x5619c04a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.392+0000 7f61675e9740 1 --2- 172.21.15.37:0/2247990727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049e1b0 0x5619c049e630 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.392+0000 7f61675e9740 1 --2- 172.21.15.37:0/2247990727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049ef20 0x5619c049dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.392+0000 7f61675e9740 1 -- 172.21.15.37:0/2247990727 >> 172.21.15.37:0/2247990727 conn(0x5619c04655b0 msgr2=0x5619c0498ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.392+0000 7f61675e9740 1 -- 172.21.15.37:0/2247990727 shutdown_connections 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.392+0000 7f61675e9740 1 -- 172.21.15.37:0/2247990727 wait complete. 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.392+0000 7f61675e9740 1 -- start start 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.392+0000 7f61675e9740 1 Processor -- start 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f61675e9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049e1b0 0x5619c04a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f61675e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f61675e9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c04a7730 0x5619c04a5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f61675e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c04aa400 con 0x5619c049ef20 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f61675e9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c03bed90 con 0x5619c049e1b0 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f61675e9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c04aa580 con 0x5619c04a7730 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f61675e9740 10 client.0 _ll_register_callbacks cb 0x5619c0116a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6155d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6155d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04a4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57180/0 (socket says 172.21.15.37:57180) 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6155d9b640 1 -- 172.21.15.37:0/1618477200 learned_addr learned my addr 172.21.15.37:0/1618477200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f615e59c640 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049e1b0 0x5619c04a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f615ed9d640 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c04a7730 0x5619c04a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6155d9b640 1 -- 172.21.15.37:0/1618477200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c04a7730 msgr2=0x5619c04a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6155d9b640 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c04a7730 0x5619c04a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6155d9b640 1 -- 172.21.15.37:0/1618477200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049e1b0 msgr2=0x5619c04a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6155d9b640 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049e1b0 0x5619c04a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6155d9b640 1 -- 172.21.15.37:0/1618477200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c01e3620 con 0x5619c049ef20 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6155d9b640 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04a4ce0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f614809b4e0 tx=0x7f61480973d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f673d2a08246d2d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6145ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6145ffb640 1 -- 172.21.15.37:0/1618477200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61480ccd50 con 0x5619c049ef20 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f6145ffb640 1 -- 172.21.15.37:0/1618477200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f61480afbc0 con 0x5619c049ef20 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.393+0000 7f61675e9740 1 -- 172.21.15.37:0/1618477200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6148010720 con 0x5619c049ef20 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f6145ffb640 1 -- 172.21.15.37:0/1618477200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61480a3f80 con 0x5619c049ef20 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f61675e9740 10 client.15633 fetch_fsmap learned FSMap version 29 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f61675e9740 1 -- 172.21.15.37:0/1618477200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5619c04b6760 con 0x5619c049ef20 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f6145ffb640 1 -- 172.21.15.37:0/1618477200 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6148010720 con 0x5619c049ef20 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f6145ffb640 1 -- 172.21.15.37:0/1618477200 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6148097530 con 0x5619c049ef20 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f61675e9740 10 client.15633 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T06:24:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f61675e9740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f61675e9740 20 client.15633 populate_metadata read hostname 'smithi037' 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f61675e9740 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5619c04b7ab0 0x5619c04b9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f61675e9740 4 client.15633 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f61675e9740 1 -- 172.21.15.37:0/1618477200 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619c040fb30 con 0x5619c04b7ab0 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.394+0000 7f615e59c640 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5619c04b7ab0 0x5619c04b9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f6145ffb640 10 client.15633 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f615e59c640 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5619c04b7ab0 0x5619c04b9ea0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158216e54952668e server_cookie=d832248ba735a2b9 in_seq=0 out_seq=0 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f6145ffb640 1 -- 172.21.15.37:0/1618477200 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619c040fb30 con 0x5619c04b7ab0 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f6145ffb640 10 client.15633 handle_command_reply: tid=0 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f61675e9740 1 client.15633 shutdown 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f61675e9740 1 -- 172.21.15.37:0/1618477200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 msgr2=0x5619c04a4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f61675e9740 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04a4ce0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f614809b4e0 tx=0x7f61480973d0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f61675e9740 1 -- 172.21.15.37:0/1618477200 shutdown_connections 2024-03-26T06:24:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f61675e9740 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5619c04b7ab0 0x5619c04b9ea0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f61675e9740 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c04a7730 0x5619c04a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f61675e9740 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04a4ce0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f61675e9740 1 --2- 172.21.15.37:0/1618477200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049e1b0 0x5619c04a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f6145ffb640 0 client.15633 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.395+0000 7f61675e9740 1 -- 172.21.15.37:0/1618477200 >> 172.21.15.37:0/1618477200 conn(0x5619c04655b0 msgr2=0x5619c049abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.396+0000 7f61675e9740 1 -- 172.21.15.37:0/1618477200 shutdown_connections 2024-03-26T06:24:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.396+0000 7f61675e9740 1 -- 172.21.15.37:0/1618477200 wait complete. 2024-03-26T06:24:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.396+0000 7f61675e9740 20 client.15633 trim_cache size 0 max 16384 2024-03-26T06:24:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f61675e9740 1 Processor -- start 2024-03-26T06:24:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f61675e9740 1 -- start start 2024-03-26T06:24:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f61675e9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049e1b0 0x5619c04a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f61675e9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049ef20 0x5619c04a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f61675e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c04a7730 0x5619c04a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f61675e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601acae0 con 0x5619c04a7730 2024-03-26T06:24:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f61675e9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601ae520 con 0x5619c049ef20 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f61675e9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601ad140 con 0x5619c049e1b0 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f615ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c04a7730 0x5619c04a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f615e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049e1b0 0x5619c04a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f615ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c04a7730 0x5619c04a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57182/0 (socket says 172.21.15.37:57182) 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f615e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049e1b0 0x5619c04a6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:49712/0 (socket says 172.21.15.37:49712) 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f615ed9d640 1 -- 172.21.15.37:0/796431541 learned_addr learned my addr 172.21.15.37:0/796431541 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f6155d9b640 1 --2- 172.21.15.37:0/796431541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049ef20 0x5619c04a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f615ed9d640 1 -- 172.21.15.37:0/796431541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049e1b0 msgr2=0x5619c04a6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.409+0000 7f615ed9d640 1 --2- 172.21.15.37:0/796431541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049e1b0 0x5619c04a6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f615ed9d640 1 -- 172.21.15.37:0/796431541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049ef20 msgr2=0x5619c04a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f615ed9d640 1 --2- 172.21.15.37:0/796431541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049ef20 0x5619c04a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f615ed9d640 1 -- 172.21.15.37:0/796431541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c04a9f80 con 0x5619c04a7730 2024-03-26T06:24:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f615ed9d640 1 --2- 172.21.15.37:0/796431541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c04a7730 0x5619c04a1f80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6158012cd0 tx=0x7f615801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1760f66de021df6e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f6147fff640 1 -- 172.21.15.37:0/796431541 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158026070 con 0x5619c04a7730 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f6147fff640 1 -- 172.21.15.37:0/796431541 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61580045b0 con 0x5619c04a7730 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f6147fff640 1 -- 172.21.15.37:0/796431541 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158013c20 con 0x5619c04a7730 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f61675e9740 1 -- 172.21.15.37:0/796431541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c04a7730 msgr2=0x5619c04a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f61675e9740 1 --2- 172.21.15.37:0/796431541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c04a7730 0x5619c04a1f80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6158012cd0 tx=0x7f615801d000 comp rx=0 tx=0).stop 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f61675e9740 1 -- 172.21.15.37:0/796431541 shutdown_connections 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f61675e9740 1 --2- 172.21.15.37:0/796431541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c04a7730 0x5619c04a1f80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f61675e9740 1 --2- 172.21.15.37:0/796431541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049ef20 0x5619c04a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f61675e9740 1 --2- 172.21.15.37:0/796431541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c049e1b0 0x5619c04a6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f61675e9740 1 -- 172.21.15.37:0/796431541 >> 172.21.15.37:0/796431541 conn(0x5619c04655b0 msgr2=0x5619c04a3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f61675e9740 1 -- 172.21.15.37:0/796431541 shutdown_connections 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.410+0000 7f61675e9740 1 -- 172.21.15.37:0/796431541 wait complete. 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f61675e9740 1 -- start start 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f61675e9740 1 Processor -- start 2024-03-26T06:24:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f61675e9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049e1b0 0x5619c04adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f61675e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f61675e9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c04a7730 0x5619c0507c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f61675e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601ade00 con 0x5619c049ef20 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f61675e9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601ad7a0 con 0x5619c049e1b0 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f61675e9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601aad80 con 0x5619c04a7730 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f61675e9740 10 client.0 _ll_register_callbacks cb 0x7f614800ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f6155d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f6155d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57194/0 (socket says 172.21.15.37:57194) 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f6155d9b640 1 -- 172.21.15.37:0/1786563102 learned_addr learned my addr 172.21.15.37:0/1786563102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f615e59c640 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049e1b0 0x5619c04adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f615ed9d640 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c04a7730 0x5619c0507c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f6155d9b640 1 -- 172.21.15.37:0/1786563102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c04a7730 msgr2=0x5619c0507c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f6155d9b640 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c04a7730 0x5619c0507c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f6155d9b640 1 -- 172.21.15.37:0/1786563102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049e1b0 msgr2=0x5619c04adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f6155d9b640 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049e1b0 0x5619c04adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f6155d9b640 1 -- 172.21.15.37:0/1786563102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c0508240 con 0x5619c049ef20 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.411+0000 7f615ed9d640 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c04a7730 0x5619c0507c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f6155d9b640 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04ae090 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6148009780 tx=0x7f6148010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f8a7d559c63d85 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f6145ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f6145ffb640 1 -- 172.21.15.37:0/1786563102 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148097740 con 0x5619c049ef20 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f6145ffb640 1 -- 172.21.15.37:0/1786563102 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f614809ac30 con 0x5619c049ef20 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f61675e9740 1 -- 172.21.15.37:0/1786563102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61580045b0 con 0x5619c049ef20 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f6145ffb640 1 -- 172.21.15.37:0/1786563102 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6148010720 con 0x5619c049ef20 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f61675e9740 10 client.15639 fetch_fsmap learned FSMap version 29 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f61675e9740 1 -- 172.21.15.37:0/1786563102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5619c0508240 con 0x5619c049ef20 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f6145ffb640 1 -- 172.21.15.37:0/1786563102 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61580045b0 con 0x5619c049ef20 2024-03-26T06:24:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f6145ffb640 1 -- 172.21.15.37:0/1786563102 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f61480cc450 con 0x5619c049ef20 2024-03-26T06:24:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f61675e9740 10 client.15639 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T06:24:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f61675e9740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:24:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f61675e9740 20 client.15639 populate_metadata read hostname 'smithi037' 2024-03-26T06:24:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f61675e9740 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5619c050f9b0 0x5619c0511dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f61675e9740 4 client.15639 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:24:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.412+0000 7f61675e9740 1 -- 172.21.15.37:0/1786563102 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619c0409e80 con 0x5619c050f9b0 2024-03-26T06:24:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.413+0000 7f615e59c640 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5619c050f9b0 0x5619c0511dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.413+0000 7f6145ffb640 10 client.15639 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.413+0000 7f615e59c640 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5619c050f9b0 0x5619c0511dd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482e105e6fb26774 server_cookie=9a1541f46ae797c5 in_seq=0 out_seq=0 2024-03-26T06:24:46.484 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46869 12.85 MB/sec warmup 21 sec latency 808.558 ms 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f6145ffb640 1 -- 172.21.15.37:0/1786563102 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619c0409e80 con 0x5619c050f9b0 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f6145ffb640 10 client.15639 handle_command_reply: tid=0 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 1 client.15639 shutdown 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 1 -- 172.21.15.37:0/1786563102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 msgr2=0x5619c04ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04ae090 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6148009780 tx=0x7f6148010cf0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 1 -- 172.21.15.37:0/1786563102 shutdown_connections 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5619c050f9b0 0x5619c0511dd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619c04a7730 0x5619c0507c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f6145ffb640 0 client.15639 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619c049ef20 0x5619c04ae090 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 1 --2- 172.21.15.37:0/1786563102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5619c049e1b0 0x5619c04adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 1 -- 172.21.15.37:0/1786563102 >> 172.21.15.37:0/1786563102 conn(0x5619c04655b0 msgr2=0x5619c04a4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 1 -- 172.21.15.37:0/1786563102 shutdown_connections 2024-03-26T06:24:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 1 -- 172.21.15.37:0/1786563102 wait complete. 2024-03-26T06:24:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.485+0000 7f61675e9740 20 client.15639 trim_cache size 0 max 16384 2024-03-26T06:24:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:24:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:24:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:24:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:24:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 -- 172.21.15.37:0/2515760681 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f614c0a2960 msgr2=0x7f614c0a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f614c0a2960 0x7f614c0a4e20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6158005fd0 tx=0x7f6158014b70 comp rx=0 tx=0).stop 2024-03-26T06:24:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 -- 172.21.15.37:0/2515760681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6160102bd0 msgr2=0x7f616019dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6160102bd0 0x7f616019dbd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f614800e9f0 tx=0x7f614800eeb0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 -- 172.21.15.37:0/2515760681 shutdown_connections 2024-03-26T06:24:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f614c0a2960 0x7f614c0a4e20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f616010b6d0 0x7f6160197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6160102bd0 0x7f616019dbd0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 --2- 172.21.15.37:0/2515760681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61601022b0 0x7f616019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 -- 172.21.15.37:0/2515760681 >> 172.21.15.37:0/2515760681 conn(0x7f61600fbaf0 msgr2=0x7f61600fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 -- 172.21.15.37:0/2515760681 shutdown_connections 2024-03-26T06:24:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:46.486+0000 7f615f59e640 1 -- 172.21.15.37:0/2515760681 wait complete. 2024-03-26T06:24:47.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:46 smithi066 ceph-mon[36504]: pgmap v56: 97 pgs: 97 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.6 MiB/s wr, 333 op/s 2024-03-26T06:24:47.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:46 smithi149 ceph-mon[36681]: pgmap v56: 97 pgs: 97 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.6 MiB/s wr, 333 op/s 2024-03-26T06:24:47.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:46 smithi037 ceph-mon[31974]: pgmap v56: 97 pgs: 97 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1.6 MiB/s wr, 333 op/s 2024-03-26T06:24:47.484 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46869 12.26 MB/sec warmup 22 sec latency 1808.668 ms 2024-03-26T06:24:48.484 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46869 11.73 MB/sec warmup 23 sec latency 2808.759 ms 2024-03-26T06:24:49.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:48 smithi066 ceph-mon[36504]: pgmap v57: 97 pgs: 97 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 913 KiB/s wr, 123 op/s 2024-03-26T06:24:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:48 smithi149 ceph-mon[36681]: pgmap v57: 97 pgs: 97 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 913 KiB/s wr, 123 op/s 2024-03-26T06:24:49.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:48 smithi037 ceph-mon[31974]: pgmap v57: 97 pgs: 97 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 913 KiB/s wr, 123 op/s 2024-03-26T06:24:49.484 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46869 11.24 MB/sec warmup 24 sec latency 3808.878 ms 2024-03-26T06:24:50.484 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46870 10.79 MB/sec warmup 25 sec latency 4496.139 ms 2024-03-26T06:24:51.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:50 smithi066 ceph-mon[36504]: pgmap v58: 97 pgs: 97 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 913 KiB/s wr, 123 op/s 2024-03-26T06:24:51.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:50 smithi149 ceph-mon[36681]: pgmap v58: 97 pgs: 97 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 913 KiB/s wr, 123 op/s 2024-03-26T06:24:51.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:50 smithi037 ceph-mon[31974]: pgmap v58: 97 pgs: 97 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 913 KiB/s wr, 123 op/s 2024-03-26T06:24:51.484 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46870 10.38 MB/sec warmup 26 sec latency 1312.928 ms 2024-03-26T06:24:52.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:51 smithi066 ceph-mon[36504]: pgmap v59: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 985 KiB/s wr, 196 op/s 2024-03-26T06:24:52.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:51 smithi149 ceph-mon[36681]: pgmap v59: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 985 KiB/s wr, 196 op/s 2024-03-26T06:24:52.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:51 smithi037 ceph-mon[31974]: pgmap v59: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 985 KiB/s wr, 196 op/s 2024-03-26T06:24:52.485 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46937 10.00 MB/sec warmup 27 sec latency 1613.938 ms 2024-03-26T06:24:53.485 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46937 9.64 MB/sec warmup 28 sec latency 1669.203 ms 2024-03-26T06:24:54.097 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:53 smithi066 ceph-mon[36504]: pgmap v60: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 261 KiB/s wr, 102 op/s 2024-03-26T06:24:54.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:53 smithi149 ceph-mon[36681]: pgmap v60: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 261 KiB/s wr, 102 op/s 2024-03-26T06:24:54.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:53 smithi037 ceph-mon[31974]: pgmap v60: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 261 KiB/s wr, 102 op/s 2024-03-26T06:24:54.485 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46937 9.31 MB/sec warmup 29 sec latency 2669.323 ms 2024-03-26T06:24:55.485 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46938 9.00 MB/sec warmup 30 sec latency 3357.449 ms 2024-03-26T06:24:56.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:55 smithi066 ceph-mon[36504]: pgmap v61: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 5.0 MiB/s wr, 229 op/s 2024-03-26T06:24:56.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:55 smithi149 ceph-mon[36681]: pgmap v61: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 5.0 MiB/s wr, 229 op/s 2024-03-26T06:24:56.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:55 smithi037 ceph-mon[31974]: pgmap v61: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 5.0 MiB/s wr, 229 op/s 2024-03-26T06:24:56.485 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46938 8.71 MB/sec warmup 31 sec latency 1312.047 ms 2024-03-26T06:24:57.485 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46938 8.44 MB/sec warmup 32 sec latency 2312.137 ms 2024-03-26T06:24:58.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:57 smithi149 ceph-mon[36681]: pgmap v62: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-03-26T06:24:58.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:57 smithi037 ceph-mon[31974]: pgmap v62: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-03-26T06:24:58.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:57 smithi066 ceph-mon[36504]: pgmap v62: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-03-26T06:24:58.485 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46938 8.18 MB/sec warmup 33 sec latency 3312.190 ms 2024-03-26T06:24:59.485 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46938 7.94 MB/sec warmup 34 sec latency 4312.280 ms 2024-03-26T06:24:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.583+0000 7fa6fce20640 1 -- 172.21.15.37:0/341608187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6f8068970 msgr2=0x7fa6f8068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.583+0000 7fa6fce20640 1 --2- 172.21.15.37:0/341608187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6f8068970 0x7fa6f8068df0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009900 tx=0x7fa6ec02bcd0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.583+0000 7fa6fce20640 1 -- 172.21.15.37:0/341608187 shutdown_connections 2024-03-26T06:24:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.583+0000 7fa6fce20640 1 --2- 172.21.15.37:0/341608187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f8069330 0x7fa6f810e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.583+0000 7fa6fce20640 1 --2- 172.21.15.37:0/341608187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6f8068970 0x7fa6f8068df0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.583+0000 7fa6fce20640 1 --2- 172.21.15.37:0/341608187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6f8107400 0x7fa6f81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.583+0000 7fa6fce20640 1 -- 172.21.15.37:0/341608187 >> 172.21.15.37:0/341608187 conn(0x7fa6f8077040 msgr2=0x7fa6f8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.583+0000 7fa6fce20640 1 -- 172.21.15.37:0/341608187 shutdown_connections 2024-03-26T06:24:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.583+0000 7fa6fce20640 1 -- 172.21.15.37:0/341608187 wait complete. 2024-03-26T06:24:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6fce20640 1 Processor -- start 2024-03-26T06:24:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6fce20640 1 -- start start 2024-03-26T06:24:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6fce20640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6f8068970 0x7fa6f819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6fce20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6f8069330 0x7fa6f819dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6fce20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f8107400 0x7fa6f8197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6fce20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8111640 con 0x7fa6f8069330 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6fce20640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81114c0 con 0x7fa6f8107400 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6fce20640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81117c0 con 0x7fa6f8068970 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6f5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6f8069330 0x7fa6f819dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6f5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6f8069330 0x7fa6f819dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48734/0 (socket says 172.21.15.37:48734) 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6f5d9b640 1 -- 172.21.15.37:0/815457467 learned_addr learned my addr 172.21.15.37:0/815457467 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6f659c640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6f8068970 0x7fa6f819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6f6d9d640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f8107400 0x7fa6f8197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6f5d9b640 1 -- 172.21.15.37:0/815457467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6f8068970 msgr2=0x7fa6f819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6f8068970 0x7fa6f819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6f5d9b640 1 -- 172.21.15.37:0/815457467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f8107400 msgr2=0x7fa6f8197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f8107400 0x7fa6f8197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6f5d9b640 1 -- 172.21.15.37:0/815457467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f81980a0 con 0x7fa6f8069330 2024-03-26T06:24:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.584+0000 7fa6f659c640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6f8068970 0x7fa6f819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:24:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.585+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6f8069330 0x7fa6f819dcb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec006010 tx=0x7fa6ec0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.585+0000 7fa6e77fe640 1 -- 172.21.15.37:0/815457467 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec042020 con 0x7fa6f8069330 2024-03-26T06:24:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.585+0000 7fa6e77fe640 1 -- 172.21.15.37:0/815457467 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec0054e0 con 0x7fa6f8069330 2024-03-26T06:24:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.585+0000 7fa6fce20640 1 -- 172.21.15.37:0/815457467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f8198330 con 0x7fa6f8069330 2024-03-26T06:24:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.585+0000 7fa6e77fe640 1 -- 172.21.15.37:0/815457467 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec003be0 con 0x7fa6f8069330 2024-03-26T06:24:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.585+0000 7fa6fce20640 1 -- 172.21.15.37:0/815457467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f81a4630 con 0x7fa6f8069330 2024-03-26T06:24:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.587+0000 7fa6e77fe640 1 -- 172.21.15.37:0/815457467 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa6ec045020 con 0x7fa6f8069330 2024-03-26T06:24:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.587+0000 7fa6ffa00740 1 Processor -- start 2024-03-26T06:24:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6ffa00740 1 -- start start 2024-03-26T06:24:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6e77fe640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa6e00a2a10 0x7fa6e00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b824041a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b824041fd0 0x55b824042450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82404f960 0x55b824051d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b823fa65f0 con 0x55b82404f960 2024-03-26T06:24:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b823f4f930 con 0x55b824042d40 2024-03-26T06:24:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b823fa2690 con 0x55b824041fd0 2024-03-26T06:24:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6f6d9d640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa6e00a2a10 0x7fa6e00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6e77fe640 1 -- 172.21.15.37:0/815457467 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fa6ec0f0bf0 con 0x7fa6f8069330 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6f659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b824041a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6f659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b824041a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:41242/0 (socket says 172.21.15.37:41242) 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6f659c640 1 -- 172.21.15.37:0/2670840625 learned_addr learned my addr 172.21.15.37:0/2670840625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6f6d9d640 1 --2- 172.21.15.37:0/2670840625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82404f960 0x55b824051d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.588+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/2670840625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b824041fd0 0x55b824042450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.589+0000 7fa6f659c640 1 -- 172.21.15.37:0/2670840625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b824041fd0 msgr2=0x55b824042450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.589+0000 7fa6f659c640 1 --2- 172.21.15.37:0/2670840625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b824041fd0 0x55b824042450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.589+0000 7fa6f659c640 1 -- 172.21.15.37:0/2670840625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82404f960 msgr2=0x55b824051d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.589+0000 7fa6f659c640 1 --2- 172.21.15.37:0/2670840625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82404f960 0x55b824051d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.589+0000 7fa6f659c640 1 -- 172.21.15.37:0/2670840625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b823cbb210 con 0x55b824042d40 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.589+0000 7fa6f6d9d640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa6e00a2a10 0x7fa6e00a4ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00058c0 tx=0x7fa6f00138e0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.589+0000 7fa6f659c640 1 --2- 172.21.15.37:0/2670840625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b824041a90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009f70 tx=0x7fa6e8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae94ace78b7731af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.589+0000 7fa6e4ff9640 1 -- 172.21.15.37:0/2670840625 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8016070 con 0x55b824042d40 2024-03-26T06:24:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.589+0000 7fa6e4ff9640 1 -- 172.21.15.37:0/2670840625 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6e8003b70 con 0x55b824042d40 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.589+0000 7fa6e4ff9640 1 -- 172.21.15.37:0/2670840625 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80127d0 con 0x55b824042d40 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.590+0000 7fa6ffa00740 1 -- 172.21.15.37:0/2670840625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 msgr2=0x55b824041a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.590+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/2670840625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b824041a90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009f70 tx=0x7fa6e8010800 comp rx=0 tx=0).stop 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.590+0000 7fa6ffa00740 1 -- 172.21.15.37:0/2670840625 shutdown_connections 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.590+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/2670840625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82404f960 0x55b824051d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.590+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/2670840625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b824041fd0 0x55b824042450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.590+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/2670840625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b824041a90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.590+0000 7fa6ffa00740 1 -- 172.21.15.37:0/2670840625 >> 172.21.15.37:0/2670840625 conn(0x55b8240095b0 msgr2=0x55b82403ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.590+0000 7fa6ffa00740 1 -- 172.21.15.37:0/2670840625 shutdown_connections 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.590+0000 7fa6ffa00740 1 -- 172.21.15.37:0/2670840625 wait complete. 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.590+0000 7fa6ffa00740 1 -- start start 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.590+0000 7fa6ffa00740 1 Processor -- start 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b82404ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b82404cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b82404f960 0x55b82404d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b824052690 con 0x55b824041fd0 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b823f62d90 con 0x55b824042d40 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b824052810 con 0x55b82404f960 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6ffa00740 10 client.0 _ll_register_callbacks cb 0x55b823cbaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b82404ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b82404ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48754/0 (socket says 172.21.15.37:48754) 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f659c640 1 -- 172.21.15.37:0/4213980598 learned_addr learned my addr 172.21.15.37:0/4213980598 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b82404cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f6d9d640 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b82404f960 0x55b82404d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f659c640 1 -- 172.21.15.37:0/4213980598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b82404f960 msgr2=0x55b82404d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f659c640 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b82404f960 0x55b82404d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f659c640 1 -- 172.21.15.37:0/4213980598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 msgr2=0x55b82404cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f659c640 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b82404cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f659c640 1 -- 172.21.15.37:0/4213980598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b823d87620 con 0x55b824041fd0 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f6d9d640 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b82404f960 0x55b82404d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6f659c640 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b82404ca20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8006010 tx=0x7fa6e8003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f526822691bf203a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.591+0000 7fa6ddffb640 1 -- 172.21.15.37:0/4213980598 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8016070 con 0x55b824041fd0 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ddffb640 1 -- 172.21.15.37:0/4213980598 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fa6e8003970 con 0x55b824041fd0 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ffa00740 1 -- 172.21.15.37:0/4213980598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6e8003b70 con 0x55b824041fd0 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ddffb640 1 -- 172.21.15.37:0/4213980598 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6e8009b40 con 0x55b824041fd0 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ffa00740 10 client.15651 fetch_fsmap learned FSMap version 29 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ffa00740 1 -- 172.21.15.37:0/4213980598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b82405e9f0 con 0x55b824041fd0 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ddffb640 1 -- 172.21.15.37:0/4213980598 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6e8003b70 con 0x55b824041fd0 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ddffb640 1 -- 172.21.15.37:0/4213980598 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa6e800f040 con 0x55b824041fd0 2024-03-26T06:24:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ffa00740 10 client.15651 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ffa00740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ffa00740 20 client.15651 populate_metadata read hostname 'smithi037' 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b82405fe50 0x55b824062240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ffa00740 4 client.15651 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6ffa00740 1 -- 172.21.15.37:0/4213980598 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b823fb3b30 con 0x55b82405fe50 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.592+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b82405fe50 0x55b824062240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.593+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b82405fe50 0x55b824062240 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a9d567294f2590 server_cookie=410c9726befdadf3 in_seq=0 out_seq=0 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.593+0000 7fa6ddffb640 10 client.15651 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.593+0000 7fa6ddffb640 1 -- 172.21.15.37:0/4213980598 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b823fb3b30 con 0x55b82405fe50 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.593+0000 7fa6ddffb640 10 client.15651 handle_command_reply: tid=0 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.593+0000 7fa6ffa00740 1 client.15651 shutdown 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ffa00740 1 -- 172.21.15.37:0/4213980598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 msgr2=0x55b82404ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b82404ca20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8006010 tx=0x7fa6e8003f50 comp rx=0 tx=0).stop 2024-03-26T06:24:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ffa00740 1 -- 172.21.15.37:0/4213980598 shutdown_connections 2024-03-26T06:24:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b82405fe50 0x55b824062240 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b82404f960 0x55b82404d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b82404cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/4213980598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b82404ca20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ddffb640 0 client.15651 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ffa00740 1 -- 172.21.15.37:0/4213980598 >> 172.21.15.37:0/4213980598 conn(0x55b8240095b0 msgr2=0x55b82403ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ffa00740 1 -- 172.21.15.37:0/4213980598 shutdown_connections 2024-03-26T06:24:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ffa00740 1 -- 172.21.15.37:0/4213980598 wait complete. 2024-03-26T06:24:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.594+0000 7fa6ffa00740 20 client.15651 trim_cache size 0 max 16384 2024-03-26T06:24:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.607+0000 7fa6ffa00740 1 Processor -- start 2024-03-26T06:24:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.607+0000 7fa6ffa00740 1 -- start start 2024-03-26T06:24:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.607+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824049ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.607+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b82404a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.607+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b82404f960 0x55b82404c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.607+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81aa950 con 0x55b824041fd0 2024-03-26T06:24:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.607+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81ac390 con 0x55b824042d40 2024-03-26T06:24:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.607+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81aafb0 con 0x55b82404f960 2024-03-26T06:24:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.607+0000 7fa6f659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824049ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6f659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824049ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48758/0 (socket says 172.21.15.37:48758) 2024-03-26T06:24:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6f659c640 1 -- 172.21.15.37:0/399800330 learned_addr learned my addr 172.21.15.37:0/399800330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6f6d9d640 1 --2- 172.21.15.37:0/399800330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b82404f960 0x55b82404c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/399800330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b82404a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6f659c640 1 -- 172.21.15.37:0/399800330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b82404f960 msgr2=0x55b82404c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6f659c640 1 --2- 172.21.15.37:0/399800330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b82404f960 0x55b82404c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6f659c640 1 -- 172.21.15.37:0/399800330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 msgr2=0x55b82404a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6f659c640 1 --2- 172.21.15.37:0/399800330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b82404a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6f659c640 1 -- 172.21.15.37:0/399800330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8240412a0 con 0x55b824041fd0 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/399800330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b82404a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6f659c640 1 --2- 172.21.15.37:0/399800330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824049ca0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8025470 tx=0x7fa6e8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f15491e40b886c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6dffff640 1 -- 172.21.15.37:0/399800330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8032030 con 0x55b824041fd0 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6dffff640 1 -- 172.21.15.37:0/399800330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6e8003970 con 0x55b824041fd0 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.608+0000 7fa6dffff640 1 -- 172.21.15.37:0/399800330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e801e020 con 0x55b824041fd0 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 -- 172.21.15.37:0/399800330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 msgr2=0x55b824049ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/399800330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824049ca0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8025470 tx=0x7fa6e8005bd0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 -- 172.21.15.37:0/399800330 shutdown_connections 2024-03-26T06:24:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/399800330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b82404f960 0x55b82404c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/399800330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b824042d40 0x55b82404a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/399800330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824049ca0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 -- 172.21.15.37:0/399800330 >> 172.21.15.37:0/399800330 conn(0x55b8240095b0 msgr2=0x55b82404b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 -- 172.21.15.37:0/399800330 shutdown_connections 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 -- 172.21.15.37:0/399800330 wait complete. 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 -- start start 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 Processor -- start 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824056ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b824042d40 0x55b824055280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b82404f960 0x55b824055900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81abc70 con 0x55b824041fd0 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81ab610 con 0x55b82404f960 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81a8c10 con 0x55b824042d40 2024-03-26T06:24:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6ffa00740 10 client.0 _ll_register_callbacks cb 0x7fa6e8009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6f659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824056ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.609+0000 7fa6f659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824056ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48762/0 (socket says 172.21.15.37:48762) 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6f659c640 1 -- 172.21.15.37:0/3023067044 learned_addr learned my addr 172.21.15.37:0/3023067044 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6f6d9d640 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b82404f960 0x55b824055900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b824042d40 0x55b824055280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6f659c640 1 -- 172.21.15.37:0/3023067044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b824042d40 msgr2=0x55b824055280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6f659c640 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b824042d40 0x55b824055280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6f659c640 1 -- 172.21.15.37:0/3023067044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b82404f960 msgr2=0x55b824055900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6f659c640 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b82404f960 0x55b824055900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6f659c640 1 -- 172.21.15.37:0/3023067044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8240b61d0 con 0x55b824041fd0 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b824042d40 0x55b824055280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:24:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6f659c640 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824056ee0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8010840 tx=0x7fa6e8003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cef156df53072f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6ddffb640 1 -- 172.21.15.37:0/3023067044 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e803c030 con 0x55b824041fd0 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6ddffb640 1 -- 172.21.15.37:0/3023067044 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fa6e8021030 con 0x55b824041fd0 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6ffa00740 1 -- 172.21.15.37:0/3023067044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6e8003970 con 0x55b824041fd0 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6ddffb640 1 -- 172.21.15.37:0/3023067044 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6e8046050 con 0x55b824041fd0 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6ffa00740 10 client.15657 fetch_fsmap learned FSMap version 29 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6ffa00740 1 -- 172.21.15.37:0/3023067044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b8240b7310 con 0x55b824041fd0 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.610+0000 7fa6ddffb640 1 -- 172.21.15.37:0/3023067044 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6e8003970 con 0x55b824041fd0 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.611+0000 7fa6ddffb640 1 -- 172.21.15.37:0/3023067044 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa6e8012550 con 0x55b824041fd0 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.611+0000 7fa6ffa00740 10 client.15657 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.611+0000 7fa6ffa00740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.611+0000 7fa6ffa00740 20 client.15657 populate_metadata read hostname 'smithi037' 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.611+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b8240b7a10 0x55b8240b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:24:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.611+0000 7fa6ffa00740 4 client.15657 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:24:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.611+0000 7fa6ffa00740 1 -- 172.21.15.37:0/3023067044 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b823fade80 con 0x55b8240b7a10 2024-03-26T06:24:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.611+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b8240b7a10 0x55b8240b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:24:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.611+0000 7fa6f5d9b640 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b8240b7a10 0x55b8240b9e30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43b78c067fb71970 server_cookie=855a08a39e75017 in_seq=0 out_seq=0 2024-03-26T06:24:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.611+0000 7fa6ddffb640 10 client.15657 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.640+0000 7fa6ddffb640 1 -- 172.21.15.37:0/3023067044 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b823fade80 con 0x55b8240b7a10 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.640+0000 7fa6ddffb640 10 client.15657 handle_command_reply: tid=0 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.640+0000 7fa6ffa00740 1 client.15657 shutdown 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ffa00740 1 -- 172.21.15.37:0/3023067044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 msgr2=0x55b824056ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824056ee0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8010840 tx=0x7fa6e8003f30 comp rx=0 tx=0).stop 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ffa00740 1 -- 172.21.15.37:0/3023067044 shutdown_connections 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b8240b7a10 0x55b8240b9e30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b82404f960 0x55b824055900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ddffb640 0 client.15657 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b824042d40 0x55b824055280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ffa00740 1 --2- 172.21.15.37:0/3023067044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b824041fd0 0x55b824056ee0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ffa00740 1 -- 172.21.15.37:0/3023067044 >> 172.21.15.37:0/3023067044 conn(0x55b8240095b0 msgr2=0x55b82403b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ffa00740 1 -- 172.21.15.37:0/3023067044 shutdown_connections 2024-03-26T06:24:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ffa00740 1 -- 172.21.15.37:0/3023067044 wait complete. 2024-03-26T06:24:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.641+0000 7fa6ffa00740 20 client.15657 trim_cache size 0 max 16384 2024-03-26T06:24:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:24:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:24:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:24:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 -- 172.21.15.37:0/815457467 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa6e00a2a10 msgr2=0x7fa6e00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa6e00a2a10 0x7fa6e00a4ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00058c0 tx=0x7fa6f00138e0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 -- 172.21.15.37:0/815457467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6f8069330 msgr2=0x7fa6f819dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6f8069330 0x7fa6f819dcb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec006010 tx=0x7fa6ec0043b0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 -- 172.21.15.37:0/815457467 shutdown_connections 2024-03-26T06:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa6e00a2a10 0x7fa6e00a4ed0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f8107400 0x7fa6f8197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6f8069330 0x7fa6f819dcb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 --2- 172.21.15.37:0/815457467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6f8068970 0x7fa6f819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:24:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 -- 172.21.15.37:0/815457467 >> 172.21.15.37:0/815457467 conn(0x7fa6f8077040 msgr2=0x7fa6f80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:24:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 -- 172.21.15.37:0/815457467 shutdown_connections 2024-03-26T06:24:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:24:59.642+0000 7fa6fce20640 1 -- 172.21.15.37:0/815457467 wait complete. 2024-03-26T06:25:00.150 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:24:59 smithi066 ceph-mon[36504]: pgmap v63: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-03-26T06:25:00.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:24:59 smithi149 ceph-mon[36681]: pgmap v63: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-03-26T06:25:00.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:24:59 smithi037 ceph-mon[31974]: pgmap v63: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-03-26T06:25:00.485 INFO:tasks.workunit.client.0.smithi037.stdout: 1 47303 7.76 MB/sec warmup 35 sec latency 4994.347 ms 2024-03-26T06:25:01.485 INFO:tasks.workunit.client.0.smithi037.stdout: 1 48905 7.79 MB/sec warmup 36 sec latency 479.085 ms 2024-03-26T06:25:02.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:01 smithi149 ceph-mon[36681]: pgmap v64: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 4.9 MiB/s wr, 215 op/s 2024-03-26T06:25:02.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:01 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:25:02.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:01 smithi037 ceph-mon[31974]: pgmap v64: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 4.9 MiB/s wr, 215 op/s 2024-03-26T06:25:02.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:01 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:25:02.347 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:01 smithi066 ceph-mon[36504]: pgmap v64: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 4.9 MiB/s wr, 215 op/s 2024-03-26T06:25:02.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:01 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:25:02.485 INFO:tasks.workunit.client.0.smithi037.stdout: 1 48905 7.58 MB/sec warmup 37 sec latency 1479.177 ms 2024-03-26T06:25:03.486 INFO:tasks.workunit.client.0.smithi037.stdout: 1 48905 7.38 MB/sec warmup 38 sec latency 2479.264 ms 2024-03-26T06:25:04.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:03 smithi149 ceph-mon[36681]: pgmap v65: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 4.8 MiB/s wr, 141 op/s 2024-03-26T06:25:04.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:03 smithi037 ceph-mon[31974]: pgmap v65: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 4.8 MiB/s wr, 141 op/s 2024-03-26T06:25:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:03 smithi066 ceph-mon[36504]: pgmap v65: 97 pgs: 97 active+clean; 26 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 4.8 MiB/s wr, 141 op/s 2024-03-26T06:25:04.486 INFO:tasks.workunit.client.0.smithi037.stdout: 1 48905 7.19 MB/sec warmup 39 sec latency 3479.387 ms 2024-03-26T06:25:05.486 INFO:tasks.workunit.client.0.smithi037.stdout: 1 48905 7.01 MB/sec warmup 40 sec latency 4479.440 ms 2024-03-26T06:25:06.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:05 smithi149 ceph-mon[36681]: pgmap v66: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 4.9 MiB/s wr, 181 op/s 2024-03-26T06:25:06.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:05 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:25:06.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:05 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:05 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:05 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:05 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:05 smithi037 ceph-mon[31974]: pgmap v66: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 4.9 MiB/s wr, 181 op/s 2024-03-26T06:25:06.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:05 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:25:06.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:05 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:05 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:05 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:05 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:05 smithi066 ceph-mon[36504]: pgmap v66: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 4.9 MiB/s wr, 181 op/s 2024-03-26T06:25:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:05 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:25:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:05 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:05 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:05 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:05 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:06.486 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49045 6.86 MB/sec warmup 41 sec latency 4810.498 ms 2024-03-26T06:25:07.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:06 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:25:07.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:06 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:25:07.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:06 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:25:07.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:06 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:07.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:06 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:25:07.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:06 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:25:07.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:06 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:25:07.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:06 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:07.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:06 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:25:07.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:06 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:25:07.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:06 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:25:07.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:06 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:25:07.486 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49045 6.70 MB/sec warmup 42 sec latency 1622.908 ms 2024-03-26T06:25:08.486 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49045 6.54 MB/sec warmup 43 sec latency 2623.013 ms 2024-03-26T06:25:08.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:08 smithi149 ceph-mon[36681]: pgmap v67: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 155 KiB/s wr, 54 op/s 2024-03-26T06:25:08.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:08 smithi037 ceph-mon[31974]: pgmap v67: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 155 KiB/s wr, 54 op/s 2024-03-26T06:25:08.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:08 smithi066 ceph-mon[36504]: pgmap v67: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 155 KiB/s wr, 54 op/s 2024-03-26T06:25:09.486 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49045 6.40 MB/sec warmup 44 sec latency 3623.164 ms 2024-03-26T06:25:10.486 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49045 6.25 MB/sec warmup 45 sec latency 4623.263 ms 2024-03-26T06:25:10.662 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:10 smithi149 ceph-mon[36681]: pgmap v68: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 155 KiB/s wr, 54 op/s 2024-03-26T06:25:10.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:10 smithi037 ceph-mon[31974]: pgmap v68: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 155 KiB/s wr, 54 op/s 2024-03-26T06:25:10.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:10 smithi066 ceph-mon[36504]: pgmap v68: 97 pgs: 97 active+clean; 27 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 155 KiB/s wr, 54 op/s 2024-03-26T06:25:11.486 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49074 6.13 MB/sec warmup 46 sec latency 4956.591 ms 2024-03-26T06:25:12.487 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49074 6.00 MB/sec warmup 47 sec latency 1644.945 ms 2024-03-26T06:25:12.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:12 smithi149 ceph-mon[36681]: pgmap v69: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 166 KiB/s wr, 62 op/s 2024-03-26T06:25:12.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:12 smithi037 ceph-mon[31974]: pgmap v69: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 166 KiB/s wr, 62 op/s 2024-03-26T06:25:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.739+0000 7ff25272b640 1 -- 172.21.15.37:0/2557368381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff24c100450 msgr2=0x7ff24c105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.739+0000 7ff25272b640 1 --2- 172.21.15.37:0/2557368381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff24c100450 0x7ff24c105f80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff240009930 tx=0x7ff24002bc80 comp rx=0 tx=0).stop 2024-03-26T06:25:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.739+0000 7ff25272b640 1 -- 172.21.15.37:0/2557368381 shutdown_connections 2024-03-26T06:25:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.739+0000 7ff25272b640 1 --2- 172.21.15.37:0/2557368381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24c1066b0 0x7ff24c10e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.739+0000 7ff25272b640 1 --2- 172.21.15.37:0/2557368381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff24c100450 0x7ff24c105f80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.739+0000 7ff25272b640 1 --2- 172.21.15.37:0/2557368381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff24c0ffaa0 0x7ff24c0ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.739+0000 7ff25272b640 1 -- 172.21.15.37:0/2557368381 >> 172.21.15.37:0/2557368381 conn(0x7ff24c0767b0 msgr2=0x7ff24c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.739+0000 7ff25272b640 1 -- 172.21.15.37:0/2557368381 shutdown_connections 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.739+0000 7ff25272b640 1 -- 172.21.15.37:0/2557368381 wait complete. 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.740+0000 7ff25272b640 1 Processor -- start 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.740+0000 7ff25272b640 1 -- start start 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.740+0000 7ff25272b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff24c0ffaa0 0x7ff24c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.740+0000 7ff25272b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24c1066b0 0x7ff24c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.740+0000 7ff25272b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff24c197940 0x7ff24c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.740+0000 7ff25272b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c111680 con 0x7ff24c197940 2024-03-26T06:25:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.740+0000 7ff25272b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c111500 con 0x7ff24c0ffaa0 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.740+0000 7ff25272b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c111800 con 0x7ff24c1066b0 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff250cc8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff24c197940 0x7ff24c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff250cc8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff24c197940 0x7ff24c197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57438/0 (socket says 172.21.15.37:57438) 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff250cc8640 1 -- 172.21.15.37:0/2116742098 learned_addr learned my addr 172.21.15.37:0/2116742098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff24bfff640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff24c0ffaa0 0x7ff24c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24c1066b0 0x7ff24c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff250cc8640 1 -- 172.21.15.37:0/2116742098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24c1066b0 msgr2=0x7ff24c19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff250cc8640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24c1066b0 0x7ff24c19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff250cc8640 1 -- 172.21.15.37:0/2116742098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff24c0ffaa0 msgr2=0x7ff24c19d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff250cc8640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff24c0ffaa0 0x7ff24c19d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff250cc8640 1 -- 172.21.15.37:0/2116742098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c198460 con 0x7ff24c197940 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24c1066b0 0x7ff24c19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:25:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff24bfff640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff24c0ffaa0 0x7ff24c19d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:25:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff250cc8640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff24c197940 0x7ff24c197df0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff24400eaa0 tx=0x7ff24400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff2497fa640 1 -- 172.21.15.37:0/2116742098 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24400cdb0 con 0x7ff24c197940 2024-03-26T06:25:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff2497fa640 1 -- 172.21.15.37:0/2116742098 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff244005020 con 0x7ff24c197940 2024-03-26T06:25:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff25272b640 1 -- 172.21.15.37:0/2116742098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24c198750 con 0x7ff24c197940 2024-03-26T06:25:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff2497fa640 1 -- 172.21.15.37:0/2116742098 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440116b0 con 0x7ff24c197940 2024-03-26T06:25:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.741+0000 7ff25272b640 1 -- 172.21.15.37:0/2116742098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24c1a4900 con 0x7ff24c197940 2024-03-26T06:25:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff2497fa640 1 -- 172.21.15.37:0/2116742098 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff244005560 con 0x7ff24c197940 2024-03-26T06:25:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff2553e0740 1 Processor -- start 2024-03-26T06:25:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff2553e0740 1 -- start start 2024-03-26T06:25:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff2497fa640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2340a2a10 0x7ff2340a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff24444d40 0x55ff24443a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff24444450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff2444d550 0x55ff2444f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff2553e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff243a85f0 con 0x55ff24443fd0 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff2553e0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff24351930 con 0x55ff2444d550 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff2553e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff243a4690 con 0x55ff24444d40 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff2497fa640 1 -- 172.21.15.37:0/2116742098 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff2440cf0a0 con 0x7ff24c197940 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.744+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2340a2a10 0x7ff2340a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff24444450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff24444450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57440/0 (socket says 172.21.15.37:57440) 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff24b7fe640 1 -- 172.21.15.37:0/402378142 learned_addr learned my addr 172.21.15.37:0/402378142 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff24bfff640 1 --2- 172.21.15.37:0/402378142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff24444d40 0x55ff24443a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff250cc8640 1 --2- 172.21.15.37:0/402378142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff2444d550 0x55ff2444f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff24b7fe640 1 -- 172.21.15.37:0/402378142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff24444d40 msgr2=0x55ff24443a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/402378142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff24444d40 0x55ff24443a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff24b7fe640 1 -- 172.21.15.37:0/402378142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff2444d550 msgr2=0x55ff2444f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/402378142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff2444d550 0x55ff2444f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff24b7fe640 1 -- 172.21.15.37:0/402378142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff240bd210 con 0x55ff24443fd0 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2340a2a10 0x7ff2340a4ed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff24c069420 tx=0x7ff240005e60 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/402378142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff24444450 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff240035000 tx=0x7ff240046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1f077d4bd0d615 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff2327fc640 1 -- 172.21.15.37:0/402378142 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240005300 con 0x55ff24443fd0 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff2327fc640 1 -- 172.21.15.37:0/402378142 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2400054a0 con 0x55ff24443fd0 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.745+0000 7ff2327fc640 1 -- 172.21.15.37:0/402378142 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240003920 con 0x55ff24443fd0 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.746+0000 7ff2553e0740 1 -- 172.21.15.37:0/402378142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 msgr2=0x55ff24444450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.746+0000 7ff2553e0740 1 --2- 172.21.15.37:0/402378142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff24444450 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff240035000 tx=0x7ff240046000 comp rx=0 tx=0).stop 2024-03-26T06:25:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.746+0000 7ff2553e0740 1 -- 172.21.15.37:0/402378142 shutdown_connections 2024-03-26T06:25:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.746+0000 7ff2553e0740 1 --2- 172.21.15.37:0/402378142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff2444d550 0x55ff2444f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.746+0000 7ff2553e0740 1 --2- 172.21.15.37:0/402378142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff24444450 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.746+0000 7ff2553e0740 1 --2- 172.21.15.37:0/402378142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff24444d40 0x55ff24443a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.746+0000 7ff2553e0740 1 -- 172.21.15.37:0/402378142 >> 172.21.15.37:0/402378142 conn(0x55ff2440b5b0 msgr2=0x55ff2443ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.746+0000 7ff2553e0740 1 -- 172.21.15.37:0/402378142 shutdown_connections 2024-03-26T06:25:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.746+0000 7ff2553e0740 1 -- 172.21.15.37:0/402378142 wait complete. 2024-03-26T06:25:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.746+0000 7ff2553e0740 1 -- start start 2024-03-26T06:25:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff2553e0740 1 Processor -- start 2024-03-26T06:25:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24443fd0 0x55ff2444a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff2444ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff2444b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff2553e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff24450220 con 0x55ff24444d40 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff2553e0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff24364d90 con 0x55ff24443fd0 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff2553e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff244503a0 con 0x55ff2444d550 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff2553e0740 10 client.0 _ll_register_callbacks cb 0x55ff240bca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff2444ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff2444ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57452/0 (socket says 172.21.15.37:57452) 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff24b7fe640 1 -- 172.21.15.37:0/255417817 learned_addr learned my addr 172.21.15.37:0/255417817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff24bfff640 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24443fd0 0x55ff2444a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff250cc8640 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff2444b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff24b7fe640 1 -- 172.21.15.37:0/255417817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 msgr2=0x55ff2444b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff2444b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff24b7fe640 1 -- 172.21.15.37:0/255417817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24443fd0 msgr2=0x55ff2444a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24443fd0 0x55ff2444a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.747+0000 7ff24b7fe640 1 -- 172.21.15.37:0/255417817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff24189620 con 0x55ff24444d40 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff2444ab00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff2400048f0 tx=0x7ff240004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e60dcd3b90894d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff22b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff22b7fe640 1 -- 172.21.15.37:0/255417817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240003660 con 0x55ff24444d40 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff22b7fe640 1 -- 172.21.15.37:0/255417817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff240003800 con 0x55ff24444d40 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff2553e0740 1 -- 172.21.15.37:0/255417817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2400054a0 con 0x55ff24444d40 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff22b7fe640 1 -- 172.21.15.37:0/255417817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff240003a00 con 0x55ff24444d40 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff2553e0740 10 client.15669 fetch_fsmap learned FSMap version 31 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff2553e0740 1 -- 172.21.15.37:0/255417817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ff24450220 con 0x55ff24444d40 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff22b7fe640 1 -- 172.21.15.37:0/255417817 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2400054a0 con 0x55ff24444d40 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff22b7fe640 1 -- 172.21.15.37:0/255417817 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff2400429e0 con 0x55ff24444d40 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff2553e0740 10 client.15669 fetch_fsmap finished waiting for FSMap version 31 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff2553e0740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.748+0000 7ff2553e0740 20 client.15669 populate_metadata read hostname 'smithi037' 2024-03-26T06:25:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.749+0000 7ff2553e0740 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ff2445d720 0x55ff2445fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.749+0000 7ff2553e0740 4 client.15669 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.749+0000 7ff2553e0740 1 -- 172.21.15.37:0/255417817 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff243b5b30 con 0x55ff2445d720 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.749+0000 7ff24bfff640 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ff2445d720 0x55ff2445fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.749+0000 7ff22b7fe640 10 client.15669 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.749+0000 7ff24bfff640 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ff2445d720 0x55ff2445fb10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e08b08060f9b21 server_cookie=91bfd17649a490cd in_seq=0 out_seq=0 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff22b7fe640 1 -- 172.21.15.37:0/255417817 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff243b5b30 con 0x55ff2445d720 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff22b7fe640 10 client.15669 handle_command_reply: tid=0 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 1 client.15669 shutdown 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 1 -- 172.21.15.37:0/255417817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 msgr2=0x55ff2444ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff2444ab00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff2400048f0 tx=0x7ff240004e90 comp rx=0 tx=0).stop 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 1 -- 172.21.15.37:0/255417817 shutdown_connections 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ff2445d720 0x55ff2445fb10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff2444b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff2444ab00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 1 --2- 172.21.15.37:0/255417817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24443fd0 0x55ff2444a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff22b7fe640 0 client.15669 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 1 -- 172.21.15.37:0/255417817 >> 172.21.15.37:0/255417817 conn(0x55ff2440b5b0 msgr2=0x55ff24440a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 1 -- 172.21.15.37:0/255417817 shutdown_connections 2024-03-26T06:25:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 1 -- 172.21.15.37:0/255417817 wait complete. 2024-03-26T06:25:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.750+0000 7ff2553e0740 20 client.15669 trim_cache size 0 max 16384 2024-03-26T06:25:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff2553e0740 1 Processor -- start 2024-03-26T06:25:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff2553e0740 1 -- start start 2024-03-26T06:25:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff2444ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24444d40 0x55ff24454660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff24454ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff2553e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1acbb0 con 0x55ff24443fd0 2024-03-26T06:25:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff2553e0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1ae650 con 0x55ff24444d40 2024-03-26T06:25:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff2553e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1ad210 con 0x55ff2444d550 2024-03-26T06:25:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff2444ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff2444ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57466/0 (socket says 172.21.15.37:57466) 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff24bfff640 1 -- 172.21.15.37:0/182752587 learned_addr learned my addr 172.21.15.37:0/182752587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/182752587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24444d40 0x55ff24454660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff250cc8640 1 --2- 172.21.15.37:0/182752587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff24454ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff24bfff640 1 -- 172.21.15.37:0/182752587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 msgr2=0x55ff24454ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff24bfff640 1 --2- 172.21.15.37:0/182752587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff24454ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff24bfff640 1 -- 172.21.15.37:0/182752587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24444d40 msgr2=0x55ff24454660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff24bfff640 1 --2- 172.21.15.37:0/182752587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24444d40 0x55ff24454660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff24bfff640 1 -- 172.21.15.37:0/182752587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff244432a0 con 0x55ff24443fd0 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/182752587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24444d40 0x55ff24454660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.764+0000 7ff24bfff640 1 --2- 172.21.15.37:0/182752587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff2444ccf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0126a0 tx=0x7ff23c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def99a4eae207d0b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff231ffb640 1 -- 172.21.15.37:0/182752587 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c024070 con 0x55ff24443fd0 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff231ffb640 1 -- 172.21.15.37:0/182752587 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff23c0044d0 con 0x55ff24443fd0 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff231ffb640 1 -- 172.21.15.37:0/182752587 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c020460 con 0x55ff24443fd0 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff2553e0740 1 -- 172.21.15.37:0/182752587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 msgr2=0x55ff2444ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff2553e0740 1 --2- 172.21.15.37:0/182752587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff2444ccf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0126a0 tx=0x7ff23c005f90 comp rx=0 tx=0).stop 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff2553e0740 1 -- 172.21.15.37:0/182752587 shutdown_connections 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff2553e0740 1 --2- 172.21.15.37:0/182752587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff24454ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff2553e0740 1 --2- 172.21.15.37:0/182752587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24444d40 0x55ff24454660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff2553e0740 1 --2- 172.21.15.37:0/182752587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24443fd0 0x55ff2444ccf0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff2553e0740 1 -- 172.21.15.37:0/182752587 >> 172.21.15.37:0/182752587 conn(0x55ff2440b5b0 msgr2=0x55ff2443e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff2553e0740 1 -- 172.21.15.37:0/182752587 shutdown_connections 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.765+0000 7ff2553e0740 1 -- 172.21.15.37:0/182752587 wait complete. 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff2553e0740 1 -- start start 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff2553e0740 1 Processor -- start 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24443fd0 0x55ff24451140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff24451680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff2553e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff24451bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff2553e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1adf30 con 0x55ff24444d40 2024-03-26T06:25:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff2553e0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1ad8d0 con 0x55ff24443fd0 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff2553e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1aae70 con 0x55ff2444d550 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff2553e0740 10 client.0 _ll_register_callbacks cb 0x7ff23c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff24451680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff24451680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57480/0 (socket says 172.21.15.37:57480) 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff24b7fe640 1 -- 172.21.15.37:0/3492894010 learned_addr learned my addr 172.21.15.37:0/3492894010 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff24bfff640 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24443fd0 0x55ff24451140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.766+0000 7ff250cc8640 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff24451bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff24b7fe640 1 -- 172.21.15.37:0/3492894010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 msgr2=0x55ff24451bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff24451bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff24b7fe640 1 -- 172.21.15.37:0/3492894010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24443fd0 msgr2=0x55ff24451140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24443fd0 0x55ff24451140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff24b7fe640 1 -- 172.21.15.37:0/3492894010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff244b3b00 con 0x55ff24444d40 2024-03-26T06:25:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff24b7fe640 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff24451680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff2400092a0 tx=0x7ff24002c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d844af57a9b1475 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff22b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff22b7fe640 1 -- 172.21.15.37:0/3492894010 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240004920 con 0x55ff24444d40 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff22b7fe640 1 -- 172.21.15.37:0/3492894010 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff240003800 con 0x55ff24444d40 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff2553e0740 1 -- 172.21.15.37:0/3492894010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff23c0044d0 con 0x55ff24444d40 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff22b7fe640 1 -- 172.21.15.37:0/3492894010 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff24006e050 con 0x55ff24444d40 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff2553e0740 10 client.15675 fetch_fsmap learned FSMap version 31 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff2553e0740 1 -- 172.21.15.37:0/3492894010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ff244b4d80 con 0x55ff24444d40 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff22b7fe640 1 -- 172.21.15.37:0/3492894010 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff23c0044d0 con 0x55ff24444d40 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff22b7fe640 1 -- 172.21.15.37:0/3492894010 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff240067060 con 0x55ff24444d40 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.767+0000 7ff2553e0740 10 client.15675 fetch_fsmap finished waiting for FSMap version 31 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.768+0000 7ff2553e0740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.768+0000 7ff2553e0740 20 client.15675 populate_metadata read hostname 'smithi037' 2024-03-26T06:25:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.768+0000 7ff2553e0740 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ff244b5560 0x55ff244b7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.768+0000 7ff2553e0740 4 client.15675 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:25:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.768+0000 7ff2553e0740 1 -- 172.21.15.37:0/3492894010 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff243afe80 con 0x55ff244b5560 2024-03-26T06:25:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.768+0000 7ff24bfff640 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ff244b5560 0x55ff244b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.768+0000 7ff24bfff640 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ff244b5560 0x55ff244b7980 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e96c2fafd5d3bc5 server_cookie=f02719986d6a380e in_seq=0 out_seq=0 2024-03-26T06:25:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.768+0000 7ff22b7fe640 10 client.15675 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff22b7fe640 1 -- 172.21.15.37:0/3492894010 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff243afe80 con 0x55ff244b5560 2024-03-26T06:25:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff22b7fe640 10 client.15675 handle_command_reply: tid=0 2024-03-26T06:25:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff2553e0740 1 client.15675 shutdown 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff2553e0740 1 -- 172.21.15.37:0/3492894010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 msgr2=0x55ff24451680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff2553e0740 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff24451680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff2400092a0 tx=0x7ff24002c4a0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff2553e0740 1 -- 172.21.15.37:0/3492894010 shutdown_connections 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff2553e0740 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ff244b5560 0x55ff244b7980 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff2553e0740 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff2444d550 0x55ff24451bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff2553e0740 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ff24444d40 0x55ff24451680 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff2553e0740 1 --2- 172.21.15.37:0/3492894010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ff24443fd0 0x55ff24451140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff22b7fe640 0 client.15675 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.794+0000 7ff2553e0740 1 -- 172.21.15.37:0/3492894010 >> 172.21.15.37:0/3492894010 conn(0x55ff2440b5b0 msgr2=0x55ff24448dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff2553e0740 1 -- 172.21.15.37:0/3492894010 shutdown_connections 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff2553e0740 1 -- 172.21.15.37:0/3492894010 wait complete. 2024-03-26T06:25:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff2553e0740 20 client.15675 trim_cache size 0 max 16384 2024-03-26T06:25:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:25:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:25:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:25:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:25:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 -- 172.21.15.37:0/2116742098 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2340a2a10 msgr2=0x7ff2340a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2340a2a10 0x7ff2340a4ed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff24c069420 tx=0x7ff240005e60 comp rx=0 tx=0).stop 2024-03-26T06:25:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 -- 172.21.15.37:0/2116742098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff24c197940 msgr2=0x7ff24c197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff24c197940 0x7ff24c197df0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff24400eaa0 tx=0x7ff24400ef60 comp rx=0 tx=0).stop 2024-03-26T06:25:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 -- 172.21.15.37:0/2116742098 shutdown_connections 2024-03-26T06:25:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2340a2a10 0x7ff2340a4ed0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff24c197940 0x7ff24c197df0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24c1066b0 0x7ff24c19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 --2- 172.21.15.37:0/2116742098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff24c0ffaa0 0x7ff24c19d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 -- 172.21.15.37:0/2116742098 >> 172.21.15.37:0/2116742098 conn(0x7ff24c0767b0 msgr2=0x7ff24c0fbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 -- 172.21.15.37:0/2116742098 shutdown_connections 2024-03-26T06:25:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:12.795+0000 7ff25272b640 1 -- 172.21.15.37:0/2116742098 wait complete. 2024-03-26T06:25:12.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:12 smithi066 ceph-mon[36504]: pgmap v69: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 166 KiB/s wr, 62 op/s 2024-03-26T06:25:13.486 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49074 5.88 MB/sec warmup 48 sec latency 2644.998 ms 2024-03-26T06:25:14.487 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49074 5.76 MB/sec warmup 49 sec latency 3645.077 ms 2024-03-26T06:25:14.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:14 smithi149 ceph-mon[36681]: pgmap v70: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 153 KiB/s wr, 47 op/s 2024-03-26T06:25:14.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:14 smithi037 ceph-mon[31974]: pgmap v70: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 153 KiB/s wr, 47 op/s 2024-03-26T06:25:14.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:14 smithi066 ceph-mon[36504]: pgmap v70: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 153 KiB/s wr, 47 op/s 2024-03-26T06:25:15.487 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49074 5.64 MB/sec warmup 50 sec latency 4645.181 ms 2024-03-26T06:25:16.487 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49352 5.55 MB/sec warmup 51 sec latency 4982.316 ms 2024-03-26T06:25:16.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:16 smithi149 ceph-mon[36681]: pgmap v71: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 171 KiB/s wr, 62 op/s 2024-03-26T06:25:16.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:16 smithi037 ceph-mon[31974]: pgmap v71: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 171 KiB/s wr, 62 op/s 2024-03-26T06:25:16.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:16 smithi066 ceph-mon[36504]: pgmap v71: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 171 KiB/s wr, 62 op/s 2024-03-26T06:25:17.487 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49352 5.44 MB/sec warmup 52 sec latency 1541.078 ms 2024-03-26T06:25:18.487 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49352 5.34 MB/sec warmup 53 sec latency 2541.133 ms 2024-03-26T06:25:18.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:18 smithi149 ceph-mon[36681]: pgmap v72: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-03-26T06:25:18.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:18 smithi037 ceph-mon[31974]: pgmap v72: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-03-26T06:25:18.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:18 smithi066 ceph-mon[36504]: pgmap v72: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-03-26T06:25:19.487 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49352 5.24 MB/sec warmup 54 sec latency 3541.250 ms 2024-03-26T06:25:20.487 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49352 5.14 MB/sec warmup 55 sec latency 4541.297 ms 2024-03-26T06:25:20.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:20 smithi149 ceph-mon[36681]: pgmap v73: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-03-26T06:25:20.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:20 smithi037 ceph-mon[31974]: pgmap v73: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-03-26T06:25:20.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:20 smithi066 ceph-mon[36504]: pgmap v73: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-03-26T06:25:21.487 INFO:tasks.workunit.client.0.smithi037.stdout: 1 50198 5.15 MB/sec warmup 56 sec latency 4881.134 ms 2024-03-26T06:25:22.487 INFO:tasks.workunit.client.0.smithi037.stdout: 1 50411 5.07 MB/sec warmup 57 sec latency 850.581 ms 2024-03-26T06:25:22.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:22 smithi149 ceph-mon[36681]: pgmap v74: 97 pgs: 97 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 KiB/s wr, 32 op/s 2024-03-26T06:25:22.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:22 smithi037 ceph-mon[31974]: pgmap v74: 97 pgs: 97 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 KiB/s wr, 32 op/s 2024-03-26T06:25:22.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:22 smithi066 ceph-mon[36504]: pgmap v74: 97 pgs: 97 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 KiB/s wr, 32 op/s 2024-03-26T06:25:23.487 INFO:tasks.workunit.client.0.smithi037.stdout: 1 50411 4.98 MB/sec warmup 58 sec latency 1850.667 ms 2024-03-26T06:25:24.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 50411 4.90 MB/sec warmup 59 sec latency 2850.771 ms 2024-03-26T06:25:24.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:24 smithi149 ceph-mon[36681]: pgmap v75: 97 pgs: 97 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-03-26T06:25:24.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:24 smithi037 ceph-mon[31974]: pgmap v75: 97 pgs: 97 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-03-26T06:25:24.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:24 smithi066 ceph-mon[36504]: pgmap v75: 97 pgs: 97 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-03-26T06:25:25.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 50479 4.82 MB/sec warmup 60 sec latency 3532.063 ms 2024-03-26T06:25:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.893+0000 7fe33359e640 1 -- 172.21.15.37:0/522830093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334106410 msgr2=0x7fe33410dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.893+0000 7fe33359e640 1 --2- 172.21.15.37:0/522830093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334106410 0x7fe33410dfe0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009ca0 tx=0x7fe32c02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.893+0000 7fe33359e640 1 -- 172.21.15.37:0/522830093 shutdown_connections 2024-03-26T06:25:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.893+0000 7fe33359e640 1 --2- 172.21.15.37:0/522830093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334106410 0x7fe33410dfe0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.893+0000 7fe33359e640 1 --2- 172.21.15.37:0/522830093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe334101cf0 0x7fe334105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.893+0000 7fe33359e640 1 --2- 172.21.15.37:0/522830093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe334101340 0x7fe334101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.893+0000 7fe33359e640 1 -- 172.21.15.37:0/522830093 >> 172.21.15.37:0/522830093 conn(0x7fe3340fcc00 msgr2=0x7fe3340ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.893+0000 7fe33359e640 1 -- 172.21.15.37:0/522830093 shutdown_connections 2024-03-26T06:25:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.893+0000 7fe33359e640 1 -- 172.21.15.37:0/522830093 wait complete. 2024-03-26T06:25:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.894+0000 7fe33359e640 1 Processor -- start 2024-03-26T06:25:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.894+0000 7fe33359e640 1 -- start start 2024-03-26T06:25:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe33359e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe334101340 0x7fe3340735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe33359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334106410 0x7fe334073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe33359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe334074070 0x7fe3341a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe33359e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341114d0 con 0x7fe334106410 2024-03-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe33359e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe334111350 con 0x7fe334074070 2024-03-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe33359e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe334111650 con 0x7fe334101340 2024-03-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe329d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334106410 0x7fe334073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe329d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334106410 0x7fe334073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54926/0 (socket says 172.21.15.37:54926) 2024-03-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe329d9b640 1 -- 172.21.15.37:0/255376329 learned_addr learned my addr 172.21.15.37:0/255376329 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe33259c640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe334101340 0x7fe3340735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe332d9d640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe334074070 0x7fe3341a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe329d9b640 1 -- 172.21.15.37:0/255376329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe334101340 msgr2=0x7fe3340735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe329d9b640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe334101340 0x7fe3340735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe329d9b640 1 -- 172.21.15.37:0/255376329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe334074070 msgr2=0x7fe3341a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe329d9b640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe334074070 0x7fe3341a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe329d9b640 1 -- 172.21.15.37:0/255376329 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3341a6280 con 0x7fe334106410 2024-03-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.895+0000 7fe329d9b640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334106410 0x7fe334073b30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d770 tx=0x7fe31c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.896+0000 7fe32bfff640 1 -- 172.21.15.37:0/255376329 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c017070 con 0x7fe334106410 2024-03-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.896+0000 7fe33359e640 1 -- 172.21.15.37:0/255376329 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3341a6570 con 0x7fe334106410 2024-03-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.896+0000 7fe32bfff640 1 -- 172.21.15.37:0/255376329 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe31c009c30 con 0x7fe334106410 2024-03-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.896+0000 7fe32bfff640 1 -- 172.21.15.37:0/255376329 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c00f620 con 0x7fe334106410 2024-03-26T06:25:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.896+0000 7fe33359e640 1 -- 172.21.15.37:0/255376329 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3341a6b10 con 0x7fe334106410 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.898+0000 7fe32bfff640 1 -- 172.21.15.37:0/255376329 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe31c01a070 con 0x7fe334106410 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.898+0000 7fe33b3e6740 1 Processor -- start 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.898+0000 7fe33b3e6740 1 -- start start 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921aef20 0x5633921adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921ae1b0 0x5633921ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633921125f0 con 0x5633921b7730 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633920bb930 con 0x5633921aef20 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339210e690 con 0x5633921ae1b0 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe32bfff640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3200a2960 0x7fe3200a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe329d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe332d9d640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3200a2960 0x7fe3200a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe329d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54942/0 (socket says 172.21.15.37:54942) 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe329d9b640 1 -- 172.21.15.37:0/155510126 learned_addr learned my addr 172.21.15.37:0/155510126 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe332d9d640 1 --2- 172.21.15.37:0/155510126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921aef20 0x5633921adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe33259c640 1 --2- 172.21.15.37:0/155510126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921ae1b0 0x5633921ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe32bfff640 1 -- 172.21.15.37:0/255376329 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fe31c0ce810 con 0x7fe334106410 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe329d9b640 1 -- 172.21.15.37:0/155510126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921ae1b0 msgr2=0x5633921ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe329d9b640 1 --2- 172.21.15.37:0/155510126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921ae1b0 0x5633921ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe329d9b640 1 -- 172.21.15.37:0/155510126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921aef20 msgr2=0x5633921adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.899+0000 7fe329d9b640 1 --2- 172.21.15.37:0/155510126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921aef20 0x5633921adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.900+0000 7fe329d9b640 1 -- 172.21.15.37:0/155510126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563391e27210 con 0x5633921b7730 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.900+0000 7fe332d9d640 1 --2- 172.21.15.37:0/155510126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921aef20 0x5633921adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.900+0000 7fe332d9d640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3200a2960 0x7fe3200a4e20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe32c004af0 tx=0x7fe32c032340 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.900+0000 7fe329d9b640 1 --2- 172.21.15.37:0/155510126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b9b20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009610 tx=0x7fe31c096bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bea14e91a5f4f59 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.900+0000 7fe328d99640 1 -- 172.21.15.37:0/155510126 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c09af50 con 0x5633921b7730 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.900+0000 7fe328d99640 1 -- 172.21.15.37:0/155510126 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe31c09eff0 con 0x5633921b7730 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.900+0000 7fe328d99640 1 -- 172.21.15.37:0/155510126 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c09a690 con 0x5633921b7730 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 -- 172.21.15.37:0/155510126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 msgr2=0x5633921b9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/155510126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b9b20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009610 tx=0x7fe31c096bd0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 -- 172.21.15.37:0/155510126 shutdown_connections 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/155510126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b9b20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/155510126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921ae1b0 0x5633921ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/155510126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921aef20 0x5633921adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 -- 172.21.15.37:0/155510126 >> 172.21.15.37:0/155510126 conn(0x5633921755b0 msgr2=0x5633921a8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 -- 172.21.15.37:0/155510126 shutdown_connections 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 -- 172.21.15.37:0/155510126 wait complete. 2024-03-26T06:25:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 -- start start 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 Processor -- start 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921ae1b0 0x5633921b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921aef20 0x5633921b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633921ba460 con 0x5633921b7730 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633920ced90 con 0x5633921ae1b0 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633921ba5e0 con 0x5633921aef20 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.901+0000 7fe33b3e6740 10 client.0 _ll_register_callbacks cb 0x563391e26a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe332d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe332d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54946/0 (socket says 172.21.15.37:54946) 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe332d9d640 1 -- 172.21.15.37:0/2277132216 learned_addr learned my addr 172.21.15.37:0/2277132216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe33259c640 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921ae1b0 0x5633921b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe329d9b640 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921aef20 0x5633921b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe332d9d640 1 -- 172.21.15.37:0/2277132216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921aef20 msgr2=0x5633921b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe332d9d640 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921aef20 0x5633921b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe332d9d640 1 -- 172.21.15.37:0/2277132216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921ae1b0 msgr2=0x5633921b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe332d9d640 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921ae1b0 0x5633921b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe332d9d640 1 -- 172.21.15.37:0/2277132216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563391ef3620 con 0x5633921b7730 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe332d9d640 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b5220 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe32c036000 tx=0x7fe32c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56702f66e4b0c809 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe319ffb640 1 -- 172.21.15.37:0/2277132216 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c051060 con 0x5633921b7730 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe319ffb640 1 -- 172.21.15.37:0/2277132216 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fe32c004240 con 0x5633921b7730 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe33b3e6740 1 -- 172.21.15.37:0/2277132216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe31c09eff0 con 0x5633921b7730 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe319ffb640 1 -- 172.21.15.37:0/2277132216 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe32c039040 con 0x5633921b7730 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe33b3e6740 10 client.15687 fetch_fsmap learned FSMap version 31 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.902+0000 7fe33b3e6740 1 -- 172.21.15.37:0/2277132216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5633921c6790 con 0x5633921b7730 2024-03-26T06:25:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.903+0000 7fe319ffb640 1 -- 172.21.15.37:0/2277132216 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe31c09eff0 con 0x5633921b7730 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.903+0000 7fe319ffb640 1 -- 172.21.15.37:0/2277132216 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe32c03f040 con 0x5633921b7730 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.903+0000 7fe33b3e6740 10 client.15687 fetch_fsmap finished waiting for FSMap version 31 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.903+0000 7fe33b3e6740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.903+0000 7fe33b3e6740 20 client.15687 populate_metadata read hostname 'smithi037' 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.903+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5633921c7cf0 0x5633921ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.903+0000 7fe33b3e6740 4 client.15687 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.903+0000 7fe33b3e6740 1 -- 172.21.15.37:0/2277132216 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56339211fb30 con 0x5633921c7cf0 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.903+0000 7fe33259c640 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5633921c7cf0 0x5633921ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe33259c640 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5633921c7cf0 0x5633921ca140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=314e09a6af7fb380 server_cookie=9bc2ce5335e6c9f4 in_seq=0 out_seq=0 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe319ffb640 10 client.15687 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe319ffb640 1 -- 172.21.15.37:0/2277132216 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56339211fb30 con 0x5633921c7cf0 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe319ffb640 10 client.15687 handle_command_reply: tid=0 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe33b3e6740 1 client.15687 shutdown 2024-03-26T06:25:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe33b3e6740 1 -- 172.21.15.37:0/2277132216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 msgr2=0x5633921b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b5220 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe32c036000 tx=0x7fe32c005c50 comp rx=0 tx=0).stop 2024-03-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe33b3e6740 1 -- 172.21.15.37:0/2277132216 shutdown_connections 2024-03-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5633921c7cf0 0x5633921ca140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b5220 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921aef20 0x5633921b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe319ffb640 0 client.15687 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/2277132216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921ae1b0 0x5633921b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.904+0000 7fe33b3e6740 1 -- 172.21.15.37:0/2277132216 >> 172.21.15.37:0/2277132216 conn(0x5633921755b0 msgr2=0x5633921aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.905+0000 7fe33b3e6740 1 -- 172.21.15.37:0/2277132216 shutdown_connections 2024-03-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.905+0000 7fe33b3e6740 1 -- 172.21.15.37:0/2277132216 wait complete. 2024-03-26T06:25:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.905+0000 7fe33b3e6740 20 client.15687 trim_cache size 0 max 16384 2024-03-26T06:25:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.918+0000 7fe33b3e6740 1 Processor -- start 2024-03-26T06:25:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.918+0000 7fe33b3e6740 1 -- start start 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.918+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921ae1b0 0x5633921b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.918+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921aef20 0x5633921be850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.918+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921b7730 0x5633921bed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.918+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341aede0 con 0x5633921ae1b0 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.918+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341b0880 con 0x5633921b7730 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.918+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341af440 con 0x5633921aef20 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.918+0000 7fe33259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921ae1b0 0x5633921b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.918+0000 7fe33259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921ae1b0 0x5633921b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54956/0 (socket says 172.21.15.37:54956) 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.918+0000 7fe33259c640 1 -- 172.21.15.37:0/3616590391 learned_addr learned my addr 172.21.15.37:0/3616590391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe332d9d640 1 --2- 172.21.15.37:0/3616590391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921b7730 0x5633921bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe329d9b640 1 --2- 172.21.15.37:0/3616590391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921aef20 0x5633921be850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe33259c640 1 -- 172.21.15.37:0/3616590391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921aef20 msgr2=0x5633921be850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe33259c640 1 --2- 172.21.15.37:0/3616590391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921aef20 0x5633921be850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe33259c640 1 -- 172.21.15.37:0/3616590391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921b7730 msgr2=0x5633921bed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe33259c640 1 --2- 172.21.15.37:0/3616590391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921b7730 0x5633921bed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe33259c640 1 -- 172.21.15.37:0/3616590391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633921b2d10 con 0x5633921ae1b0 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe332d9d640 1 --2- 172.21.15.37:0/3616590391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921b7730 0x5633921bed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe33259c640 1 --2- 172.21.15.37:0/3616590391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921ae1b0 0x5633921b6ed0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe324005fd0 tx=0x7fe32400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8e3e645f1261a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe31bfff640 1 -- 172.21.15.37:0/3616590391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324005020 con 0x5633921ae1b0 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe31bfff640 1 -- 172.21.15.37:0/3616590391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3240047a0 con 0x5633921ae1b0 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.919+0000 7fe31bfff640 1 -- 172.21.15.37:0/3616590391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32401f350 con 0x5633921ae1b0 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 -- 172.21.15.37:0/3616590391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921ae1b0 msgr2=0x5633921b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/3616590391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921ae1b0 0x5633921b6ed0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe324005fd0 tx=0x7fe32400b390 comp rx=0 tx=0).stop 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 -- 172.21.15.37:0/3616590391 shutdown_connections 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/3616590391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921b7730 0x5633921bed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/3616590391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921aef20 0x5633921be850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/3616590391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921ae1b0 0x5633921b6ed0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 -- 172.21.15.37:0/3616590391 >> 172.21.15.37:0/3616590391 conn(0x5633921755b0 msgr2=0x5633921afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 -- 172.21.15.37:0/3616590391 shutdown_connections 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 -- 172.21.15.37:0/3616590391 wait complete. 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 -- start start 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 Processor -- start 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921ae1b0 0x5633921bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921aef20 0x5633921b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341b0160 con 0x5633921b7730 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341afb00 con 0x5633921aef20 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341ad080 con 0x5633921ae1b0 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33b3e6740 10 client.0 _ll_register_callbacks cb 0x7fe31c00de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe332d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe332d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54964/0 (socket says 172.21.15.37:54964) 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe332d9d640 1 -- 172.21.15.37:0/1179679440 learned_addr learned my addr 172.21.15.37:0/1179679440 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.920+0000 7fe33259c640 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921ae1b0 0x5633921bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe329d9b640 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921aef20 0x5633921b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe332d9d640 1 -- 172.21.15.37:0/1179679440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921ae1b0 msgr2=0x5633921bed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe332d9d640 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921ae1b0 0x5633921bed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe332d9d640 1 -- 172.21.15.37:0/1179679440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921aef20 msgr2=0x5633921b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe332d9d640 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921aef20 0x5633921b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe332d9d640 1 -- 172.21.15.37:0/1179679440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56339221e220 con 0x5633921b7730 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe332d9d640 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b1a50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe32c02c630 tx=0x7fe32c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e94c67a78937686 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe319ffb640 1 -- 172.21.15.37:0/1179679440 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c051040 con 0x5633921b7730 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe319ffb640 1 -- 172.21.15.37:0/1179679440 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fe32c0041b0 con 0x5633921b7730 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe33b3e6740 1 -- 172.21.15.37:0/1179679440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3240047a0 con 0x5633921b7730 2024-03-26T06:25:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe319ffb640 1 -- 172.21.15.37:0/1179679440 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe32c066050 con 0x5633921b7730 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe33b3e6740 10 client.15693 fetch_fsmap learned FSMap version 31 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe33b3e6740 1 -- 172.21.15.37:0/1179679440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56339221f3d0 con 0x5633921b7730 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe319ffb640 1 -- 172.21.15.37:0/1179679440 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3240047a0 con 0x5633921b7730 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.921+0000 7fe319ffb640 1 -- 172.21.15.37:0/1179679440 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe32c05e020 con 0x5633921b7730 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.922+0000 7fe33b3e6740 10 client.15693 fetch_fsmap finished waiting for FSMap version 31 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.922+0000 7fe33b3e6740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.922+0000 7fe33b3e6740 20 client.15693 populate_metadata read hostname 'smithi037' 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.922+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56339221faf0 0x563392221f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.922+0000 7fe33b3e6740 4 client.15693 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.922+0000 7fe33b3e6740 1 -- 172.21.15.37:0/1179679440 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563392119e80 con 0x56339221faf0 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.922+0000 7fe33259c640 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56339221faf0 0x563392221f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.922+0000 7fe319ffb640 10 client.15693 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.922+0000 7fe33259c640 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56339221faf0 0x563392221f10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eea8e6bf6afd72d server_cookie=a5f9599e9322b0 in_seq=0 out_seq=0 2024-03-26T06:25:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.945+0000 7fe319ffb640 1 -- 172.21.15.37:0/1179679440 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563392119e80 con 0x56339221faf0 2024-03-26T06:25:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.945+0000 7fe319ffb640 10 client.15693 handle_command_reply: tid=0 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.945+0000 7fe33b3e6740 1 client.15693 shutdown 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.945+0000 7fe33b3e6740 1 -- 172.21.15.37:0/1179679440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 msgr2=0x5633921b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.945+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b1a50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe32c02c630 tx=0x7fe32c005520 comp rx=0 tx=0).stop 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33b3e6740 1 -- 172.21.15.37:0/1179679440 shutdown_connections 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56339221faf0 0x563392221f10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633921b7730 0x5633921b1a50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633921aef20 0x5633921b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33b3e6740 1 --2- 172.21.15.37:0/1179679440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633921ae1b0 0x5633921bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe319ffb640 0 client.15693 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33b3e6740 1 -- 172.21.15.37:0/1179679440 >> 172.21.15.37:0/1179679440 conn(0x5633921755b0 msgr2=0x5633921a89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33b3e6740 1 -- 172.21.15.37:0/1179679440 shutdown_connections 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33b3e6740 1 -- 172.21.15.37:0/1179679440 wait complete. 2024-03-26T06:25:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33b3e6740 20 client.15693 trim_cache size 0 max 16384 2024-03-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:25:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33359e640 1 -- 172.21.15.37:0/255376329 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3200a2960 msgr2=0x7fe3200a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33359e640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3200a2960 0x7fe3200a4e20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe32c004af0 tx=0x7fe32c032340 comp rx=0 tx=0).stop 2024-03-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33359e640 1 -- 172.21.15.37:0/255376329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334106410 msgr2=0x7fe334073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33359e640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334106410 0x7fe334073b30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d770 tx=0x7fe31c00dc30 comp rx=0 tx=0).stop 2024-03-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33359e640 1 -- 172.21.15.37:0/255376329 shutdown_connections 2024-03-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33359e640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3200a2960 0x7fe3200a4e20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33359e640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe334074070 0x7fe3341a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33359e640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334106410 0x7fe334073b30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33359e640 1 --2- 172.21.15.37:0/255376329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe334101340 0x7fe3340735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.946+0000 7fe33359e640 1 -- 172.21.15.37:0/255376329 >> 172.21.15.37:0/255376329 conn(0x7fe3340fcc00 msgr2=0x7fe3340fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.947+0000 7fe33359e640 1 -- 172.21.15.37:0/255376329 shutdown_connections 2024-03-26T06:25:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:25.947+0000 7fe33359e640 1 -- 172.21.15.37:0/255376329 wait complete. 2024-03-26T06:25:26.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 50479 4.74 MB/sec warmup 61 sec latency 1270.207 ms 2024-03-26T06:25:26.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:26 smithi149 ceph-mon[36681]: pgmap v76: 97 pgs: 97 active+clean; 28 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 119 KiB/s wr, 48 op/s 2024-03-26T06:25:26.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:26 smithi037 ceph-mon[31974]: pgmap v76: 97 pgs: 97 active+clean; 28 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 119 KiB/s wr, 48 op/s 2024-03-26T06:25:26.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:26 smithi066 ceph-mon[36504]: pgmap v76: 97 pgs: 97 active+clean; 28 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 119 KiB/s wr, 48 op/s 2024-03-26T06:25:27.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 50479 4.66 MB/sec warmup 62 sec latency 2270.303 ms 2024-03-26T06:25:28.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 50479 4.59 MB/sec warmup 63 sec latency 3270.372 ms 2024-03-26T06:25:28.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:28 smithi149 ceph-mon[36681]: pgmap v77: 97 pgs: 97 active+clean; 28 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 101 KiB/s wr, 33 op/s 2024-03-26T06:25:28.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:28 smithi037 ceph-mon[31974]: pgmap v77: 97 pgs: 97 active+clean; 28 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 101 KiB/s wr, 33 op/s 2024-03-26T06:25:28.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:28 smithi066 ceph-mon[36504]: pgmap v77: 97 pgs: 97 active+clean; 28 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 101 KiB/s wr, 33 op/s 2024-03-26T06:25:29.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 50479 4.52 MB/sec warmup 64 sec latency 4270.476 ms 2024-03-26T06:25:30.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 51116 4.52 MB/sec warmup 65 sec latency 4952.389 ms 2024-03-26T06:25:30.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:30 smithi149 ceph-mon[36681]: pgmap v78: 97 pgs: 97 active+clean; 28 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 101 KiB/s wr, 33 op/s 2024-03-26T06:25:30.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:30 smithi037 ceph-mon[31974]: pgmap v78: 97 pgs: 97 active+clean; 28 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 101 KiB/s wr, 33 op/s 2024-03-26T06:25:30.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:30 smithi066 ceph-mon[36504]: pgmap v78: 97 pgs: 97 active+clean; 28 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 101 KiB/s wr, 33 op/s 2024-03-26T06:25:31.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 53463 4.66 MB/sec warmup 66 sec latency 10.381 ms 2024-03-26T06:25:32.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 53903 4.60 MB/sec warmup 67 sec latency 485.166 ms 2024-03-26T06:25:32.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:32 smithi149 ceph-mon[36681]: pgmap v79: 97 pgs: 97 active+clean; 24 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 131 KiB/s wr, 47 op/s 2024-03-26T06:25:32.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:32 smithi037 ceph-mon[31974]: pgmap v79: 97 pgs: 97 active+clean; 24 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 131 KiB/s wr, 47 op/s 2024-03-26T06:25:32.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:32 smithi066 ceph-mon[36504]: pgmap v79: 97 pgs: 97 active+clean; 24 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 131 KiB/s wr, 47 op/s 2024-03-26T06:25:33.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 53903 4.53 MB/sec warmup 68 sec latency 1485.280 ms 2024-03-26T06:25:34.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 53903 4.46 MB/sec warmup 69 sec latency 2485.382 ms 2024-03-26T06:25:34.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:34 smithi149 ceph-mon[36681]: pgmap v80: 97 pgs: 97 active+clean; 24 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 115 KiB/s wr, 38 op/s 2024-03-26T06:25:34.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:34 smithi037 ceph-mon[31974]: pgmap v80: 97 pgs: 97 active+clean; 24 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 115 KiB/s wr, 38 op/s 2024-03-26T06:25:34.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:34 smithi066 ceph-mon[36504]: pgmap v80: 97 pgs: 97 active+clean; 24 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 115 KiB/s wr, 38 op/s 2024-03-26T06:25:35.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 54421 4.43 MB/sec warmup 70 sec latency 3168.190 ms 2024-03-26T06:25:36.488 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57036 4.60 MB/sec warmup 71 sec latency 11.681 ms 2024-03-26T06:25:36.660 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:36 smithi149 ceph-mon[36681]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 481 KiB/s wr, 90 op/s 2024-03-26T06:25:36.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:36 smithi037 ceph-mon[31974]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 481 KiB/s wr, 90 op/s 2024-03-26T06:25:36.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:36 smithi066 ceph-mon[36504]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 481 KiB/s wr, 90 op/s 2024-03-26T06:25:37.489 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57447 4.54 MB/sec warmup 72 sec latency 572.205 ms 2024-03-26T06:25:38.489 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57447 4.48 MB/sec warmup 73 sec latency 1572.296 ms 2024-03-26T06:25:38.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:38 smithi037 ceph-mon[31974]: pgmap v82: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 396 KiB/s wr, 66 op/s 2024-03-26T06:25:38.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:38 smithi066 ceph-mon[36504]: pgmap v82: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 396 KiB/s wr, 66 op/s 2024-03-26T06:25:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:38 smithi149 ceph-mon[36681]: pgmap v82: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 396 KiB/s wr, 66 op/s 2024-03-26T06:25:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.045+0000 7f29caafc640 1 -- 172.21.15.37:0/3048122690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c4101ec0 msgr2=0x7f29c4105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.045+0000 7f29caafc640 1 --2- 172.21.15.37:0/3048122690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c4101ec0 0x7f29c4105f00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009900 tx=0x7f29b402bc30 comp rx=0 tx=0).stop 2024-03-26T06:25:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.045+0000 7f29caafc640 1 -- 172.21.15.37:0/3048122690 shutdown_connections 2024-03-26T06:25:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.045+0000 7f29caafc640 1 --2- 172.21.15.37:0/3048122690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29c41065e0 0x7f29c410e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.045+0000 7f29caafc640 1 --2- 172.21.15.37:0/3048122690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c4101ec0 0x7f29c4105f00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.045+0000 7f29caafc640 1 --2- 172.21.15.37:0/3048122690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c4101510 0x7f29c41018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.045+0000 7f29caafc640 1 -- 172.21.15.37:0/3048122690 >> 172.21.15.37:0/3048122690 conn(0x7f29c40fcdb0 msgr2=0x7f29c40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.045+0000 7f29caafc640 1 -- 172.21.15.37:0/3048122690 shutdown_connections 2024-03-26T06:25:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.045+0000 7f29caafc640 1 -- 172.21.15.37:0/3048122690 wait complete. 2024-03-26T06:25:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29caafc640 1 Processor -- start 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29caafc640 1 -- start start 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29caafc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29c4101510 0x7f29c419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29caafc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c41065e0 0x7f29c419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29caafc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c4197930 0x7f29c4197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29caafc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c4111670 con 0x7f29c4197930 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29caafc640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41114f0 con 0x7f29c4101510 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29caafc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41117f0 con 0x7f29c41065e0 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29c9099640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c4197930 0x7f29c4197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29c9099640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c4197930 0x7f29c4197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36546/0 (socket says 172.21.15.37:36546) 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29c9099640 1 -- 172.21.15.37:0/1343077 learned_addr learned my addr 172.21.15.37:0/1343077 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29c8898640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29c4101510 0x7f29c419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29bbfff640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c41065e0 0x7f29c419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29c9099640 1 -- 172.21.15.37:0/1343077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c41065e0 msgr2=0x7f29c419dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29c9099640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c41065e0 0x7f29c419dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29c9099640 1 -- 172.21.15.37:0/1343077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29c4101510 msgr2=0x7f29c419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29c9099640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29c4101510 0x7f29c419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.046+0000 7f29c9099640 1 -- 172.21.15.37:0/1343077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c4198450 con 0x7f29c4197930 2024-03-26T06:25:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.047+0000 7f29c9099640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c4197930 0x7f29c4197de0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00eaa0 tx=0x7f29bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.047+0000 7f29b9ffb640 1 -- 172.21.15.37:0/1343077 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00cdb0 con 0x7f29c4197930 2024-03-26T06:25:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.047+0000 7f29b9ffb640 1 -- 172.21.15.37:0/1343077 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29bc005020 con 0x7f29c4197930 2024-03-26T06:25:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.047+0000 7f29b9ffb640 1 -- 172.21.15.37:0/1343077 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0116b0 con 0x7f29c4197930 2024-03-26T06:25:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.047+0000 7f29caafc640 1 -- 172.21.15.37:0/1343077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c4198740 con 0x7f29c4197930 2024-03-26T06:25:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.047+0000 7f29caafc640 1 -- 172.21.15.37:0/1343077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c41a48f0 con 0x7f29c4197930 2024-03-26T06:25:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.050+0000 7f29b9ffb640 1 -- 172.21.15.37:0/1343077 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f29bc005560 con 0x7f29c4197930 2024-03-26T06:25:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.050+0000 7f29cd7b1740 1 Processor -- start 2024-03-26T06:25:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.050+0000 7f29cd7b1740 1 -- start start 2024-03-26T06:25:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.050+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d2d40 0x56310e8d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.050+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.050+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8db550 0x56310e8dd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.050+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310e8365f0 con 0x56310e8d1fd0 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.050+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310e7df930 con 0x56310e8d2d40 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.050+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310e832690 con 0x56310e8db550 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29b9ffb640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29ac0a2a10 0x7f29ac0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29c8898640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29c8898640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8d2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36552/0 (socket says 172.21.15.37:36552) 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29c8898640 1 -- 172.21.15.37:0/2055698241 learned_addr learned my addr 172.21.15.37:0/2055698241 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29c9099640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29ac0a2a10 0x7f29ac0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29bbfff640 1 --2- 172.21.15.37:0/2055698241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d2d40 0x56310e8d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29bbfff640 1 --2- 172.21.15.37:0/2055698241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8db550 0x56310e8dd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29b9ffb640 1 -- 172.21.15.37:0/1343077 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f29bc0cf050 con 0x7f29c4197930 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29c8898640 1 -- 172.21.15.37:0/2055698241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8db550 msgr2=0x56310e8dd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29c8898640 1 --2- 172.21.15.37:0/2055698241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8db550 0x56310e8dd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29c8898640 1 -- 172.21.15.37:0/2055698241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d2d40 msgr2=0x56310e8d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29c8898640 1 --2- 172.21.15.37:0/2055698241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d2d40 0x56310e8d1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29c8898640 1 -- 172.21.15.37:0/2055698241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56310e54b210 con 0x56310e8d1fd0 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29c8898640 1 --2- 172.21.15.37:0/2055698241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8d2450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f29c0004650 tx=0x7f29c000bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4cd342aa522010 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29c9099640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29ac0a2a10 0x7f29ac0a4ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0cc880 tx=0x7f29bc058040 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29aaffd640 1 -- 172.21.15.37:0/2055698241 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c0003bd0 con 0x56310e8d1fd0 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29aaffd640 1 -- 172.21.15.37:0/2055698241 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29c0015070 con 0x56310e8d1fd0 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.051+0000 7f29aaffd640 1 -- 172.21.15.37:0/2055698241 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c0004e60 con 0x56310e8d1fd0 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.052+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2055698241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 msgr2=0x56310e8d2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.052+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2055698241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8d2450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f29c0004650 tx=0x7f29c000bba0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.052+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2055698241 shutdown_connections 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.052+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2055698241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8db550 0x56310e8dd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.052+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2055698241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8d2450 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.052+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2055698241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d2d40 0x56310e8d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.052+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2055698241 >> 172.21.15.37:0/2055698241 conn(0x56310e8995b0 msgr2=0x56310e8cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.052+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2055698241 shutdown_connections 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.052+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2055698241 wait complete. 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29cd7b1740 1 -- start start 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29cd7b1740 1 Processor -- start 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d1fd0 0x56310e8d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8d2d40 0x56310e8d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8db550 0x56310e8d9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310e8de280 con 0x56310e8db550 2024-03-26T06:25:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310e7f2d90 con 0x56310e8d1fd0 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310e8de400 con 0x56310e8d2d40 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29cd7b1740 10 client.0 _ll_register_callbacks cb 0x56310e54aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8db550 0x56310e8d9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8db550 0x56310e8d9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36558/0 (socket says 172.21.15.37:36558) 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29bbfff640 1 -- 172.21.15.37:0/2609960195 learned_addr learned my addr 172.21.15.37:0/2609960195 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29c8898640 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8d2d40 0x56310e8d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29bbfff640 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d1fd0 0x56310e8d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29bbfff640 1 -- 172.21.15.37:0/2609960195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8d2d40 msgr2=0x56310e8d8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29bbfff640 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8d2d40 0x56310e8d8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.053+0000 7f29bbfff640 1 -- 172.21.15.37:0/2609960195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d1fd0 msgr2=0x56310e8d85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29bbfff640 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d1fd0 0x56310e8d85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29bbfff640 1 -- 172.21.15.37:0/2609960195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56310e617620 con 0x56310e8db550 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29bbfff640 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8db550 0x56310e8d9030 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009340 tx=0x7f29b40092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7ce5429b6d5548 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29a3fff640 1 -- 172.21.15.37:0/2609960195 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40038b0 con 0x56310e8db550 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29a3fff640 1 -- 172.21.15.37:0/2609960195 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f29b4003a50 con 0x56310e8db550 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2609960195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29c0015070 con 0x56310e8db550 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29a3fff640 1 -- 172.21.15.37:0/2609960195 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29b4053050 con 0x56310e8db550 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29cd7b1740 10 client.15705 fetch_fsmap learned FSMap version 31 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2609960195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56310e8ea590 con 0x56310e8db550 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29a3fff640 1 -- 172.21.15.37:0/2609960195 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29c0015070 con 0x56310e8db550 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.054+0000 7f29a3fff640 1 -- 172.21.15.37:0/2609960195 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f29b4041430 con 0x56310e8db550 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.055+0000 7f29cd7b1740 10 client.15705 fetch_fsmap finished waiting for FSMap version 31 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.055+0000 7f29cd7b1740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.055+0000 7f29cd7b1740 20 client.15705 populate_metadata read hostname 'smithi037' 2024-03-26T06:25:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.055+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56310e8eba50 0x56310e8ede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.055+0000 7f29cd7b1740 4 client.15705 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.055+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2609960195 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56310e843b30 con 0x56310e8eba50 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.055+0000 7f29c8898640 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56310e8eba50 0x56310e8ede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.055+0000 7f29c8898640 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56310e8eba50 0x56310e8ede40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e056982028e41180 server_cookie=100c3e7767818728 in_seq=0 out_seq=0 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.055+0000 7f29a3fff640 10 client.15705 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29a3fff640 1 -- 172.21.15.37:0/2609960195 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56310e843b30 con 0x56310e8eba50 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29a3fff640 10 client.15705 handle_command_reply: tid=0 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 1 client.15705 shutdown 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2609960195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8db550 msgr2=0x56310e8d9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8db550 0x56310e8d9030 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009340 tx=0x7f29b40092d0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2609960195 shutdown_connections 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56310e8eba50 0x56310e8ede40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8db550 0x56310e8d9030 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8d2d40 0x56310e8d8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29a3fff640 0 client.15705 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2609960195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d1fd0 0x56310e8d85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2609960195 >> 172.21.15.37:0/2609960195 conn(0x56310e8995b0 msgr2=0x56310e8cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2609960195 shutdown_connections 2024-03-26T06:25:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2609960195 wait complete. 2024-03-26T06:25:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.056+0000 7f29cd7b1740 20 client.15705 trim_cache size 0 max 16384 2024-03-26T06:25:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29cd7b1740 1 Processor -- start 2024-03-26T06:25:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29cd7b1740 1 -- start start 2024-03-26T06:25:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8dace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d2d40 0x56310e8e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8db550 0x56310e8e2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41acb50 con 0x56310e8d1fd0 2024-03-26T06:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41ae5f0 con 0x56310e8d2d40 2024-03-26T06:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41ad1b0 con 0x56310e8db550 2024-03-26T06:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8dace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36566/0 (socket says 172.21.15.37:36566) 2024-03-26T06:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29bbfff640 1 -- 172.21.15.37:0/3752788246 learned_addr learned my addr 172.21.15.37:0/3752788246 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.070+0000 7f29c8898640 1 --2- 172.21.15.37:0/3752788246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d2d40 0x56310e8e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.071+0000 7f29bbfff640 1 -- 172.21.15.37:0/3752788246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8db550 msgr2=0x56310e8e2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.071+0000 7f29bbfff640 1 --2- 172.21.15.37:0/3752788246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8db550 0x56310e8e2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.071+0000 7f29bbfff640 1 -- 172.21.15.37:0/3752788246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d2d40 msgr2=0x56310e8e2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.071+0000 7f29bbfff640 1 --2- 172.21.15.37:0/3752788246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d2d40 0x56310e8e2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.071+0000 7f29bbfff640 1 -- 172.21.15.37:0/3752788246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56310e8d12a0 con 0x56310e8d1fd0 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.071+0000 7f29bbfff640 1 --2- 172.21.15.37:0/3752788246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8dace0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f29b402cc90 tx=0x7f29b402ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697b3c5c8d847cae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.071+0000 7f29aa7fc640 1 -- 172.21.15.37:0/3752788246 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4050020 con 0x56310e8d1fd0 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.071+0000 7f29aa7fc640 1 -- 172.21.15.37:0/3752788246 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29b4045040 con 0x56310e8d1fd0 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.071+0000 7f29aa7fc640 1 -- 172.21.15.37:0/3752788246 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4003ef0 con 0x56310e8d1fd0 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.071+0000 7f29cd7b1740 1 -- 172.21.15.37:0/3752788246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 msgr2=0x56310e8dace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.071+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/3752788246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8dace0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f29b402cc90 tx=0x7f29b402ccc0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 -- 172.21.15.37:0/3752788246 shutdown_connections 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/3752788246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8db550 0x56310e8e2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/3752788246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8d2d40 0x56310e8e2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/3752788246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d1fd0 0x56310e8dace0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 -- 172.21.15.37:0/3752788246 >> 172.21.15.37:0/3752788246 conn(0x56310e8995b0 msgr2=0x56310e8cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 -- 172.21.15.37:0/3752788246 shutdown_connections 2024-03-26T06:25:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 -- 172.21.15.37:0/3752788246 wait complete. 2024-03-26T06:25:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 -- start start 2024-03-26T06:25:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 Processor -- start 2024-03-26T06:25:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8d1fd0 0x56310e8e2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d2d40 0x56310e8d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8db550 0x56310e8d5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41aded0 con 0x56310e8d2d40 2024-03-26T06:25:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41ad870 con 0x56310e8db550 2024-03-26T06:25:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41aadf0 con 0x56310e8d1fd0 2024-03-26T06:25:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.072+0000 7f29cd7b1740 10 client.0 _ll_register_callbacks cb 0x7f29c0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29c8898640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d2d40 0x56310e8d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29c8898640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d2d40 0x56310e8d4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36578/0 (socket says 172.21.15.37:36578) 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29c8898640 1 -- 172.21.15.37:0/2657195276 learned_addr learned my addr 172.21.15.37:0/2657195276 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29bbfff640 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8d1fd0 0x56310e8e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29bbfff640 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8db550 0x56310e8d5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29c8898640 1 -- 172.21.15.37:0/2657195276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8d1fd0 msgr2=0x56310e8e2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29c8898640 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8d1fd0 0x56310e8e2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29c8898640 1 -- 172.21.15.37:0/2657195276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8db550 msgr2=0x56310e8d5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29c8898640 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8db550 0x56310e8d5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29c8898640 1 -- 172.21.15.37:0/2657195276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56310e8d56d0 con 0x56310e8d2d40 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29c8898640 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d2d40 0x56310e8d4c50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006010 tx=0x7f29c0004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2de4c95558f802 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:25:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29a3fff640 1 -- 172.21.15.37:0/2657195276 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c0029130 con 0x56310e8d2d40 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29a3fff640 1 -- 172.21.15.37:0/2657195276 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f29c00292d0 con 0x56310e8d2d40 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2657195276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29b4045040 con 0x56310e8d2d40 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29a3fff640 1 -- 172.21.15.37:0/2657195276 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29c00294d0 con 0x56310e8d2d40 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29cd7b1740 10 client.15711 fetch_fsmap learned FSMap version 31 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.073+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2657195276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56310e942b70 con 0x56310e8d2d40 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.074+0000 7f29a3fff640 1 -- 172.21.15.37:0/2657195276 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29b4045040 con 0x56310e8d2d40 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.074+0000 7f29a3fff640 1 -- 172.21.15.37:0/2657195276 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f29c003bc00 con 0x56310e8d2d40 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.074+0000 7f29cd7b1740 10 client.15711 fetch_fsmap finished waiting for FSMap version 31 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.074+0000 7f29cd7b1740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.074+0000 7f29cd7b1740 20 client.15711 populate_metadata read hostname 'smithi037' 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.074+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56310e943070 0x56310e945490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.074+0000 7f29cd7b1740 4 client.15711 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.074+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2657195276 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56310e83de80 con 0x56310e943070 2024-03-26T06:25:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.074+0000 7f29bbfff640 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56310e943070 0x56310e945490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.074+0000 7f29a3fff640 10 client.15711 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.074+0000 7f29bbfff640 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56310e943070 0x56310e945490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2a714cd242fb07 server_cookie=8841c9ba8d4116b8 in_seq=0 out_seq=0 2024-03-26T06:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29a3fff640 1 -- 172.21.15.37:0/2657195276 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56310e83de80 con 0x56310e943070 2024-03-26T06:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29a3fff640 10 client.15711 handle_command_reply: tid=0 2024-03-26T06:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 1 client.15711 shutdown 2024-03-26T06:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2657195276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d2d40 msgr2=0x56310e8d4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d2d40 0x56310e8d4c50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006010 tx=0x7f29c0004680 comp rx=0 tx=0).stop 2024-03-26T06:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2657195276 shutdown_connections 2024-03-26T06:25:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56310e943070 0x56310e945490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56310e8db550 0x56310e8d5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56310e8d2d40 0x56310e8d4c50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29a3fff640 0 client.15711 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 1 --2- 172.21.15.37:0/2657195276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56310e8d1fd0 0x56310e8e2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2657195276 >> 172.21.15.37:0/2657195276 conn(0x56310e8995b0 msgr2=0x56310e8d6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2657195276 shutdown_connections 2024-03-26T06:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 1 -- 172.21.15.37:0/2657195276 wait complete. 2024-03-26T06:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.107+0000 7f29cd7b1740 20 client.15711 trim_cache size 0 max 16384 2024-03-26T06:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:25:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:25:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:25:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 -- 172.21.15.37:0/1343077 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29ac0a2a10 msgr2=0x7f29ac0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29ac0a2a10 0x7f29ac0a4ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0cc880 tx=0x7f29bc058040 comp rx=0 tx=0).stop 2024-03-26T06:25:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 -- 172.21.15.37:0/1343077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c4197930 msgr2=0x7f29c4197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c4197930 0x7f29c4197de0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00eaa0 tx=0x7f29bc00ef60 comp rx=0 tx=0).stop 2024-03-26T06:25:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 -- 172.21.15.37:0/1343077 shutdown_connections 2024-03-26T06:25:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f29ac0a2a10 0x7f29ac0a4ed0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c4197930 0x7f29c4197de0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c41065e0 0x7f29c419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 --2- 172.21.15.37:0/1343077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f29c4101510 0x7f29c419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 -- 172.21.15.37:0/1343077 >> 172.21.15.37:0/1343077 conn(0x7f29c40fcdb0 msgr2=0x7f29c40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 -- 172.21.15.37:0/1343077 shutdown_connections 2024-03-26T06:25:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:39.108+0000 7f29caafc640 1 -- 172.21.15.37:0/1343077 wait complete. 2024-03-26T06:25:39.489 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57447 4.42 MB/sec warmup 74 sec latency 2572.416 ms 2024-03-26T06:25:40.489 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57503 4.36 MB/sec warmup 75 sec latency 3254.926 ms 2024-03-26T06:25:40.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:40 smithi037 ceph-mon[31974]: pgmap v83: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 396 KiB/s wr, 66 op/s 2024-03-26T06:25:40.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:40 smithi066 ceph-mon[36504]: pgmap v83: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 396 KiB/s wr, 66 op/s 2024-03-26T06:25:40.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:40 smithi149 ceph-mon[36681]: pgmap v83: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 396 KiB/s wr, 66 op/s 2024-03-26T06:25:41.489 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57503 4.30 MB/sec warmup 76 sec latency 1279.935 ms 2024-03-26T06:25:42.489 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57503 4.25 MB/sec warmup 77 sec latency 2280.001 ms 2024-03-26T06:25:42.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:42 smithi037 ceph-mon[31974]: pgmap v84: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 686 KiB/s wr, 125 op/s 2024-03-26T06:25:42.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:42 smithi066 ceph-mon[36504]: pgmap v84: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 686 KiB/s wr, 125 op/s 2024-03-26T06:25:42.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:42 smithi149 ceph-mon[36681]: pgmap v84: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 686 KiB/s wr, 125 op/s 2024-03-26T06:25:43.489 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57503 4.19 MB/sec warmup 78 sec latency 3280.098 ms 2024-03-26T06:25:44.489 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57503 4.14 MB/sec warmup 79 sec latency 4280.216 ms 2024-03-26T06:25:44.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:44 smithi037 ceph-mon[31974]: pgmap v85: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 656 KiB/s wr, 110 op/s 2024-03-26T06:25:44.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:44 smithi066 ceph-mon[36504]: pgmap v85: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 656 KiB/s wr, 110 op/s 2024-03-26T06:25:44.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:44 smithi149 ceph-mon[36681]: pgmap v85: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 656 KiB/s wr, 110 op/s 2024-03-26T06:25:45.489 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57571 4.09 MB/sec warmup 80 sec latency 4963.822 ms 2024-03-26T06:25:46.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57571 4.04 MB/sec warmup 81 sec latency 1272.901 ms 2024-03-26T06:25:46.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:46 smithi037 ceph-mon[31974]: pgmap v86: 97 pgs: 97 active+clean; 30 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 739 KiB/s wr, 121 op/s 2024-03-26T06:25:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:46 smithi066 ceph-mon[36504]: pgmap v86: 97 pgs: 97 active+clean; 30 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 739 KiB/s wr, 121 op/s 2024-03-26T06:25:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:46 smithi149 ceph-mon[36681]: pgmap v86: 97 pgs: 97 active+clean; 30 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 739 KiB/s wr, 121 op/s 2024-03-26T06:25:47.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57571 3.99 MB/sec warmup 82 sec latency 2273.022 ms 2024-03-26T06:25:48.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57571 3.94 MB/sec warmup 83 sec latency 3273.158 ms 2024-03-26T06:25:48.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:48 smithi037 ceph-mon[31974]: pgmap v87: 97 pgs: 97 active+clean; 30 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 372 KiB/s wr, 68 op/s 2024-03-26T06:25:48.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:48 smithi066 ceph-mon[36504]: pgmap v87: 97 pgs: 97 active+clean; 30 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 372 KiB/s wr, 68 op/s 2024-03-26T06:25:48.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:48 smithi149 ceph-mon[36681]: pgmap v87: 97 pgs: 97 active+clean; 30 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 372 KiB/s wr, 68 op/s 2024-03-26T06:25:49.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57571 3.90 MB/sec warmup 84 sec latency 4273.278 ms 2024-03-26T06:25:50.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 58273 3.91 MB/sec warmup 85 sec latency 4953.265 ms 2024-03-26T06:25:50.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:50 smithi037 ceph-mon[31974]: pgmap v88: 97 pgs: 97 active+clean; 30 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 372 KiB/s wr, 68 op/s 2024-03-26T06:25:50.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:50 smithi066 ceph-mon[36504]: pgmap v88: 97 pgs: 97 active+clean; 30 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 372 KiB/s wr, 68 op/s 2024-03-26T06:25:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:50 smithi149 ceph-mon[36681]: pgmap v88: 97 pgs: 97 active+clean; 30 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 372 KiB/s wr, 68 op/s 2024-03-26T06:25:51.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 60716 4.02 MB/sec warmup 86 sec latency 11.795 ms 2024-03-26T06:25:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.200+0000 7f1efce4b640 1 -- 172.21.15.37:0/596841360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ef8105190 msgr2=0x7f1ef8105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.200+0000 7f1efce4b640 1 --2- 172.21.15.37:0/596841360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ef8105190 0x7f1ef8105570 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009900 tx=0x7f1ef002bc30 comp rx=0 tx=0).stop 2024-03-26T06:25:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.200+0000 7f1efce4b640 1 -- 172.21.15.37:0/596841360 shutdown_connections 2024-03-26T06:25:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.200+0000 7f1efce4b640 1 --2- 172.21.15.37:0/596841360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef8100e00 0x7f1ef810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.200+0000 7f1efce4b640 1 --2- 172.21.15.37:0/596841360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ef8100440 0x7f1ef81008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.200+0000 7f1efce4b640 1 --2- 172.21.15.37:0/596841360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ef8105190 0x7f1ef8105570 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.200+0000 7f1efce4b640 1 -- 172.21.15.37:0/596841360 >> 172.21.15.37:0/596841360 conn(0x7f1ef8077040 msgr2=0x7f1ef8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.200+0000 7f1efce4b640 1 -- 172.21.15.37:0/596841360 shutdown_connections 2024-03-26T06:25:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.200+0000 7f1efce4b640 1 -- 172.21.15.37:0/596841360 wait complete. 2024-03-26T06:25:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1efce4b640 1 Processor -- start 2024-03-26T06:25:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1efce4b640 1 -- start start 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1efce4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ef8100440 0x7f1ef819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1efce4b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef8100e00 0x7f1ef819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1efce4b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ef8105190 0x7f1ef8197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1efce4b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef8110650 con 0x7f1ef8100440 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1efce4b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81104d0 con 0x7f1ef8100e00 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1efce4b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81107d0 con 0x7f1ef8105190 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1ef659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ef8100440 0x7f1ef819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1ef659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ef8100440 0x7f1ef819d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37904/0 (socket says 172.21.15.37:37904) 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1ef659c640 1 -- 172.21.15.37:0/738938832 learned_addr learned my addr 172.21.15.37:0/738938832 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1ef5d9b640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef8100e00 0x7f1ef819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1ef6d9d640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ef8105190 0x7f1ef8197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1ef659c640 1 -- 172.21.15.37:0/738938832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ef8105190 msgr2=0x7f1ef8197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1ef659c640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ef8105190 0x7f1ef8197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1ef659c640 1 -- 172.21.15.37:0/738938832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef8100e00 msgr2=0x7f1ef819dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1ef659c640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef8100e00 0x7f1ef819dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.201+0000 7f1ef659c640 1 -- 172.21.15.37:0/738938832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef8198190 con 0x7f1ef8100440 2024-03-26T06:25:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.202+0000 7f1ef659c640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ef8100440 0x7f1ef819d860 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0005940 tx=0x7f1ef0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.202+0000 7f1eef7fe640 1 -- 172.21.15.37:0/738938832 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0041070 con 0x7f1ef8100440 2024-03-26T06:25:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.202+0000 7f1eef7fe640 1 -- 172.21.15.37:0/738938832 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ef00054e0 con 0x7f1ef8100440 2024-03-26T06:25:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.202+0000 7f1efce4b640 1 -- 172.21.15.37:0/738938832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef8198420 con 0x7f1ef8100440 2024-03-26T06:25:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.202+0000 7f1eef7fe640 1 -- 172.21.15.37:0/738938832 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0003d00 con 0x7f1ef8100440 2024-03-26T06:25:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.202+0000 7f1efce4b640 1 -- 172.21.15.37:0/738938832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef81a4720 con 0x7f1ef8100440 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.204+0000 7f1eef7fe640 1 -- 172.21.15.37:0/738938832 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1ef0044070 con 0x7f1ef8100440 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1effa2b740 1 Processor -- start 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1effa2b740 1 -- start start 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125ba2d40 0x557125ba1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba1fd0 0x557125ba2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125bab550 0x557125bad940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1eef7fe640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1ee40a2a10 0x7f1ee40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1effa2b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557125b065f0 con 0x557125ba1fd0 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1effa2b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557125aaf930 con 0x557125bab550 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1effa2b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557125b02690 con 0x557125ba2d40 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1ef659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba1fd0 0x557125ba2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1ef6d9d640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1ee40a2a10 0x7f1ee40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1ef5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125bab550 0x557125bad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1ef5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125bab550 0x557125bad940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:45806/0 (socket says 172.21.15.37:45806) 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/766320158 learned_addr learned my addr 172.21.15.37:0/766320158 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1eef7fe640 1 -- 172.21.15.37:0/738938832 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1ef00f0a20 con 0x7f1ef8100440 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1ef6d9d640 1 --2- 172.21.15.37:0/766320158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125ba2d40 0x557125ba1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/766320158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125ba2d40 msgr2=0x557125ba1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.205+0000 7f1ef5d9b640 1 --2- 172.21.15.37:0/766320158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125ba2d40 0x557125ba1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.206+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/766320158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba1fd0 msgr2=0x557125ba2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.206+0000 7f1ef5d9b640 1 --2- 172.21.15.37:0/766320158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba1fd0 0x557125ba2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.206+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/766320158 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55712581b210 con 0x557125bab550 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.206+0000 7f1ef6d9d640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1ee40a2a10 0x7f1ee40a4ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009fd0 tx=0x7f1ee8012b90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.206+0000 7f1ef5d9b640 1 --2- 172.21.15.37:0/766320158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125bab550 0x557125bad940 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009c60 tx=0x7f1ee0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5816e7bb2f62b37 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.206+0000 7f1eecff9640 1 -- 172.21.15.37:0/766320158 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0004230 con 0x557125bab550 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.206+0000 7f1eecff9640 1 -- 172.21.15.37:0/766320158 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ee00043d0 con 0x557125bab550 2024-03-26T06:25:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.206+0000 7f1eecff9640 1 -- 172.21.15.37:0/766320158 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0019840 con 0x557125bab550 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.207+0000 7f1effa2b740 1 -- 172.21.15.37:0/766320158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125bab550 msgr2=0x557125bad940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.207+0000 7f1effa2b740 1 --2- 172.21.15.37:0/766320158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125bab550 0x557125bad940 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009c60 tx=0x7f1ee0011810 comp rx=0 tx=0).stop 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.207+0000 7f1effa2b740 1 -- 172.21.15.37:0/766320158 shutdown_connections 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.207+0000 7f1effa2b740 1 --2- 172.21.15.37:0/766320158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125bab550 0x557125bad940 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.207+0000 7f1effa2b740 1 --2- 172.21.15.37:0/766320158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba1fd0 0x557125ba2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.207+0000 7f1effa2b740 1 --2- 172.21.15.37:0/766320158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125ba2d40 0x557125ba1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.207+0000 7f1effa2b740 1 -- 172.21.15.37:0/766320158 >> 172.21.15.37:0/766320158 conn(0x557125b695b0 msgr2=0x557125b9ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.207+0000 7f1effa2b740 1 -- 172.21.15.37:0/766320158 shutdown_connections 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.207+0000 7f1effa2b740 1 -- 172.21.15.37:0/766320158 wait complete. 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.207+0000 7f1effa2b740 1 -- start start 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.207+0000 7f1effa2b740 1 Processor -- start 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba1fd0 0x557125ba84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125bab550 0x557125ba8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1effa2b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557125bae280 con 0x557125ba2d40 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1effa2b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557125ac2d90 con 0x557125ba1fd0 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1effa2b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557125bae400 con 0x557125bab550 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1effa2b740 10 client.0 _ll_register_callbacks cb 0x55712581aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37930/0 (socket says 172.21.15.37:37930) 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba1fd0 0x557125ba84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/1623523181 learned_addr learned my addr 172.21.15.37:0/1623523181 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef6d9d640 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125bab550 0x557125ba8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/1623523181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125bab550 msgr2=0x557125ba8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef5d9b640 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125bab550 0x557125ba8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/1623523181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba1fd0 msgr2=0x557125ba84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef5d9b640 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba1fd0 0x557125ba84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/1623523181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571258e7620 con 0x557125ba2d40 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef659c640 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba1fd0 0x557125ba84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1ef5d9b640 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba8a20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009650 tx=0x7f1ee0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b705e0f469d2ed1a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.208+0000 7f1eddffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1eddffb640 1 -- 172.21.15.37:0/1623523181 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0028020 con 0x557125ba2d40 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1eddffb640 1 -- 172.21.15.37:0/1623523181 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1ee0005250 con 0x557125ba2d40 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1effa2b740 1 -- 172.21.15.37:0/1623523181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ee00043d0 con 0x557125ba2d40 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1eddffb640 1 -- 172.21.15.37:0/1623523181 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ee0033050 con 0x557125ba2d40 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1effa2b740 10 client.15723 fetch_fsmap learned FSMap version 31 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1effa2b740 1 -- 172.21.15.37:0/1623523181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557125bba4c0 con 0x557125ba2d40 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1eddffb640 1 -- 172.21.15.37:0/1623523181 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ee00043d0 con 0x557125ba2d40 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1eddffb640 1 -- 172.21.15.37:0/1623523181 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1ee0018070 con 0x557125ba2d40 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1effa2b740 10 client.15723 fetch_fsmap finished waiting for FSMap version 31 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1effa2b740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:25:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1effa2b740 20 client.15723 populate_metadata read hostname 'smithi037' 2024-03-26T06:25:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1effa2b740 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557125bbb8c0 0x557125bbdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1effa2b740 4 client.15723 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:25:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1effa2b740 1 -- 172.21.15.37:0/1623523181 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557125b13b30 con 0x557125bbb8c0 2024-03-26T06:25:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.209+0000 7f1ef659c640 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557125bbb8c0 0x557125bbdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.210+0000 7f1ef659c640 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557125bbb8c0 0x557125bbdcb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d18a2e1047aa4b9 server_cookie=1116876a77cef777 in_seq=0 out_seq=0 2024-03-26T06:25:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.210+0000 7f1eddffb640 10 client.15723 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1eddffb640 1 -- 172.21.15.37:0/1623523181 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557125b13b30 con 0x557125bbb8c0 2024-03-26T06:25:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1eddffb640 10 client.15723 handle_command_reply: tid=0 2024-03-26T06:25:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 1 client.15723 shutdown 2024-03-26T06:25:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 1 -- 172.21.15.37:0/1623523181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 msgr2=0x557125ba8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba8a20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009650 tx=0x7f1ee0009680 comp rx=0 tx=0).stop 2024-03-26T06:25:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 1 -- 172.21.15.37:0/1623523181 shutdown_connections 2024-03-26T06:25:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557125bbb8c0 0x557125bbdcb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125bab550 0x557125ba8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba8a20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1eddffb640 0 client.15723 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 1 --2- 172.21.15.37:0/1623523181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba1fd0 0x557125ba84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 1 -- 172.21.15.37:0/1623523181 >> 172.21.15.37:0/1623523181 conn(0x557125b695b0 msgr2=0x557125b9ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 1 -- 172.21.15.37:0/1623523181 shutdown_connections 2024-03-26T06:25:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 1 -- 172.21.15.37:0/1623523181 wait complete. 2024-03-26T06:25:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.211+0000 7f1effa2b740 20 client.15723 trim_cache size 0 max 16384 2024-03-26T06:25:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1effa2b740 1 Processor -- start 2024-03-26T06:25:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1effa2b740 1 -- start start 2024-03-26T06:25:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125ba1fd0 0x557125baaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba2d40 0x557125bb2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125bab550 0x557125bb2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1effa2b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81acba0 con 0x557125bab550 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1effa2b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81ae640 con 0x557125ba2d40 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1effa2b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81ad200 con 0x557125ba1fd0 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1ef6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125bab550 0x557125bb2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1ef6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125bab550 0x557125bb2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37940/0 (socket says 172.21.15.37:37940) 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1ef6d9d640 1 -- 172.21.15.37:0/2107482455 learned_addr learned my addr 172.21.15.37:0/2107482455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1ef5d9b640 1 --2- 172.21.15.37:0/2107482455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba2d40 0x557125bb2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1ef659c640 1 --2- 172.21.15.37:0/2107482455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125ba1fd0 0x557125baaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1ef6d9d640 1 -- 172.21.15.37:0/2107482455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125ba1fd0 msgr2=0x557125baaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1ef6d9d640 1 --2- 172.21.15.37:0/2107482455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125ba1fd0 0x557125baaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1ef6d9d640 1 -- 172.21.15.37:0/2107482455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba2d40 msgr2=0x557125bb2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1ef6d9d640 1 --2- 172.21.15.37:0/2107482455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba2d40 0x557125bb2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.225+0000 7f1ef6d9d640 1 -- 172.21.15.37:0/2107482455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557125ba3440 con 0x557125bab550 2024-03-26T06:25:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.226+0000 7f1ef6d9d640 1 --2- 172.21.15.37:0/2107482455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125bab550 0x557125bb2bb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8010d10 tx=0x7f1ee8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8ec59c0e6651e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.226+0000 7f1edffff640 1 -- 172.21.15.37:0/2107482455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8010e00 con 0x557125bab550 2024-03-26T06:25:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.226+0000 7f1edffff640 1 -- 172.21.15.37:0/2107482455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ee8003d40 con 0x557125bab550 2024-03-26T06:25:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.226+0000 7f1edffff640 1 -- 172.21.15.37:0/2107482455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee801c3a0 con 0x557125bab550 2024-03-26T06:25:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.226+0000 7f1effa2b740 1 -- 172.21.15.37:0/2107482455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125bab550 msgr2=0x557125bb2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.226+0000 7f1effa2b740 1 --2- 172.21.15.37:0/2107482455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125bab550 0x557125bb2bb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8010d10 tx=0x7f1ee8004ab0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.226+0000 7f1effa2b740 1 -- 172.21.15.37:0/2107482455 shutdown_connections 2024-03-26T06:25:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.226+0000 7f1effa2b740 1 --2- 172.21.15.37:0/2107482455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125bab550 0x557125bb2bb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.226+0000 7f1effa2b740 1 --2- 172.21.15.37:0/2107482455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba2d40 0x557125bb2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.226+0000 7f1effa2b740 1 --2- 172.21.15.37:0/2107482455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125ba1fd0 0x557125baaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.226+0000 7f1effa2b740 1 -- 172.21.15.37:0/2107482455 >> 172.21.15.37:0/2107482455 conn(0x557125b695b0 msgr2=0x557125b9c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1effa2b740 1 -- 172.21.15.37:0/2107482455 shutdown_connections 2024-03-26T06:25:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1effa2b740 1 -- 172.21.15.37:0/2107482455 wait complete. 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1effa2b740 1 -- start start 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1effa2b740 1 Processor -- start 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba1fd0 0x557125ba5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1effa2b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125bab550 0x557125ba5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1effa2b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81adf20 con 0x557125ba2d40 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1effa2b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81ad8c0 con 0x557125ba1fd0 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1effa2b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81aae60 con 0x557125bab550 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1effa2b740 10 client.0 _ll_register_callbacks cb 0x7f1ee0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1ef5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1ef5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37952/0 (socket says 172.21.15.37:37952) 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/2499324479 learned_addr learned my addr 172.21.15.37:0/2499324479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:25:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1ef6d9d640 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125bab550 0x557125ba5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1ef659c640 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba1fd0 0x557125ba5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/2499324479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125bab550 msgr2=0x557125ba5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1ef5d9b640 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125bab550 0x557125ba5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/2499324479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba1fd0 msgr2=0x557125ba5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1ef5d9b640 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba1fd0 0x557125ba5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.227+0000 7f1ef5d9b640 1 -- 172.21.15.37:0/2499324479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557125ba12a0 con 0x557125ba2d40 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1ef5d9b640 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba5740 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009620 tx=0x7f1ee00098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0cac6cebe66174 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1eddffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1eddffb640 1 -- 172.21.15.37:0/2499324479 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0028030 con 0x557125ba2d40 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1eddffb640 1 -- 172.21.15.37:0/2499324479 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1ee0004ac0 con 0x557125ba2d40 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1effa2b740 1 -- 172.21.15.37:0/2499324479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ee8003d40 con 0x557125ba2d40 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1eddffb640 1 -- 172.21.15.37:0/2499324479 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ee0004ce0 con 0x557125ba2d40 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1effa2b740 10 client.15729 fetch_fsmap learned FSMap version 31 2024-03-26T06:25:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1effa2b740 1 -- 172.21.15.37:0/2499324479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557125c12f30 con 0x557125ba2d40 2024-03-26T06:25:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1eddffb640 1 -- 172.21.15.37:0/2499324479 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ee8003d40 con 0x557125ba2d40 2024-03-26T06:25:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1eddffb640 1 -- 172.21.15.37:0/2499324479 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1ee0018040 con 0x557125ba2d40 2024-03-26T06:25:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1effa2b740 10 client.15729 fetch_fsmap finished waiting for FSMap version 31 2024-03-26T06:25:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1effa2b740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:25:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1effa2b740 20 client.15729 populate_metadata read hostname 'smithi037' 2024-03-26T06:25:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1effa2b740 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557125c13680 0x557125c15aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:25:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1effa2b740 4 client.15729 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:25:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1effa2b740 1 -- 172.21.15.37:0/2499324479 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557125b0de80 con 0x557125c13680 2024-03-26T06:25:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.228+0000 7f1ef659c640 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557125c13680 0x557125c15aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:25:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.229+0000 7f1eddffb640 10 client.15729 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.229+0000 7f1ef659c640 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557125c13680 0x557125c15aa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff71980cf7d23fa server_cookie=597b18895b24d6a1 in_seq=0 out_seq=0 2024-03-26T06:25:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1eddffb640 1 -- 172.21.15.37:0/2499324479 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557125b0de80 con 0x557125c13680 2024-03-26T06:25:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1eddffb640 10 client.15729 handle_command_reply: tid=0 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1effa2b740 1 client.15729 shutdown 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1effa2b740 1 -- 172.21.15.37:0/2499324479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 msgr2=0x557125ba5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1effa2b740 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba5740 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009620 tx=0x7f1ee00098a0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1effa2b740 1 -- 172.21.15.37:0/2499324479 shutdown_connections 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1effa2b740 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557125c13680 0x557125c15aa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1effa2b740 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557125bab550 0x557125ba5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1eddffb640 0 client.15729 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1effa2b740 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557125ba2d40 0x557125ba5740 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1effa2b740 1 --2- 172.21.15.37:0/2499324479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557125ba1fd0 0x557125ba5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1effa2b740 1 -- 172.21.15.37:0/2499324479 >> 172.21.15.37:0/2499324479 conn(0x557125b695b0 msgr2=0x557125ba6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1effa2b740 1 -- 172.21.15.37:0/2499324479 shutdown_connections 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.265+0000 7f1effa2b740 1 -- 172.21.15.37:0/2499324479 wait complete. 2024-03-26T06:25:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1effa2b740 20 client.15729 trim_cache size 0 max 16384 2024-03-26T06:25:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:25:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:25:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:25:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:25:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 -- 172.21.15.37:0/738938832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1ee40a2a10 msgr2=0x7f1ee40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1ee40a2a10 0x7f1ee40a4ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009fd0 tx=0x7f1ee8012b90 comp rx=0 tx=0).stop 2024-03-26T06:25:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 -- 172.21.15.37:0/738938832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ef8100440 msgr2=0x7f1ef819d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:25:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ef8100440 0x7f1ef819d860 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0005940 tx=0x7f1ef0004270 comp rx=0 tx=0).stop 2024-03-26T06:25:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 -- 172.21.15.37:0/738938832 shutdown_connections 2024-03-26T06:25:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1ee40a2a10 0x7f1ee40a4ed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ef8105190 0x7f1ef8197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ef8100e00 0x7f1ef819dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 --2- 172.21.15.37:0/738938832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ef8100440 0x7f1ef819d860 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:25:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 -- 172.21.15.37:0/738938832 >> 172.21.15.37:0/738938832 conn(0x7f1ef8077040 msgr2=0x7f1ef8068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:25:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 -- 172.21.15.37:0/738938832 shutdown_connections 2024-03-26T06:25:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:25:52.266+0000 7f1efce4b640 1 -- 172.21.15.37:0/738938832 wait complete. 2024-03-26T06:25:52.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 62891 4.07 MB/sec warmup 87 sec latency 12.946 ms 2024-03-26T06:25:52.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:52 smithi037 ceph-mon[31974]: pgmap v89: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 487 KiB/s wr, 78 op/s 2024-03-26T06:25:52.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:52 smithi066 ceph-mon[36504]: pgmap v89: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 487 KiB/s wr, 78 op/s 2024-03-26T06:25:52.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:52 smithi149 ceph-mon[36681]: pgmap v89: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 487 KiB/s wr, 78 op/s 2024-03-26T06:25:53.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 64536 4.15 MB/sec warmup 88 sec latency 14.891 ms 2024-03-26T06:25:54.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 64536 4.10 MB/sec warmup 89 sec latency 1003.190 ms 2024-03-26T06:25:54.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:54 smithi037 ceph-mon[31974]: pgmap v90: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 198 KiB/s wr, 19 op/s 2024-03-26T06:25:54.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:54 smithi066 ceph-mon[36504]: pgmap v90: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 198 KiB/s wr, 19 op/s 2024-03-26T06:25:54.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:54 smithi149 ceph-mon[36681]: pgmap v90: 97 pgs: 97 active+clean; 29 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 198 KiB/s wr, 19 op/s 2024-03-26T06:25:55.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 65077 4.08 MB/sec warmup 90 sec latency 1685.109 ms 2024-03-26T06:25:56.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 67658 4.22 MB/sec warmup 91 sec latency 13.059 ms 2024-03-26T06:25:56.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:56 smithi037 ceph-mon[31974]: pgmap v91: 97 pgs: 97 active+clean; 33 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 650 KiB/s wr, 106 op/s 2024-03-26T06:25:56.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:56 smithi066 ceph-mon[36504]: pgmap v91: 97 pgs: 97 active+clean; 33 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 650 KiB/s wr, 106 op/s 2024-03-26T06:25:56.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:56 smithi149 ceph-mon[36681]: pgmap v91: 97 pgs: 97 active+clean; 33 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 650 KiB/s wr, 106 op/s 2024-03-26T06:25:57.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 68211 4.18 MB/sec warmup 92 sec latency 453.076 ms 2024-03-26T06:25:58.490 INFO:tasks.workunit.client.0.smithi037.stdout: 1 68211 4.14 MB/sec warmup 93 sec latency 1453.137 ms 2024-03-26T06:25:58.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:25:58 smithi037 ceph-mon[31974]: pgmap v92: 97 pgs: 97 active+clean; 33 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 567 KiB/s wr, 95 op/s 2024-03-26T06:25:58.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:25:58 smithi149 ceph-mon[36681]: pgmap v92: 97 pgs: 97 active+clean; 33 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 567 KiB/s wr, 95 op/s 2024-03-26T06:25:58.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:25:58 smithi066 ceph-mon[36504]: pgmap v92: 97 pgs: 97 active+clean; 33 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 567 KiB/s wr, 95 op/s 2024-03-26T06:25:59.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 68211 4.09 MB/sec warmup 94 sec latency 2453.231 ms 2024-03-26T06:26:00.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 68846 4.10 MB/sec warmup 95 sec latency 3134.434 ms 2024-03-26T06:26:00.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:00 smithi037 ceph-mon[31974]: pgmap v93: 97 pgs: 97 active+clean; 33 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 567 KiB/s wr, 95 op/s 2024-03-26T06:26:00.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:00 smithi066 ceph-mon[36504]: pgmap v93: 97 pgs: 97 active+clean; 33 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 567 KiB/s wr, 95 op/s 2024-03-26T06:26:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:00 smithi149 ceph-mon[36681]: pgmap v93: 97 pgs: 97 active+clean; 33 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 567 KiB/s wr, 95 op/s 2024-03-26T06:26:01.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 70946 4.18 MB/sec warmup 96 sec latency 20.899 ms 2024-03-26T06:26:02.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 71753 4.16 MB/sec warmup 97 sec latency 269.079 ms 2024-03-26T06:26:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:02 smithi037 ceph-mon[31974]: pgmap v94: 97 pgs: 97 active+clean; 35 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 833 KiB/s wr, 146 op/s 2024-03-26T06:26:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:02 smithi066 ceph-mon[36504]: pgmap v94: 97 pgs: 97 active+clean; 35 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 833 KiB/s wr, 146 op/s 2024-03-26T06:26:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:02 smithi149 ceph-mon[36681]: pgmap v94: 97 pgs: 97 active+clean; 35 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 833 KiB/s wr, 146 op/s 2024-03-26T06:26:03.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 71753 4.12 MB/sec warmup 98 sec latency 1269.110 ms 2024-03-26T06:26:04.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 71753 4.08 MB/sec warmup 99 sec latency 2269.147 ms 2024-03-26T06:26:04.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:04 smithi037 ceph-mon[31974]: pgmap v95: 97 pgs: 97 active+clean; 35 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 718 KiB/s wr, 136 op/s 2024-03-26T06:26:04.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:04 smithi066 ceph-mon[36504]: pgmap v95: 97 pgs: 97 active+clean; 35 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 718 KiB/s wr, 136 op/s 2024-03-26T06:26:04.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:04 smithi149 ceph-mon[36681]: pgmap v95: 97 pgs: 97 active+clean; 35 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 718 KiB/s wr, 136 op/s 2024-03-26T06:26:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.370+0000 7f25be7ab640 1 -- 172.21.15.37:0/1658599296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25b8101ee0 msgr2=0x7f25b8105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.370+0000 7f25be7ab640 1 --2- 172.21.15.37:0/1658599296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25b8101ee0 0x7f25b8105f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009860 tx=0x7f25ac02bb70 comp rx=0 tx=0).stop 2024-03-26T06:26:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.370+0000 7f25be7ab640 1 -- 172.21.15.37:0/1658599296 shutdown_connections 2024-03-26T06:26:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.370+0000 7f25be7ab640 1 --2- 172.21.15.37:0/1658599296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b8106600 0x7f25b810e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.370+0000 7f25be7ab640 1 --2- 172.21.15.37:0/1658599296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25b8101ee0 0x7f25b8105f20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.370+0000 7f25be7ab640 1 --2- 172.21.15.37:0/1658599296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b8101530 0x7f25b8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.370+0000 7f25be7ab640 1 -- 172.21.15.37:0/1658599296 >> 172.21.15.37:0/1658599296 conn(0x7f25b80fcdd0 msgr2=0x7f25b80ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.370+0000 7f25be7ab640 1 -- 172.21.15.37:0/1658599296 shutdown_connections 2024-03-26T06:26:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.370+0000 7f25be7ab640 1 -- 172.21.15.37:0/1658599296 wait complete. 2024-03-26T06:26:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25be7ab640 1 Processor -- start 2024-03-26T06:26:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25be7ab640 1 -- start start 2024-03-26T06:26:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25be7ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25b8101530 0x7f25b80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25be7ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b8106600 0x7f25b8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25be7ab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b8074070 0x7f25b81a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25be7ab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8111670 con 0x7f25b8106600 2024-03-26T06:26:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25be7ab640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81114f0 con 0x7f25b8101530 2024-03-26T06:26:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25be7ab640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81117f0 con 0x7f25b8074070 2024-03-26T06:26:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b8106600 0x7f25b8073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25b8101530 0x7f25b80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25b8101530 0x7f25b80735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:46658/0 (socket says 172.21.15.37:46658) 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25b7fff640 1 -- 172.21.15.37:0/1799922573 learned_addr learned my addr 172.21.15.37:0/1799922573 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b8106600 0x7f25b8073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50324/0 (socket says 172.21.15.37:50324) 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.371+0000 7f25bcd48640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b8074070 0x7f25b81a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.372+0000 7f25b7fff640 1 -- 172.21.15.37:0/1799922573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b8074070 msgr2=0x7f25b81a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.372+0000 7f25b7fff640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b8074070 0x7f25b81a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.372+0000 7f25b7fff640 1 -- 172.21.15.37:0/1799922573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b8106600 msgr2=0x7f25b8073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.372+0000 7f25b7fff640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b8106600 0x7f25b8073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.372+0000 7f25b7fff640 1 -- 172.21.15.37:0/1799922573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b81a6440 con 0x7f25b8101530 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.372+0000 7f25b7fff640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25b8101530 0x7f25b80735f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25a800e9c0 tx=0x7f25a800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.372+0000 7f25b57fa640 1 -- 172.21.15.37:0/1799922573 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a800cde0 con 0x7f25b8101530 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.372+0000 7f25b57fa640 1 -- 172.21.15.37:0/1799922573 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25a80035f0 con 0x7f25b8101530 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.372+0000 7f25be7ab640 1 -- 172.21.15.37:0/1799922573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b81a6730 con 0x7f25b8101530 2024-03-26T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.372+0000 7f25b57fa640 1 -- 172.21.15.37:0/1799922573 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8010690 con 0x7f25b8101530 2024-03-26T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.372+0000 7f25be7ab640 1 -- 172.21.15.37:0/1799922573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b81a6ca0 con 0x7f25b8101530 2024-03-26T06:26:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.374+0000 7f25b57fa640 1 -- 172.21.15.37:0/1799922573 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f25a80037a0 con 0x7f25b8101530 2024-03-26T06:26:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25c1382740 1 Processor -- start 2024-03-26T06:26:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25c1382740 1 -- start start 2024-03-26T06:26:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25c1382740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec20a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25b57fa640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25a00a2ac0 0x7f25a00a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25c1382740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 0x55f51ec21450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25c1382740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec2c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25c1382740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51eb855f0 con 0x55f51ec21d40 2024-03-26T06:26:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25c1382740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51eb2e930 con 0x55f51ec20fd0 2024-03-26T06:26:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25c1382740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51eb81690 con 0x55f51ec2a550 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25bcd48640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec20a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25b7fff640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25a00a2ac0 0x7f25a00a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25bcd48640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec20a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50332/0 (socket says 172.21.15.37:50332) 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 0x55f51ec21450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25bcd48640 1 -- 172.21.15.37:0/102257340 learned_addr learned my addr 172.21.15.37:0/102257340 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.375+0000 7f25bcd48640 1 --2- 172.21.15.37:0/102257340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec2c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f25b57fa640 1 -- 172.21.15.37:0/1799922573 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f25a80cf0e0 con 0x7f25b8101530 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f25b77fe640 1 -- 172.21.15.37:0/102257340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 msgr2=0x55f51ec2c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f25b77fe640 1 --2- 172.21.15.37:0/102257340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec2c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f25b77fe640 1 -- 172.21.15.37:0/102257340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 msgr2=0x55f51ec20a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f25b77fe640 1 --2- 172.21.15.37:0/102257340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec20a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f25b77fe640 1 -- 172.21.15.37:0/102257340 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f51e89a210 con 0x55f51ec20fd0 2024-03-26T06:26:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f25bcd48640 1 --2- 172.21.15.37:0/102257340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec20a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f25b77fe640 1 --2- 172.21.15.37:0/102257340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 0x55f51ec21450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f25ac02c0b0 tx=0x7f25ac009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=785a56fe04a2e31e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f25b7fff640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25a00a2ac0 0x7f25a00a4f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25a80980e0 tx=0x7f25a8086040 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f259e7fc640 1 -- 172.21.15.37:0/102257340 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac043020 con 0x55f51ec20fd0 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f259e7fc640 1 -- 172.21.15.37:0/102257340 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25ac034cb0 con 0x55f51ec20fd0 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.376+0000 7f259e7fc640 1 -- 172.21.15.37:0/102257340 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac039040 con 0x55f51ec20fd0 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.377+0000 7f25c1382740 1 -- 172.21.15.37:0/102257340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 msgr2=0x55f51ec21450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.377+0000 7f25c1382740 1 --2- 172.21.15.37:0/102257340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 0x55f51ec21450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f25ac02c0b0 tx=0x7f25ac009510 comp rx=0 tx=0).stop 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.377+0000 7f25c1382740 1 -- 172.21.15.37:0/102257340 shutdown_connections 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.377+0000 7f25c1382740 1 --2- 172.21.15.37:0/102257340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec2c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.377+0000 7f25c1382740 1 --2- 172.21.15.37:0/102257340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 0x55f51ec21450 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.377+0000 7f25c1382740 1 --2- 172.21.15.37:0/102257340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec20a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.377+0000 7f25c1382740 1 -- 172.21.15.37:0/102257340 >> 172.21.15.37:0/102257340 conn(0x55f51ebe85b0 msgr2=0x55f51ec1bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.377+0000 7f25c1382740 1 -- 172.21.15.37:0/102257340 shutdown_connections 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.377+0000 7f25c1382740 1 -- 172.21.15.37:0/102257340 wait complete. 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25c1382740 1 -- start start 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25c1382740 1 Processor -- start 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25c1382740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec20fd0 0x55f51ec275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25c1382740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec27af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25c1382740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec2a550 0x55f51ec28030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25c1382740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51ec2d220 con 0x55f51ec21d40 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25c1382740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51eb41d90 con 0x55f51ec2a550 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25c1382740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51ec2d3a0 con 0x55f51ec20fd0 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25c1382740 10 client.0 _ll_register_callbacks cb 0x55f51e899a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25bcd48640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec20fd0 0x55f51ec275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec27af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25bcd48640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec20fd0 0x55f51ec275b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:58986/0 (socket says 172.21.15.37:58986) 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25bcd48640 1 -- 172.21.15.37:0/3777493643 learned_addr learned my addr 172.21.15.37:0/3777493643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.378+0000 7f25bcd48640 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec2a550 0x55f51ec28030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25bcd48640 1 -- 172.21.15.37:0/3777493643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec2a550 msgr2=0x55f51ec28030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25bcd48640 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec2a550 0x55f51ec28030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25bcd48640 1 -- 172.21.15.37:0/3777493643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 msgr2=0x55f51ec27af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25bcd48640 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec27af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25bcd48640 1 -- 172.21.15.37:0/3777493643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f51e966620 con 0x55f51ec20fd0 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25b77fe640 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec27af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25bcd48640 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec20fd0 0x55f51ec275b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f25b000f870 tx=0x7f25b0015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f595761a78a7814 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25977fe640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25977fe640 1 -- 172.21.15.37:0/3777493643 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0013120 con 0x55f51ec20fd0 2024-03-26T06:26:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25c1382740 1 -- 172.21.15.37:0/3777493643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25ac034cb0 con 0x55f51ec20fd0 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25977fe640 1 -- 172.21.15.37:0/3777493643 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f25b0003b80 con 0x55f51ec20fd0 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25977fe640 1 -- 172.21.15.37:0/3777493643 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b00132c0 con 0x55f51ec20fd0 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25c1382740 10 client.25192 fetch_fsmap learned FSMap version 31 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25c1382740 1 -- 172.21.15.37:0/3777493643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f51ec39580 con 0x55f51ec20fd0 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.379+0000 7f25977fe640 1 -- 172.21.15.37:0/3777493643 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25ac034cb0 con 0x55f51ec20fd0 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.380+0000 7f25977fe640 1 -- 172.21.15.37:0/3777493643 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f25b000ecf0 con 0x55f51ec20fd0 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.380+0000 7f25c1382740 10 client.25192 fetch_fsmap finished waiting for FSMap version 31 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.380+0000 7f25c1382740 10 client.25192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.380+0000 7f25c1382740 20 client.25192 populate_metadata read hostname 'smithi037' 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.380+0000 7f25c1382740 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f51ec3aa40 0x55f51ec3ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.380+0000 7f25c1382740 4 client.25192 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.380+0000 7f25c1382740 1 -- 172.21.15.37:0/3777493643 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f51eb92b30 con 0x55f51ec3aa40 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.380+0000 7f25b77fe640 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f51ec3aa40 0x55f51ec3ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.380+0000 7f25977fe640 10 client.25192 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.380+0000 7f25b77fe640 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f51ec3aa40 0x55f51ec3ce30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d7e473b451dc37 server_cookie=ed2ac1736f5dcb4c in_seq=0 out_seq=0 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25977fe640 1 -- 172.21.15.37:0/3777493643 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f51eb92b30 con 0x55f51ec3aa40 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25977fe640 10 client.25192 handle_command_reply: tid=0 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25c1382740 1 client.25192 shutdown 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25c1382740 1 -- 172.21.15.37:0/3777493643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec20fd0 msgr2=0x55f51ec275b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25c1382740 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec20fd0 0x55f51ec275b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f25b000f870 tx=0x7f25b0015040 comp rx=0 tx=0).stop 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25c1382740 1 -- 172.21.15.37:0/3777493643 shutdown_connections 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25c1382740 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f51ec3aa40 0x55f51ec3ce30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25c1382740 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec2a550 0x55f51ec28030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25c1382740 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec27af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25c1382740 1 --2- 172.21.15.37:0/3777493643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec20fd0 0x55f51ec275b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25977fe640 0 client.25192 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25c1382740 1 -- 172.21.15.37:0/3777493643 >> 172.21.15.37:0/3777493643 conn(0x55f51ebe85b0 msgr2=0x55f51ec1da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25c1382740 1 -- 172.21.15.37:0/3777493643 shutdown_connections 2024-03-26T06:26:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.381+0000 7f25c1382740 1 -- 172.21.15.37:0/3777493643 wait complete. 2024-03-26T06:26:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.382+0000 7f25c1382740 20 client.25192 trim_cache size 0 max 16384 2024-03-26T06:26:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.397+0000 7f25c1382740 1 Processor -- start 2024-03-26T06:26:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.397+0000 7f25c1382740 1 -- start start 2024-03-26T06:26:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25c1382740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec20fd0 0x55f51ec29ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25c1382740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec21d40 0x55f51ec31610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25c1382740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec31b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25c1382740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81aee20 con 0x55f51ec20fd0 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25c1382740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81b08c0 con 0x55f51ec21d40 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25c1382740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81af480 con 0x55f51ec2a550 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25bcd48640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec20fd0 0x55f51ec29ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25bcd48640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec20fd0 0x55f51ec29ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50346/0 (socket says 172.21.15.37:50346) 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec21d40 0x55f51ec31610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25bcd48640 1 -- 172.21.15.37:0/1926392226 learned_addr learned my addr 172.21.15.37:0/1926392226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25bcd48640 1 --2- 172.21.15.37:0/1926392226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec31b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25bcd48640 1 -- 172.21.15.37:0/1926392226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 msgr2=0x55f51ec31b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25bcd48640 1 --2- 172.21.15.37:0/1926392226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec31b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25bcd48640 1 -- 172.21.15.37:0/1926392226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec21d40 msgr2=0x55f51ec31610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25bcd48640 1 --2- 172.21.15.37:0/1926392226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec21d40 0x55f51ec31610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25bcd48640 1 -- 172.21.15.37:0/1926392226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f51ec202a0 con 0x55f51ec20fd0 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25b77fe640 1 --2- 172.21.15.37:0/1926392226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec21d40 0x55f51ec31610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:26:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f25bcd48640 1 --2- 172.21.15.37:0/1926392226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec20fd0 0x55f51ec29ce0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009d00 tx=0x7f25b0009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9de16129f818a76 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f259dffb640 1 -- 172.21.15.37:0/1926392226 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b000f450 con 0x55f51ec20fd0 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f259dffb640 1 -- 172.21.15.37:0/1926392226 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25b0013690 con 0x55f51ec20fd0 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.398+0000 7f259dffb640 1 -- 172.21.15.37:0/1926392226 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b000e9e0 con 0x55f51ec20fd0 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.399+0000 7f25c1382740 1 -- 172.21.15.37:0/1926392226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec20fd0 msgr2=0x55f51ec29ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.399+0000 7f25c1382740 1 --2- 172.21.15.37:0/1926392226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec20fd0 0x55f51ec29ce0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009d00 tx=0x7f25b0009d90 comp rx=0 tx=0).stop 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.399+0000 7f25c1382740 1 -- 172.21.15.37:0/1926392226 shutdown_connections 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.399+0000 7f25c1382740 1 --2- 172.21.15.37:0/1926392226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec31b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.399+0000 7f25c1382740 1 --2- 172.21.15.37:0/1926392226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec21d40 0x55f51ec31610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.399+0000 7f25c1382740 1 --2- 172.21.15.37:0/1926392226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec20fd0 0x55f51ec29ce0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.399+0000 7f25c1382740 1 -- 172.21.15.37:0/1926392226 >> 172.21.15.37:0/1926392226 conn(0x55f51ebe85b0 msgr2=0x55f51ec1b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.399+0000 7f25c1382740 1 -- 172.21.15.37:0/1926392226 shutdown_connections 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.399+0000 7f25c1382740 1 -- 172.21.15.37:0/1926392226 wait complete. 2024-03-26T06:26:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25c1382740 1 -- start start 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25c1382740 1 Processor -- start 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25c1382740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 0x55f51ec31770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25c1382740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec2dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25c1382740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec2e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25c1382740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81b01a0 con 0x55f51ec21d40 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25c1382740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81afb40 con 0x55f51ec20fd0 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25c1382740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81ad0c0 con 0x55f51ec2a550 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25c1382740 10 client.0 _ll_register_callbacks cb 0x7f25a8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25bcd48640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 0x55f51ec31770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec2dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25bcd48640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 0x55f51ec31770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:46694/0 (socket says 172.21.15.37:46694) 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25bcd48640 1 -- 172.21.15.37:0/2195292558 learned_addr learned my addr 172.21.15.37:0/2195292558 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.400+0000 7f25bcd48640 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec2e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25b77fe640 1 -- 172.21.15.37:0/2195292558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 msgr2=0x55f51ec2e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25b77fe640 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec2e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25b77fe640 1 -- 172.21.15.37:0/2195292558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 msgr2=0x55f51ec31770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25b77fe640 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 0x55f51ec31770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25b77fe640 1 -- 172.21.15.37:0/2195292558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f51ec2e6e0 con 0x55f51ec21d40 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25b77fe640 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec2dbb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009780 tx=0x7f25ac032500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413744bba2271a31 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25977fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25977fe640 1 -- 172.21.15.37:0/2195292558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac056070 con 0x55f51ec21d40 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25977fe640 1 -- 172.21.15.37:0/2195292558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f25ac050be0 con 0x55f51ec21d40 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25c1382740 1 -- 172.21.15.37:0/2195292558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25b0013690 con 0x55f51ec21d40 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25977fe640 1 -- 172.21.15.37:0/2195292558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ac05f050 con 0x55f51ec21d40 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25c1382740 10 client.15747 fetch_fsmap learned FSMap version 31 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25c1382740 1 -- 172.21.15.37:0/2195292558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f51ec91d40 con 0x55f51ec21d40 2024-03-26T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25977fe640 1 -- 172.21.15.37:0/2195292558 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25b0013690 con 0x55f51ec21d40 2024-03-26T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.401+0000 7f25977fe640 1 -- 172.21.15.37:0/2195292558 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f25ac059020 con 0x55f51ec21d40 2024-03-26T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.402+0000 7f25c1382740 10 client.15747 fetch_fsmap finished waiting for FSMap version 31 2024-03-26T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.402+0000 7f25c1382740 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.402+0000 7f25c1382740 20 client.15747 populate_metadata read hostname 'smithi037' 2024-03-26T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.402+0000 7f25c1382740 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f51ec92140 0x55f51ec94530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.402+0000 7f25c1382740 4 client.15747 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.402+0000 7f25c1382740 1 -- 172.21.15.37:0/2195292558 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f51eb8ce80 con 0x55f51ec92140 2024-03-26T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.402+0000 7f25bcd48640 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f51ec92140 0x55f51ec94530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.402+0000 7f25bcd48640 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f51ec92140 0x55f51ec94530 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a98d897c7067c00 server_cookie=a3eeb00c0717bcd0 in_seq=0 out_seq=0 2024-03-26T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.402+0000 7f25977fe640 10 client.15747 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.430+0000 7f25977fe640 1 -- 172.21.15.37:0/2195292558 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f51eb8ce80 con 0x55f51ec92140 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.430+0000 7f25977fe640 10 client.15747 handle_command_reply: tid=0 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.430+0000 7f25c1382740 1 client.15747 shutdown 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.430+0000 7f25c1382740 1 -- 172.21.15.37:0/2195292558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 msgr2=0x55f51ec2dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.430+0000 7f25c1382740 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec2dbb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009780 tx=0x7f25ac032500 comp rx=0 tx=0).stop 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.430+0000 7f25c1382740 1 -- 172.21.15.37:0/2195292558 shutdown_connections 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.430+0000 7f25c1382740 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f51ec92140 0x55f51ec94530 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25c1382740 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51ec2a550 0x55f51ec2e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25c1382740 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f51ec21d40 0x55f51ec2dbb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25c1382740 1 --2- 172.21.15.37:0/2195292558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f51ec20fd0 0x55f51ec31770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25c1382740 1 -- 172.21.15.37:0/2195292558 >> 172.21.15.37:0/2195292558 conn(0x55f51ebe85b0 msgr2=0x55f51ec25d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25977fe640 0 client.15747 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25c1382740 1 -- 172.21.15.37:0/2195292558 shutdown_connections 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25c1382740 1 -- 172.21.15.37:0/2195292558 wait complete. 2024-03-26T06:26:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25c1382740 20 client.15747 trim_cache size 0 max 16384 2024-03-26T06:26:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:26:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:26:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:26:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:26:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25be7ab640 1 -- 172.21.15.37:0/1799922573 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25a00a2ac0 msgr2=0x7f25a00a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25be7ab640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25a00a2ac0 0x7f25a00a4f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25a80980e0 tx=0x7f25a8086040 comp rx=0 tx=0).stop 2024-03-26T06:26:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25be7ab640 1 -- 172.21.15.37:0/1799922573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25b8101530 msgr2=0x7f25b80735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.431+0000 7f25be7ab640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25b8101530 0x7f25b80735f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25a800e9c0 tx=0x7f25a800ee80 comp rx=0 tx=0).stop 2024-03-26T06:26:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.432+0000 7f25be7ab640 1 -- 172.21.15.37:0/1799922573 shutdown_connections 2024-03-26T06:26:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.432+0000 7f25be7ab640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25a00a2ac0 0x7f25a00a4f80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.432+0000 7f25be7ab640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25b8074070 0x7f25b81a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.432+0000 7f25be7ab640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b8106600 0x7f25b8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.432+0000 7f25be7ab640 1 --2- 172.21.15.37:0/1799922573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25b8101530 0x7f25b80735f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.432+0000 7f25be7ab640 1 -- 172.21.15.37:0/1799922573 >> 172.21.15.37:0/1799922573 conn(0x7f25b80fcdd0 msgr2=0x7f25b80ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.432+0000 7f25be7ab640 1 -- 172.21.15.37:0/1799922573 shutdown_connections 2024-03-26T06:26:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:05.432+0000 7f25be7ab640 1 -- 172.21.15.37:0/1799922573 wait complete. 2024-03-26T06:26:05.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 72389 4.08 MB/sec warmup 100 sec latency 2950.515 ms 2024-03-26T06:26:06.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 74868 4.18 MB/sec warmup 101 sec latency 11.059 ms 2024-03-26T06:26:06.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:06 smithi066 ceph-mon[36504]: pgmap v96: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 226 op/s 2024-03-26T06:26:06.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:06 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:26:06.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:06 smithi149 ceph-mon[36681]: pgmap v96: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 226 op/s 2024-03-26T06:26:06.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:06 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:26:06.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:06 smithi037 ceph-mon[31974]: pgmap v96: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 226 op/s 2024-03-26T06:26:06.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:06 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:26:07.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75210 4.14 MB/sec warmup 102 sec latency 625.963 ms 2024-03-26T06:26:07.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:07 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:26:07.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:07 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:26:07.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:07 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:26:07.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:07 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:26:07.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:07 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:26:07.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:07 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:26:07.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:07 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:26:07.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:07 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:26:07.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:07 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:26:07.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:07 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:26:07.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:07 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:26:07.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:07 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:26:08.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75210 4.10 MB/sec warmup 103 sec latency 1626.042 ms 2024-03-26T06:26:08.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:08 smithi066 ceph-mon[36504]: pgmap v97: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 859 KiB/s wr, 140 op/s 2024-03-26T06:26:08.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:08 smithi149 ceph-mon[36681]: pgmap v97: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 859 KiB/s wr, 140 op/s 2024-03-26T06:26:08.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:08 smithi037 ceph-mon[31974]: pgmap v97: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 859 KiB/s wr, 140 op/s 2024-03-26T06:26:09.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75210 4.06 MB/sec warmup 104 sec latency 2626.091 ms 2024-03-26T06:26:10.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75210 4.03 MB/sec warmup 105 sec latency 3626.181 ms 2024-03-26T06:26:10.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:10 smithi066 ceph-mon[36504]: pgmap v98: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 859 KiB/s wr, 140 op/s 2024-03-26T06:26:10.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:10 smithi149 ceph-mon[36681]: pgmap v98: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 859 KiB/s wr, 140 op/s 2024-03-26T06:26:10.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:10 smithi037 ceph-mon[31974]: pgmap v98: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 859 KiB/s wr, 140 op/s 2024-03-26T06:26:11.491 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75265 3.99 MB/sec warmup 106 sec latency 4438.062 ms 2024-03-26T06:26:12.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75265 3.95 MB/sec warmup 107 sec latency 1161.853 ms 2024-03-26T06:26:12.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:12 smithi066 ceph-mon[36504]: pgmap v99: 97 pgs: 97 active+clean; 39 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-03-26T06:26:12.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:12 smithi149 ceph-mon[36681]: pgmap v99: 97 pgs: 97 active+clean; 39 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-03-26T06:26:12.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:12 smithi037 ceph-mon[31974]: pgmap v99: 97 pgs: 97 active+clean; 39 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 208 op/s 2024-03-26T06:26:13.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75265 3.92 MB/sec warmup 108 sec latency 2161.980 ms 2024-03-26T06:26:14.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75265 3.88 MB/sec warmup 109 sec latency 3162.078 ms 2024-03-26T06:26:14.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:14 smithi066 ceph-mon[36504]: pgmap v100: 97 pgs: 97 active+clean; 39 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-03-26T06:26:14.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:14 smithi149 ceph-mon[36681]: pgmap v100: 97 pgs: 97 active+clean; 39 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-03-26T06:26:14.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:14 smithi037 ceph-mon[31974]: pgmap v100: 97 pgs: 97 active+clean; 39 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-03-26T06:26:15.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75265 3.84 MB/sec warmup 110 sec latency 4162.178 ms 2024-03-26T06:26:16.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75334 3.81 MB/sec warmup 111 sec latency 4977.244 ms 2024-03-26T06:26:16.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:16 smithi066 ceph-mon[36504]: pgmap v101: 97 pgs: 97 active+clean; 36 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-03-26T06:26:16.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:16 smithi149 ceph-mon[36681]: pgmap v101: 97 pgs: 97 active+clean; 36 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-03-26T06:26:16.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:16 smithi037 ceph-mon[31974]: pgmap v101: 97 pgs: 97 active+clean; 36 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-03-26T06:26:17.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75334 3.78 MB/sec warmup 112 sec latency 1134.682 ms 2024-03-26T06:26:18.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75334 3.74 MB/sec warmup 113 sec latency 2134.727 ms 2024-03-26T06:26:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.530+0000 7f32329b9640 1 -- 172.21.15.37:0/1525259228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c105170 msgr2=0x7f322c105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.530+0000 7f32329b9640 1 --2- 172.21.15.37:0/1525259228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c105170 0x7f322c105550 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3214009900 tx=0x7f321402bc30 comp rx=0 tx=0).stop 2024-03-26T06:26:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.530+0000 7f32329b9640 1 -- 172.21.15.37:0/1525259228 shutdown_connections 2024-03-26T06:26:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.530+0000 7f32329b9640 1 --2- 172.21.15.37:0/1525259228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f322c100de0 0x7f322c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.530+0000 7f32329b9640 1 --2- 172.21.15.37:0/1525259228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f322c100440 0x7f322c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.530+0000 7f32329b9640 1 --2- 172.21.15.37:0/1525259228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c105170 0x7f322c105550 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.530+0000 7f32329b9640 1 -- 172.21.15.37:0/1525259228 >> 172.21.15.37:0/1525259228 conn(0x7f322c076fc0 msgr2=0x7f322c0773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.530+0000 7f32329b9640 1 -- 172.21.15.37:0/1525259228 shutdown_connections 2024-03-26T06:26:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.530+0000 7f32329b9640 1 -- 172.21.15.37:0/1525259228 wait complete. 2024-03-26T06:26:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.530+0000 7f32329b9640 1 Processor -- start 2024-03-26T06:26:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.530+0000 7f32329b9640 1 -- start start 2024-03-26T06:26:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f32329b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c100440 0x7f322c19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f32329b9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f322c100de0 0x7f322c19dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f32329b9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f322c105170 0x7f322c197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f32329b9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c110640 con 0x7f322c100440 2024-03-26T06:26:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f32329b9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1104c0 con 0x7f322c105170 2024-03-26T06:26:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f32329b9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1107c0 con 0x7f322c100de0 2024-03-26T06:26:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f322bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c100440 0x7f322c19d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f322bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c100440 0x7f322c19d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47178/0 (socket says 172.21.15.37:47178) 2024-03-26T06:26:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f322bfff640 1 -- 172.21.15.37:0/1820710566 learned_addr learned my addr 172.21.15.37:0/1820710566 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f322b7fe640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f322c100de0 0x7f322c19dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f3230f56640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f322c105170 0x7f322c197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f322bfff640 1 -- 172.21.15.37:0/1820710566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f322c100de0 msgr2=0x7f322c19dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f322bfff640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f322c100de0 0x7f322c19dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f322bfff640 1 -- 172.21.15.37:0/1820710566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f322c105170 msgr2=0x7f322c197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f322bfff640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f322c105170 0x7f322c197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f322bfff640 1 -- 172.21.15.37:0/1820710566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c1980b0 con 0x7f322c100440 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f322b7fe640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f322c100de0 0x7f322c19dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.531+0000 7f3230f56640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f322c105170 0x7f322c197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.532+0000 7f322bfff640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c100440 0x7f322c19d780 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3214006010 tx=0x7f321402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.532+0000 7f32297fa640 1 -- 172.21.15.37:0/1820710566 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214041070 con 0x7f322c100440 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.532+0000 7f32329b9640 1 -- 172.21.15.37:0/1820710566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c198340 con 0x7f322c100440 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.532+0000 7f32297fa640 1 -- 172.21.15.37:0/1820710566 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3214004450 con 0x7f322c100440 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.532+0000 7f32297fa640 1 -- 172.21.15.37:0/1820710566 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214034440 con 0x7f322c100440 2024-03-26T06:26:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.532+0000 7f32329b9640 1 -- 172.21.15.37:0/1820710566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c1a4640 con 0x7f322c100440 2024-03-26T06:26:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.534+0000 7f32297fa640 1 -- 172.21.15.37:0/1820710566 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3214044070 con 0x7f322c100440 2024-03-26T06:26:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.534+0000 7f3235596740 1 Processor -- start 2024-03-26T06:26:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f32297fa640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f32180a2a10 0x7f32180a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3235596740 1 -- start start 2024-03-26T06:26:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3230f56640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f32180a2a10 0x7f32180a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f32297fa640 1 -- 172.21.15.37:0/1820710566 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f32140f0a50 con 0x7f322c100440 2024-03-26T06:26:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3235596740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bafa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3235596740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bb0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3235596740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3235596740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f06b145f0 con 0x557f06bbd960 2024-03-26T06:26:18.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3235596740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f06abd930 con 0x557f06bb0d40 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3235596740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f06b10690 con 0x557f06baffd0 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3230f56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3230f56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47182/0 (socket says 172.21.15.37:47182) 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3230f56640 1 -- 172.21.15.37:0/2883401959 learned_addr learned my addr 172.21.15.37:0/2883401959 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f322bfff640 1 --2- 172.21.15.37:0/2883401959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bafa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f322b7fe640 1 --2- 172.21.15.37:0/2883401959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bb0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3230f56640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f32180a2a10 0x7f32180a4ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32240147a0 tx=0x7f3224014ba0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3230f56640 1 -- 172.21.15.37:0/2883401959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 msgr2=0x557f06bb0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3230f56640 1 --2- 172.21.15.37:0/2883401959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bb0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3230f56640 1 -- 172.21.15.37:0/2883401959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 msgr2=0x557f06bafa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3230f56640 1 --2- 172.21.15.37:0/2883401959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bafa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f3230f56640 1 -- 172.21.15.37:0/2883401959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f06829210 con 0x557f06bbd960 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.535+0000 7f322b7fe640 1 --2- 172.21.15.37:0/2883401959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bb0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.536+0000 7f3230f56640 1 --2- 172.21.15.37:0/2883401959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbfd50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3224012760 tx=0x7f322401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd95a4422052f681 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.536+0000 7f32227fc640 1 -- 172.21.15.37:0/2883401959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322400eae0 con 0x557f06bbd960 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.536+0000 7f32227fc640 1 -- 172.21.15.37:0/2883401959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f322400ec80 con 0x557f06bbd960 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.536+0000 7f32227fc640 1 -- 172.21.15.37:0/2883401959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322401f6c0 con 0x557f06bbd960 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.536+0000 7f3235596740 1 -- 172.21.15.37:0/2883401959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 msgr2=0x557f06bbfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.536+0000 7f3235596740 1 --2- 172.21.15.37:0/2883401959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbfd50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3224012760 tx=0x7f322401e000 comp rx=0 tx=0).stop 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.536+0000 7f3235596740 1 -- 172.21.15.37:0/2883401959 shutdown_connections 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 --2- 172.21.15.37:0/2883401959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbfd50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 --2- 172.21.15.37:0/2883401959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bb0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 --2- 172.21.15.37:0/2883401959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bafa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 -- 172.21.15.37:0/2883401959 >> 172.21.15.37:0/2883401959 conn(0x557f06b775b0 msgr2=0x557f06baacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 -- 172.21.15.37:0/2883401959 shutdown_connections 2024-03-26T06:26:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 -- 172.21.15.37:0/2883401959 wait complete. 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 -- start start 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 Processor -- start 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bba9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bbaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f06bc0630 con 0x557f06bbd960 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f06ad0d90 con 0x557f06bb0d40 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f06bc07b0 con 0x557f06baffd0 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.537+0000 7f3235596740 10 client.0 _ll_register_callbacks cb 0x557f06828a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f3230f56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f3230f56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47192/0 (socket says 172.21.15.37:47192) 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f3230f56640 1 -- 172.21.15.37:0/629335898 learned_addr learned my addr 172.21.15.37:0/629335898 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f322bfff640 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bba9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f322b7fe640 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bbaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f3230f56640 1 -- 172.21.15.37:0/629335898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 msgr2=0x557f06bba9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f3230f56640 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bba9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f3230f56640 1 -- 172.21.15.37:0/629335898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 msgr2=0x557f06bbaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f3230f56640 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bbaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f3230f56640 1 -- 172.21.15.37:0/629335898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f068f5620 con 0x557f06bbd960 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f322b7fe640 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bbaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f322bfff640 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bba9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f3230f56640 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbb440 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f322400e730 tx=0x7f3224009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fed487ac6d7e27 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f320b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f320b7fe640 1 -- 172.21.15.37:0/629335898 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224037060 con 0x557f06bbd960 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f320b7fe640 1 -- 172.21.15.37:0/629335898 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f3224004a40 con 0x557f06bbd960 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.538+0000 7f3235596740 1 -- 172.21.15.37:0/629335898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f322400ec80 con 0x557f06bbd960 2024-03-26T06:26:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f320b7fe640 1 -- 172.21.15.37:0/629335898 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3224004c40 con 0x557f06bbd960 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f3235596740 10 client.15759 fetch_fsmap learned FSMap version 32 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f3235596740 1 -- 172.21.15.37:0/629335898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557f06bcc970 con 0x557f06bbd960 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f320b7fe640 1 -- 172.21.15.37:0/629335898 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f322400ec80 con 0x557f06bbd960 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f320b7fe640 1 -- 172.21.15.37:0/629335898 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f322400f040 con 0x557f06bbd960 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f3235596740 10 client.15759 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f3235596740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f3235596740 20 client.15759 populate_metadata read hostname 'smithi037' 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f3235596740 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f06bcddd0 0x557f06bd01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f3235596740 4 client.15759 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f3235596740 1 -- 172.21.15.37:0/629335898 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f06b21b30 con 0x557f06bcddd0 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.539+0000 7f322bfff640 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f06bcddd0 0x557f06bd01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.540+0000 7f320b7fe640 10 client.15759 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.540+0000 7f322bfff640 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f06bcddd0 0x557f06bd01c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbffaa702c22aaa5 server_cookie=eaec3e344c225a23 in_seq=0 out_seq=0 2024-03-26T06:26:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.540+0000 7f320b7fe640 1 -- 172.21.15.37:0/629335898 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f06b21b30 con 0x557f06bcddd0 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.540+0000 7f320b7fe640 10 client.15759 handle_command_reply: tid=0 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 1 client.15759 shutdown 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 1 -- 172.21.15.37:0/629335898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 msgr2=0x557f06bbb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbb440 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f322400e730 tx=0x7f3224009b10 comp rx=0 tx=0).stop 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 1 -- 172.21.15.37:0/629335898 shutdown_connections 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f06bcddd0 0x557f06bd01c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bbb440 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bbaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 1 --2- 172.21.15.37:0/629335898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bba9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 1 -- 172.21.15.37:0/629335898 >> 172.21.15.37:0/629335898 conn(0x557f06b775b0 msgr2=0x557f06baca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f320b7fe640 0 client.15759 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 1 -- 172.21.15.37:0/629335898 shutdown_connections 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 1 -- 172.21.15.37:0/629335898 wait complete. 2024-03-26T06:26:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.541+0000 7f3235596740 20 client.15759 trim_cache size 0 max 16384 2024-03-26T06:26:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.554+0000 7f3235596740 1 Processor -- start 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.554+0000 7f3235596740 1 -- start start 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3235596740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bbd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3235596740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bc4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3235596740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bc4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3235596740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1aa9c0 con 0x557f06bbd960 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3235596740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1ac490 con 0x557f06bb0d40 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3235596740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1ab020 con 0x557f06baffd0 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3230f56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bc4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3230f56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bc4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47204/0 (socket says 172.21.15.37:47204) 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3230f56640 1 -- 172.21.15.37:0/2575184997 learned_addr learned my addr 172.21.15.37:0/2575184997 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f322bfff640 1 --2- 172.21.15.37:0/2575184997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bbd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f322b7fe640 1 --2- 172.21.15.37:0/2575184997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bc4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3230f56640 1 -- 172.21.15.37:0/2575184997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 msgr2=0x557f06bbd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3230f56640 1 --2- 172.21.15.37:0/2575184997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bbd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3230f56640 1 -- 172.21.15.37:0/2575184997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 msgr2=0x557f06bc4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3230f56640 1 --2- 172.21.15.37:0/2575184997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bc4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3230f56640 1 -- 172.21.15.37:0/2575184997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f06baf2a0 con 0x557f06bbd960 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f322b7fe640 1 --2- 172.21.15.37:0/2575184997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bc4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3230f56640 1 --2- 172.21.15.37:0/2575184997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bc4f90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3224000f80 tx=0x7f3224020d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ea87d90150b28e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3221ffb640 1 -- 172.21.15.37:0/2575184997 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224037070 con 0x557f06bbd960 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3221ffb640 1 -- 172.21.15.37:0/2575184997 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3224004a00 con 0x557f06bbd960 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.555+0000 7f3221ffb640 1 -- 172.21.15.37:0/2575184997 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224020440 con 0x557f06bbd960 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.556+0000 7f3235596740 1 -- 172.21.15.37:0/2575184997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 msgr2=0x557f06bc4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.556+0000 7f3235596740 1 --2- 172.21.15.37:0/2575184997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bc4f90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3224000f80 tx=0x7f3224020d20 comp rx=0 tx=0).stop 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.556+0000 7f3235596740 1 -- 172.21.15.37:0/2575184997 shutdown_connections 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.556+0000 7f3235596740 1 --2- 172.21.15.37:0/2575184997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bbd960 0x557f06bc4f90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.556+0000 7f3235596740 1 --2- 172.21.15.37:0/2575184997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bb0d40 0x557f06bc4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.556+0000 7f3235596740 1 --2- 172.21.15.37:0/2575184997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bbd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.556+0000 7f3235596740 1 -- 172.21.15.37:0/2575184997 >> 172.21.15.37:0/2575184997 conn(0x557f06b775b0 msgr2=0x557f06baf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.556+0000 7f3235596740 1 -- 172.21.15.37:0/2575184997 shutdown_connections 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.556+0000 7f3235596740 1 -- 172.21.15.37:0/2575184997 wait complete. 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.556+0000 7f3235596740 1 -- start start 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f3235596740 1 Processor -- start 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f3235596740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bc4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f3235596740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bb0d40 0x557f06bb72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f3235596740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bbd960 0x557f06bb7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f3235596740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1abd70 con 0x557f06bb0d40 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f3235596740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1ab710 con 0x557f06bbd960 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f3235596740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1a8c60 con 0x557f06baffd0 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f3235596740 10 client.0 _ll_register_callbacks cb 0x7f3224009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f322b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bb0d40 0x557f06bb72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f322b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bb0d40 0x557f06bb72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47216/0 (socket says 172.21.15.37:47216) 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f322b7fe640 1 -- 172.21.15.37:0/125288782 learned_addr learned my addr 172.21.15.37:0/125288782 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f3230f56640 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bbd960 0x557f06bb7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f322bfff640 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bc4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f322b7fe640 1 -- 172.21.15.37:0/125288782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 msgr2=0x557f06bc4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f322b7fe640 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bc4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f322b7fe640 1 -- 172.21.15.37:0/125288782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bbd960 msgr2=0x557f06bb7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f322b7fe640 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bbd960 0x557f06bb7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f322b7fe640 1 -- 172.21.15.37:0/125288782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f06bb8040 con 0x557f06bb0d40 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.557+0000 7f322bfff640 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bc4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f322b7fe640 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bb0d40 0x557f06bb72d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f322c199120 tx=0x7f321c003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9163a2a30308cccb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f320b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f320b7fe640 1 -- 172.21.15.37:0/125288782 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c020060 con 0x557f06bb0d40 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f320b7fe640 1 -- 172.21.15.37:0/125288782 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f321c012cd0 con 0x557f06bb0d40 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f3235596740 1 -- 172.21.15.37:0/125288782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3224004a00 con 0x557f06bb0d40 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f320b7fe640 1 -- 172.21.15.37:0/125288782 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f321c025480 con 0x557f06bb0d40 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f3235596740 10 client.15765 fetch_fsmap learned FSMap version 32 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f3235596740 1 -- 172.21.15.37:0/125288782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557f06c24ff0 con 0x557f06bb0d40 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f320b7fe640 1 -- 172.21.15.37:0/125288782 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3224004a00 con 0x557f06bb0d40 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f320b7fe640 1 -- 172.21.15.37:0/125288782 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f321c0108f0 con 0x557f06bb0d40 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f3235596740 10 client.15765 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f3235596740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f3235596740 20 client.15765 populate_metadata read hostname 'smithi037' 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f3235596740 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f06c25810 0x557f06c27c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f3235596740 4 client.15765 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.558+0000 7f3235596740 1 -- 172.21.15.37:0/125288782 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f06b1be80 con 0x557f06c25810 2024-03-26T06:26:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.559+0000 7f322bfff640 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f06c25810 0x557f06c27c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.559+0000 7f320b7fe640 10 client.15765 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.559+0000 7f322bfff640 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f06c25810 0x557f06c27c30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0221a8a5db0976 server_cookie=d8ac2a5fbb76ce7e in_seq=0 out_seq=0 2024-03-26T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f320b7fe640 1 -- 172.21.15.37:0/125288782 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f06b1be80 con 0x557f06c25810 2024-03-26T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f320b7fe640 10 client.15765 handle_command_reply: tid=0 2024-03-26T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 1 client.15765 shutdown 2024-03-26T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 1 -- 172.21.15.37:0/125288782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bb0d40 msgr2=0x557f06bb72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bb0d40 0x557f06bb72d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f322c199120 tx=0x7f321c003ed0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 1 -- 172.21.15.37:0/125288782 shutdown_connections 2024-03-26T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557f06c25810 0x557f06c27c30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f06bbd960 0x557f06bb7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06bb0d40 0x557f06bb72d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f320b7fe640 0 client.15765 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 1 --2- 172.21.15.37:0/125288782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f06baffd0 0x557f06bc4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 1 -- 172.21.15.37:0/125288782 >> 172.21.15.37:0/125288782 conn(0x557f06b775b0 msgr2=0x557f06bb9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 1 -- 172.21.15.37:0/125288782 shutdown_connections 2024-03-26T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 1 -- 172.21.15.37:0/125288782 wait complete. 2024-03-26T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.585+0000 7f3235596740 20 client.15765 trim_cache size 0 max 16384 2024-03-26T06:26:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:26:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:26:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:26:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:26:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 -- 172.21.15.37:0/1820710566 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f32180a2a10 msgr2=0x7f32180a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f32180a2a10 0x7f32180a4ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32240147a0 tx=0x7f3224014ba0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 -- 172.21.15.37:0/1820710566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c100440 msgr2=0x7f322c19d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c100440 0x7f322c19d780 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3214006010 tx=0x7f321402cc70 comp rx=0 tx=0).stop 2024-03-26T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 -- 172.21.15.37:0/1820710566 shutdown_connections 2024-03-26T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f32180a2a10 0x7f32180a4ed0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f322c105170 0x7f322c197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f322c100de0 0x7f322c19dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 --2- 172.21.15.37:0/1820710566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c100440 0x7f322c19d780 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 -- 172.21.15.37:0/1820710566 >> 172.21.15.37:0/1820710566 conn(0x7f322c076fc0 msgr2=0x7f322c069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 -- 172.21.15.37:0/1820710566 shutdown_connections 2024-03-26T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:18.586+0000 7f32329b9640 1 -- 172.21.15.37:0/1820710566 wait complete. 2024-03-26T06:26:18.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:18 smithi066 ceph-mon[36504]: pgmap v102: 97 pgs: 97 active+clean; 36 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 608 KiB/s wr, 83 op/s 2024-03-26T06:26:18.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:18 smithi149 ceph-mon[36681]: pgmap v102: 97 pgs: 97 active+clean; 36 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 608 KiB/s wr, 83 op/s 2024-03-26T06:26:18.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:18 smithi037 ceph-mon[31974]: pgmap v102: 97 pgs: 97 active+clean; 36 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 608 KiB/s wr, 83 op/s 2024-03-26T06:26:19.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75334 3.71 MB/sec warmup 114 sec latency 3134.803 ms 2024-03-26T06:26:20.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75334 3.68 MB/sec warmup 115 sec latency 4134.894 ms 2024-03-26T06:26:20.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:20 smithi066 ceph-mon[36504]: pgmap v103: 97 pgs: 97 active+clean; 36 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 608 KiB/s wr, 83 op/s 2024-03-26T06:26:20.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:20 smithi149 ceph-mon[36681]: pgmap v103: 97 pgs: 97 active+clean; 36 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 608 KiB/s wr, 83 op/s 2024-03-26T06:26:20.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:20 smithi037 ceph-mon[31974]: pgmap v103: 97 pgs: 97 active+clean; 36 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 608 KiB/s wr, 83 op/s 2024-03-26T06:26:21.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75456 3.65 MB/sec warmup 116 sec latency 4944.114 ms 2024-03-26T06:26:22.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75456 3.62 MB/sec warmup 117 sec latency 1131.670 ms 2024-03-26T06:26:22.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:22 smithi066 ceph-mon[36504]: pgmap v104: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 647 KiB/s wr, 100 op/s 2024-03-26T06:26:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:22 smithi149 ceph-mon[36681]: pgmap v104: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 647 KiB/s wr, 100 op/s 2024-03-26T06:26:22.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:22 smithi037 ceph-mon[31974]: pgmap v104: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 647 KiB/s wr, 100 op/s 2024-03-26T06:26:23.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75456 3.59 MB/sec warmup 118 sec latency 2131.721 ms 2024-03-26T06:26:24.492 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75456 3.56 MB/sec warmup 119 sec latency 3131.765 ms 2024-03-26T06:26:24.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:24 smithi066 ceph-mon[36504]: pgmap v105: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 79 KiB/s wr, 32 op/s 2024-03-26T06:26:24.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:24 smithi149 ceph-mon[36681]: pgmap v105: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 79 KiB/s wr, 32 op/s 2024-03-26T06:26:24.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:24 smithi037 ceph-mon[31974]: pgmap v105: 97 pgs: 97 active+clean; 37 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 79 KiB/s wr, 32 op/s 2024-03-26T06:26:26.493 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75523 0.13 MB/sec execute 1 sec latency 4948.176 ms 2024-03-26T06:26:26.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:26 smithi066 ceph-mon[36504]: pgmap v106: 97 pgs: 97 active+clean; 42 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 579 KiB/s wr, 52 op/s 2024-03-26T06:26:26.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:26 smithi149 ceph-mon[36681]: pgmap v106: 97 pgs: 97 active+clean; 42 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 579 KiB/s wr, 52 op/s 2024-03-26T06:26:26.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:26 smithi037 ceph-mon[31974]: pgmap v106: 97 pgs: 97 active+clean; 42 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 579 KiB/s wr, 52 op/s 2024-03-26T06:26:27.493 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75523 0.07 MB/sec execute 2 sec latency 1138.266 ms 2024-03-26T06:26:28.493 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75523 0.04 MB/sec execute 3 sec latency 2138.352 ms 2024-03-26T06:26:28.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:28 smithi066 ceph-mon[36504]: pgmap v107: 97 pgs: 97 active+clean; 42 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 539 KiB/s wr, 38 op/s 2024-03-26T06:26:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:28 smithi149 ceph-mon[36681]: pgmap v107: 97 pgs: 97 active+clean; 42 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 539 KiB/s wr, 38 op/s 2024-03-26T06:26:28.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:28 smithi037 ceph-mon[31974]: pgmap v107: 97 pgs: 97 active+clean; 42 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 539 KiB/s wr, 38 op/s 2024-03-26T06:26:29.493 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75523 0.03 MB/sec execute 4 sec latency 3138.392 ms 2024-03-26T06:26:30.493 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75523 0.03 MB/sec execute 5 sec latency 4138.496 ms 2024-03-26T06:26:30.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:30 smithi066 ceph-mon[36504]: pgmap v108: 97 pgs: 97 active+clean; 42 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 539 KiB/s wr, 38 op/s 2024-03-26T06:26:30.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:30 smithi149 ceph-mon[36681]: pgmap v108: 97 pgs: 97 active+clean; 42 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 539 KiB/s wr, 38 op/s 2024-03-26T06:26:30.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:30 smithi037 ceph-mon[31974]: pgmap v108: 97 pgs: 97 active+clean; 42 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 539 KiB/s wr, 38 op/s 2024-03-26T06:26:31.493 INFO:tasks.workunit.client.0.smithi037.stdout: 1 75968 0.72 MB/sec execute 6 sec latency 4947.989 ms 2024-03-26T06:26:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.682+0000 7fb7686f7640 1 -- 172.21.15.37:0/3643750960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb760101ec0 msgr2=0x7fb760105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.682+0000 7fb7686f7640 1 --2- 172.21.15.37:0/3643750960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb760101ec0 0x7fb760105f00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb754009900 tx=0x7fb75402bc30 comp rx=0 tx=0).stop 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.682+0000 7fb7686f7640 1 -- 172.21.15.37:0/3643750960 shutdown_connections 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.682+0000 7fb7686f7640 1 --2- 172.21.15.37:0/3643750960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7601065e0 0x7fb76010e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.682+0000 7fb7686f7640 1 --2- 172.21.15.37:0/3643750960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb760101ec0 0x7fb760105f00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.682+0000 7fb7686f7640 1 --2- 172.21.15.37:0/3643750960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb760101510 0x7fb7601018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.682+0000 7fb7686f7640 1 -- 172.21.15.37:0/3643750960 >> 172.21.15.37:0/3643750960 conn(0x7fb7600fcdb0 msgr2=0x7fb7600ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.682+0000 7fb7686f7640 1 -- 172.21.15.37:0/3643750960 shutdown_connections 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.682+0000 7fb7686f7640 1 -- 172.21.15.37:0/3643750960 wait complete. 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb7686f7640 1 Processor -- start 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb7686f7640 1 -- start start 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb7686f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb760101510 0x7fb76019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb7686f7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7601065e0 0x7fb76019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb7686f7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7601978f0 0x7fb760197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb7686f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb760111670 con 0x7fb760101510 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb7686f7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601114f0 con 0x7fb7601978f0 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb7686f7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601117f0 con 0x7fb7601065e0 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb766493640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb760101510 0x7fb76019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb766493640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb760101510 0x7fb76019d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56136/0 (socket says 172.21.15.37:56136) 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb766493640 1 -- 172.21.15.37:0/2376921540 learned_addr learned my addr 172.21.15.37:0/2376921540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb766c94640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7601978f0 0x7fb760197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb765c92640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7601065e0 0x7fb76019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb766493640 1 -- 172.21.15.37:0/2376921540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7601065e0 msgr2=0x7fb76019dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb766493640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7601065e0 0x7fb76019dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb766493640 1 -- 172.21.15.37:0/2376921540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7601978f0 msgr2=0x7fb760197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb766493640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7601978f0 0x7fb760197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb766493640 1 -- 172.21.15.37:0/2376921540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb760198410 con 0x7fb760101510 2024-03-26T06:26:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.683+0000 7fb766493640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb760101510 0x7fb76019d740 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb75800d770 tx=0x7fb75800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.684+0000 7fb75f7fe640 1 -- 172.21.15.37:0/2376921540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758017070 con 0x7fb760101510 2024-03-26T06:26:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.684+0000 7fb75f7fe640 1 -- 172.21.15.37:0/2376921540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb758005500 con 0x7fb760101510 2024-03-26T06:26:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.684+0000 7fb7686f7640 1 -- 172.21.15.37:0/2376921540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb760198700 con 0x7fb760101510 2024-03-26T06:26:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.684+0000 7fb75f7fe640 1 -- 172.21.15.37:0/2376921540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758004e00 con 0x7fb760101510 2024-03-26T06:26:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.684+0000 7fb7686f7640 1 -- 172.21.15.37:0/2376921540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7601a4900 con 0x7fb760101510 2024-03-26T06:26:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.686+0000 7fb75f7fe640 1 -- 172.21.15.37:0/2376921540 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb758004fa0 con 0x7fb760101510 2024-03-26T06:26:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.686+0000 7fb75f7fe640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb7480a2a10 0x7fb7480a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.686+0000 7fb76b3a8740 1 Processor -- start 2024-03-26T06:26:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.686+0000 7fb765c92640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb7480a2a10 0x7fb7480a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.686+0000 7fb75f7fe640 1 -- 172.21.15.37:0/2376921540 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fb7580ce7d0 con 0x7fb760101510 2024-03-26T06:26:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb76b3a8740 1 -- start start 2024-03-26T06:26:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ac9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ac9fd0 0x563699aca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699ad3550 0x563699ad5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563699a2e5f0 con 0x563699acad40 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636999d7930 con 0x563699ad3550 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563699a2a690 con 0x563699ac9fd0 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb766493640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ac9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb766493640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ac9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56142/0 (socket says 172.21.15.37:56142) 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb766493640 1 -- 172.21.15.37:0/1935414632 learned_addr learned my addr 172.21.15.37:0/1935414632 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb766c94640 1 --2- 172.21.15.37:0/1935414632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699ad3550 0x563699ad5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb765c92640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb7480a2a10 0x7fb7480a4ed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb754009230 tx=0x7fb754005fd0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb766493640 1 -- 172.21.15.37:0/1935414632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ac9fd0 msgr2=0x563699aca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb766493640 1 --2- 172.21.15.37:0/1935414632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ac9fd0 0x563699aca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb766493640 1 -- 172.21.15.37:0/1935414632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699ad3550 msgr2=0x563699ad5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb766493640 1 --2- 172.21.15.37:0/1935414632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699ad3550 0x563699ad5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb766493640 1 -- 172.21.15.37:0/1935414632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563699743210 con 0x563699acad40 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb766493640 1 --2- 172.21.15.37:0/1935414632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ac9a90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb75809aea0 tx=0x7fb758004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfba226a4b83a40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.687+0000 7fb75cff9640 1 -- 172.21.15.37:0/1935414632 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758020070 con 0x563699acad40 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.688+0000 7fb75cff9640 1 -- 172.21.15.37:0/1935414632 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb758099210 con 0x563699acad40 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.688+0000 7fb75cff9640 1 -- 172.21.15.37:0/1935414632 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75809a360 con 0x563699acad40 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.688+0000 7fb76b3a8740 1 -- 172.21.15.37:0/1935414632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 msgr2=0x563699ac9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.688+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/1935414632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ac9a90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb75809aea0 tx=0x7fb758004800 comp rx=0 tx=0).stop 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.688+0000 7fb76b3a8740 1 -- 172.21.15.37:0/1935414632 shutdown_connections 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.688+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/1935414632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699ad3550 0x563699ad5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.688+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/1935414632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ac9fd0 0x563699aca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.688+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/1935414632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ac9a90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.688+0000 7fb76b3a8740 1 -- 172.21.15.37:0/1935414632 >> 172.21.15.37:0/1935414632 conn(0x563699a915b0 msgr2=0x563699ac4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.688+0000 7fb76b3a8740 1 -- 172.21.15.37:0/1935414632 shutdown_connections 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.688+0000 7fb76b3a8740 1 -- 172.21.15.37:0/1935414632 wait complete. 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb76b3a8740 1 -- start start 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb76b3a8740 1 Processor -- start 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699acad40 0x563699ad0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699ad0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563699ad6280 con 0x563699ac9fd0 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636999ead90 con 0x563699acad40 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563699ad6400 con 0x563699ad3550 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb76b3a8740 10 client.0 _ll_register_callbacks cb 0x563699742a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb766493640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb766493640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56158/0 (socket says 172.21.15.37:56158) 2024-03-26T06:26:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb766493640 1 -- 172.21.15.37:0/2848692261 learned_addr learned my addr 172.21.15.37:0/2848692261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb765c92640 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699acad40 0x563699ad0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb766c94640 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699ad0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb766493640 1 -- 172.21.15.37:0/2848692261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 msgr2=0x563699ad0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb766493640 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699ad0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb766493640 1 -- 172.21.15.37:0/2848692261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699acad40 msgr2=0x563699ad0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb766493640 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699acad40 0x563699ad0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb766493640 1 -- 172.21.15.37:0/2848692261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56369980f620 con 0x563699ac9fd0 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.689+0000 7fb766c94640 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699ad0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb766493640 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad0570 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb7580cc000 tx=0x7fb758009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2ac80d3cc04131 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb745ffb640 1 -- 172.21.15.37:0/2848692261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75807e060 con 0x563699ac9fd0 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb745ffb640 1 -- 172.21.15.37:0/2848692261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fb75809a360 con 0x563699ac9fd0 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb76b3a8740 1 -- 172.21.15.37:0/2848692261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb758099210 con 0x563699ac9fd0 2024-03-26T06:26:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb745ffb640 1 -- 172.21.15.37:0/2848692261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb758020050 con 0x563699ac9fd0 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb76b3a8740 10 client.15777 fetch_fsmap learned FSMap version 32 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb76b3a8740 1 -- 172.21.15.37:0/2848692261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563699ae2540 con 0x563699ac9fd0 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb745ffb640 1 -- 172.21.15.37:0/2848692261 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb758099210 con 0x563699ac9fd0 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb745ffb640 1 -- 172.21.15.37:0/2848692261 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fb7580b2070 con 0x563699ac9fd0 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb76b3a8740 10 client.15777 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb76b3a8740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb76b3a8740 20 client.15777 populate_metadata read hostname 'smithi037' 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563699ae3890 0x563699ae5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb76b3a8740 4 client.15777 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.690+0000 7fb76b3a8740 1 -- 172.21.15.37:0/2848692261 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563699a3bb30 con 0x563699ae3890 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.691+0000 7fb765c92640 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563699ae3890 0x563699ae5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.691+0000 7fb745ffb640 10 client.15777 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.691+0000 7fb765c92640 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563699ae3890 0x563699ae5c80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61b75ace927997ab server_cookie=6856d62218aeff19 in_seq=0 out_seq=0 2024-03-26T06:26:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb745ffb640 1 -- 172.21.15.37:0/2848692261 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563699a3bb30 con 0x563699ae3890 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb745ffb640 10 client.15777 handle_command_reply: tid=0 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 1 client.15777 shutdown 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 1 -- 172.21.15.37:0/2848692261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 msgr2=0x563699ad0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad0570 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb7580cc000 tx=0x7fb758009c30 comp rx=0 tx=0).stop 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 1 -- 172.21.15.37:0/2848692261 shutdown_connections 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563699ae3890 0x563699ae5c80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699ad0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699acad40 0x563699ad0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/2848692261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad0570 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb745ffb640 0 client.15777 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 1 -- 172.21.15.37:0/2848692261 >> 172.21.15.37:0/2848692261 conn(0x563699a915b0 msgr2=0x563699ac6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 1 -- 172.21.15.37:0/2848692261 shutdown_connections 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 1 -- 172.21.15.37:0/2848692261 wait complete. 2024-03-26T06:26:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.692+0000 7fb76b3a8740 20 client.15777 trim_cache size 0 max 16384 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb76b3a8740 1 Processor -- start 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb76b3a8740 1 -- start start 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699acad40 0x563699acd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699acdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601acc00 con 0x563699ac9fd0 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601ae6d0 con 0x563699acad40 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601ad260 con 0x563699ad3550 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb766493640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb766493640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56162/0 (socket says 172.21.15.37:56162) 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb766493640 1 -- 172.21.15.37:0/801704731 learned_addr learned my addr 172.21.15.37:0/801704731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb765c92640 1 --2- 172.21.15.37:0/801704731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699acad40 0x563699acd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb766c94640 1 --2- 172.21.15.37:0/801704731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699acdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb766493640 1 -- 172.21.15.37:0/801704731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 msgr2=0x563699acdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb766493640 1 --2- 172.21.15.37:0/801704731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699acdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb766493640 1 -- 172.21.15.37:0/801704731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699acad40 msgr2=0x563699acd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb766493640 1 --2- 172.21.15.37:0/801704731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699acad40 0x563699acd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb766493640 1 -- 172.21.15.37:0/801704731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563699ac92a0 con 0x563699ac9fd0 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.707+0000 7fb766493640 1 --2- 172.21.15.37:0/801704731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad2ca0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb75809ac10 tx=0x7fb758097230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b6926761b2b238 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb747fff640 1 -- 172.21.15.37:0/801704731 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75807e030 con 0x563699ac9fd0 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb747fff640 1 -- 172.21.15.37:0/801704731 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7580cc1d0 con 0x563699ac9fd0 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb747fff640 1 -- 172.21.15.37:0/801704731 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7580b7070 con 0x563699ac9fd0 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb76b3a8740 1 -- 172.21.15.37:0/801704731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 msgr2=0x563699ad2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/801704731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad2ca0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb75809ac10 tx=0x7fb758097230 comp rx=0 tx=0).stop 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb76b3a8740 1 -- 172.21.15.37:0/801704731 shutdown_connections 2024-03-26T06:26:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/801704731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699acdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/801704731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699acad40 0x563699acd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/801704731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699ac9fd0 0x563699ad2ca0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb76b3a8740 1 -- 172.21.15.37:0/801704731 >> 172.21.15.37:0/801704731 conn(0x563699a915b0 msgr2=0x563699ac47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb76b3a8740 1 -- 172.21.15.37:0/801704731 shutdown_connections 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.708+0000 7fb76b3a8740 1 -- 172.21.15.37:0/801704731 wait complete. 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb76b3a8740 1 -- start start 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb76b3a8740 1 Processor -- start 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699ac9fd0 0x563699ada830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ad8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb76b3a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699ad91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601adfb0 con 0x563699acad40 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601ad950 con 0x563699ac9fd0 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb76b3a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601aaec0 con 0x563699ad3550 2024-03-26T06:26:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb76b3a8740 10 client.0 _ll_register_callbacks cb 0x7fb750009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb765c92640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ad8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb765c92640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ad8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56164/0 (socket says 172.21.15.37:56164) 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb765c92640 1 -- 172.21.15.37:0/608382529 learned_addr learned my addr 172.21.15.37:0/608382529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb766493640 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699ac9fd0 0x563699ada830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb766c94640 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699ad91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb765c92640 1 -- 172.21.15.37:0/608382529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 msgr2=0x563699ad91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb765c92640 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699ad91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb765c92640 1 -- 172.21.15.37:0/608382529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699ac9fd0 msgr2=0x563699ada830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb765c92640 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699ac9fd0 0x563699ada830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb765c92640 1 -- 172.21.15.37:0/608382529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563699adf590 con 0x563699acad40 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb765c92640 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ad8c60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb75402e020 tx=0x7fb75403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96cfaddbde29781 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.709+0000 7fb745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb745ffb640 1 -- 172.21.15.37:0/608382529 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75405f060 con 0x563699acad40 2024-03-26T06:26:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb745ffb640 1 -- 172.21.15.37:0/608382529 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fb754004450 con 0x563699acad40 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb76b3a8740 1 -- 172.21.15.37:0/608382529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7580cc1d0 con 0x563699acad40 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb745ffb640 1 -- 172.21.15.37:0/608382529 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7540046c0 con 0x563699acad40 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb76b3a8740 10 client.15783 fetch_fsmap learned FSMap version 32 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb76b3a8740 1 -- 172.21.15.37:0/608382529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563699b3a730 con 0x563699acad40 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb745ffb640 1 -- 172.21.15.37:0/608382529 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7580cc1d0 con 0x563699acad40 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb745ffb640 1 -- 172.21.15.37:0/608382529 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fb754052130 con 0x563699acad40 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb76b3a8740 10 client.15783 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb76b3a8740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb76b3a8740 20 client.15783 populate_metadata read hostname 'smithi037' 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563699b3af00 0x563699b3d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb76b3a8740 4 client.15783 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb76b3a8740 1 -- 172.21.15.37:0/608382529 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563699a35e80 con 0x563699b3af00 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.710+0000 7fb766493640 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563699b3af00 0x563699b3d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.711+0000 7fb745ffb640 10 client.15783 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.711+0000 7fb766493640 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563699b3af00 0x563699b3d320 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f6574b9de342aa server_cookie=75bffe9d48cbf30e in_seq=0 out_seq=0 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb745ffb640 1 -- 172.21.15.37:0/608382529 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563699a35e80 con 0x563699b3af00 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb745ffb640 10 client.15783 handle_command_reply: tid=0 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 1 client.15783 shutdown 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 1 -- 172.21.15.37:0/608382529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 msgr2=0x563699ad8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ad8c60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb75402e020 tx=0x7fb75403f000 comp rx=0 tx=0).stop 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 1 -- 172.21.15.37:0/608382529 shutdown_connections 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563699b3af00 0x563699b3d320 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563699ad3550 0x563699ad91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb745ffb640 0 client.15783 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563699acad40 0x563699ad8c60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 1 --2- 172.21.15.37:0/608382529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563699ac9fd0 0x563699ada830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 1 -- 172.21.15.37:0/608382529 >> 172.21.15.37:0/608382529 conn(0x563699a915b0 msgr2=0x563699acee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 1 -- 172.21.15.37:0/608382529 shutdown_connections 2024-03-26T06:26:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 1 -- 172.21.15.37:0/608382529 wait complete. 2024-03-26T06:26:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.736+0000 7fb76b3a8740 20 client.15783 trim_cache size 0 max 16384 2024-03-26T06:26:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:26:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:26:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:26:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 -- 172.21.15.37:0/2376921540 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb7480a2a10 msgr2=0x7fb7480a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb7480a2a10 0x7fb7480a4ed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb754009230 tx=0x7fb754005fd0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 -- 172.21.15.37:0/2376921540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb760101510 msgr2=0x7fb76019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb760101510 0x7fb76019d740 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb75800d770 tx=0x7fb75800dc30 comp rx=0 tx=0).stop 2024-03-26T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 -- 172.21.15.37:0/2376921540 shutdown_connections 2024-03-26T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb7480a2a10 0x7fb7480a4ed0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7601978f0 0x7fb760197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7601065e0 0x7fb76019dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 --2- 172.21.15.37:0/2376921540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb760101510 0x7fb76019d740 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 -- 172.21.15.37:0/2376921540 >> 172.21.15.37:0/2376921540 conn(0x7fb7600fcdb0 msgr2=0x7fb7600fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 -- 172.21.15.37:0/2376921540 shutdown_connections 2024-03-26T06:26:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:31.737+0000 7fb7686f7640 1 -- 172.21.15.37:0/2376921540 wait complete. 2024-03-26T06:26:32.493 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78390 2.58 MB/sec execute 7 sec latency 19.901 ms 2024-03-26T06:26:32.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:32 smithi066 ceph-mon[36504]: pgmap v109: 97 pgs: 97 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 824 KiB/s wr, 61 op/s 2024-03-26T06:26:32.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:32 smithi149 ceph-mon[36681]: pgmap v109: 97 pgs: 97 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 824 KiB/s wr, 61 op/s 2024-03-26T06:26:32.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:32 smithi037 ceph-mon[31974]: pgmap v109: 97 pgs: 97 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 824 KiB/s wr, 61 op/s 2024-03-26T06:26:33.493 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78753 2.31 MB/sec execute 8 sec latency 611.012 ms 2024-03-26T06:26:34.493 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78753 2.06 MB/sec execute 9 sec latency 1611.137 ms 2024-03-26T06:26:34.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:34 smithi066 ceph-mon[36504]: pgmap v110: 97 pgs: 97 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 785 KiB/s wr, 44 op/s 2024-03-26T06:26:34.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:34 smithi149 ceph-mon[36681]: pgmap v110: 97 pgs: 97 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 785 KiB/s wr, 44 op/s 2024-03-26T06:26:34.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:34 smithi037 ceph-mon[31974]: pgmap v110: 97 pgs: 97 active+clean; 38 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 785 KiB/s wr, 44 op/s 2024-03-26T06:26:35.493 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78753 1.85 MB/sec execute 10 sec latency 2611.275 ms 2024-03-26T06:26:36.493 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78809 1.69 MB/sec execute 11 sec latency 3419.400 ms 2024-03-26T06:26:36.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:36 smithi066 ceph-mon[36504]: pgmap v111: 97 pgs: 97 active+clean; 40 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2024-03-26T06:26:36.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:36 smithi149 ceph-mon[36681]: pgmap v111: 97 pgs: 97 active+clean; 40 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2024-03-26T06:26:36.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:36 smithi037 ceph-mon[31974]: pgmap v111: 97 pgs: 97 active+clean; 40 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2024-03-26T06:26:37.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78809 1.55 MB/sec execute 12 sec latency 1153.575 ms 2024-03-26T06:26:38.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78809 1.43 MB/sec execute 13 sec latency 2153.680 ms 2024-03-26T06:26:38.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:38 smithi066 ceph-mon[36504]: pgmap v112: 97 pgs: 97 active+clean; 40 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 717 KiB/s wr, 79 op/s 2024-03-26T06:26:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:38 smithi149 ceph-mon[36681]: pgmap v112: 97 pgs: 97 active+clean; 40 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 717 KiB/s wr, 79 op/s 2024-03-26T06:26:38.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:38 smithi037 ceph-mon[31974]: pgmap v112: 97 pgs: 97 active+clean; 40 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 717 KiB/s wr, 79 op/s 2024-03-26T06:26:39.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78809 1.33 MB/sec execute 14 sec latency 3153.767 ms 2024-03-26T06:26:40.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78809 1.24 MB/sec execute 15 sec latency 4153.860 ms 2024-03-26T06:26:40.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:40 smithi066 ceph-mon[36504]: pgmap v113: 97 pgs: 97 active+clean; 40 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 717 KiB/s wr, 79 op/s 2024-03-26T06:26:40.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:40 smithi149 ceph-mon[36681]: pgmap v113: 97 pgs: 97 active+clean; 40 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 717 KiB/s wr, 79 op/s 2024-03-26T06:26:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:40 smithi037 ceph-mon[31974]: pgmap v113: 97 pgs: 97 active+clean; 40 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 717 KiB/s wr, 79 op/s 2024-03-26T06:26:41.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78877 1.17 MB/sec execute 16 sec latency 4962.859 ms 2024-03-26T06:26:42.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78877 1.11 MB/sec execute 17 sec latency 1146.349 ms 2024-03-26T06:26:42.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:42 smithi066 ceph-mon[36504]: pgmap v114: 97 pgs: 97 active+clean; 40 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 997 KiB/s wr, 100 op/s 2024-03-26T06:26:42.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:42 smithi149 ceph-mon[36681]: pgmap v114: 97 pgs: 97 active+clean; 40 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 997 KiB/s wr, 100 op/s 2024-03-26T06:26:42.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:42 smithi037 ceph-mon[31974]: pgmap v114: 97 pgs: 97 active+clean; 40 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 997 KiB/s wr, 100 op/s 2024-03-26T06:26:43.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78877 1.04 MB/sec execute 18 sec latency 2146.416 ms 2024-03-26T06:26:44.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78877 0.99 MB/sec execute 19 sec latency 3146.480 ms 2024-03-26T06:26:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.837+0000 7f8690e2b640 1 -- 172.21.15.37:0/1444339829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f868c100440 msgr2=0x7f868c1008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.837+0000 7f8690e2b640 1 --2- 172.21.15.37:0/1444339829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f868c100440 0x7f868c1008c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8680009900 tx=0x7f868002bc30 comp rx=0 tx=0).stop 2024-03-26T06:26:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.837+0000 7f8690e2b640 1 -- 172.21.15.37:0/1444339829 shutdown_connections 2024-03-26T06:26:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.837+0000 7f8690e2b640 1 --2- 172.21.15.37:0/1444339829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f868c100e00 0x7f868c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.837+0000 7f8690e2b640 1 --2- 172.21.15.37:0/1444339829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f868c100440 0x7f868c1008c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.837+0000 7f8690e2b640 1 --2- 172.21.15.37:0/1444339829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f868c105190 0x7f868c105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.837+0000 7f8690e2b640 1 -- 172.21.15.37:0/1444339829 >> 172.21.15.37:0/1444339829 conn(0x7f868c077040 msgr2=0x7f868c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.837+0000 7f8690e2b640 1 -- 172.21.15.37:0/1444339829 shutdown_connections 2024-03-26T06:26:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.837+0000 7f8690e2b640 1 -- 172.21.15.37:0/1444339829 wait complete. 2024-03-26T06:26:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f8690e2b640 1 Processor -- start 2024-03-26T06:26:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f8690e2b640 1 -- start start 2024-03-26T06:26:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f8690e2b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f868c100440 0x7f868c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f8690e2b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f868c100e00 0x7f868c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f8690e2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f868c105190 0x7f868c1a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f8690e2b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c110650 con 0x7f868c105190 2024-03-26T06:26:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f8690e2b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1104d0 con 0x7f868c100440 2024-03-26T06:26:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f8690e2b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1107d0 con 0x7f868c100e00 2024-03-26T06:26:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f868ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f868c105190 0x7f868c1a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f868ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f868c105190 0x7f868c1a3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50082/0 (socket says 172.21.15.37:50082) 2024-03-26T06:26:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f868ad9d640 1 -- 172.21.15.37:0/1157505824 learned_addr learned my addr 172.21.15.37:0/1157505824 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f868a59c640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f868c100440 0x7f868c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.838+0000 7f8689d9b640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f868c100e00 0x7f868c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.839+0000 7f868ad9d640 1 -- 172.21.15.37:0/1157505824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f868c100e00 msgr2=0x7f868c198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.839+0000 7f868ad9d640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f868c100e00 0x7f868c198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.839+0000 7f868ad9d640 1 -- 172.21.15.37:0/1157505824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f868c100440 msgr2=0x7f868c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.839+0000 7f868ad9d640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f868c100440 0x7f868c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.839+0000 7f868ad9d640 1 -- 172.21.15.37:0/1157505824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c1a3f50 con 0x7f868c105190 2024-03-26T06:26:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.839+0000 7f868ad9d640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f868c105190 0x7f868c1a3a10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f868400e960 tx=0x7f868400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.839+0000 7f867b7fe640 1 -- 172.21.15.37:0/1157505824 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684004d20 con 0x7f868c105190 2024-03-26T06:26:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.839+0000 7f8690e2b640 1 -- 172.21.15.37:0/1157505824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c1a4120 con 0x7f868c105190 2024-03-26T06:26:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.839+0000 7f867b7fe640 1 -- 172.21.15.37:0/1157505824 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8684005020 con 0x7f868c105190 2024-03-26T06:26:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.839+0000 7f867b7fe640 1 -- 172.21.15.37:0/1157505824 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86840116d0 con 0x7f868c105190 2024-03-26T06:26:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.839+0000 7f8690e2b640 1 -- 172.21.15.37:0/1157505824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c1a46c0 con 0x7f868c105190 2024-03-26T06:26:44.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:44 smithi066 ceph-mon[36504]: pgmap v115: 97 pgs: 97 active+clean; 40 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 711 KiB/s wr, 77 op/s 2024-03-26T06:26:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.841+0000 7f867b7fe640 1 -- 172.21.15.37:0/1157505824 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f86840054d0 con 0x7f868c105190 2024-03-26T06:26:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f8693a13740 1 Processor -- start 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f8693a13740 1 -- start start 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f8693a13740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4121d40 0x562de4120a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f8693a13740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de4121450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f8693a13740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de412c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f8693a13740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de40855f0 con 0x562de4120fd0 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f8693a13740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de402e930 con 0x562de4121d40 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f8693a13740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4081690 con 0x562de412a550 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f868a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de4121450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f8689d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4121d40 0x562de4120a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f868a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de4121450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50090/0 (socket says 172.21.15.37:50090) 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f868a59c640 1 -- 172.21.15.37:0/712805611 learned_addr learned my addr 172.21.15.37:0/712805611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f867b7fe640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f86740a2a10 0x7f86740a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f8689d9b640 1 --2- 172.21.15.37:0/712805611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de412c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.842+0000 7f868ad9d640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f86740a2a10 0x7f86740a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.843+0000 7f867b7fe640 1 -- 172.21.15.37:0/1157505824 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f86840cefa0 con 0x7f868c105190 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.843+0000 7f868a59c640 1 -- 172.21.15.37:0/712805611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 msgr2=0x562de412c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.843+0000 7f868a59c640 1 --2- 172.21.15.37:0/712805611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de412c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.843+0000 7f868a59c640 1 -- 172.21.15.37:0/712805611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4121d40 msgr2=0x562de4120a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.843+0000 7f868a59c640 1 --2- 172.21.15.37:0/712805611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4121d40 0x562de4120a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.843+0000 7f868a59c640 1 -- 172.21.15.37:0/712805611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562de3d9a210 con 0x562de4120fd0 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.843+0000 7f868a59c640 1 --2- 172.21.15.37:0/712805611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de4121450 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f867c00bc90 tx=0x7f867c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b608fc2cb7a6b3ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.843+0000 7f868ad9d640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f86740a2a10 0x7f86740a4ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f86840cc7d0 tx=0x7f8684005440 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.843+0000 7f8678ff9640 1 -- 172.21.15.37:0/712805611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c01c480 con 0x562de4120fd0 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.843+0000 7f8678ff9640 1 -- 172.21.15.37:0/712805611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f867c01cb00 con 0x562de4120fd0 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.843+0000 7f8678ff9640 1 -- 172.21.15.37:0/712805611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c011b40 con 0x562de4120fd0 2024-03-26T06:26:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.844+0000 7f8693a13740 1 -- 172.21.15.37:0/712805611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 msgr2=0x562de4121450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.844+0000 7f8693a13740 1 --2- 172.21.15.37:0/712805611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de4121450 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f867c00bc90 tx=0x7f867c0107a0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.844+0000 7f8693a13740 1 -- 172.21.15.37:0/712805611 shutdown_connections 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.844+0000 7f8693a13740 1 --2- 172.21.15.37:0/712805611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de412c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.844+0000 7f8693a13740 1 --2- 172.21.15.37:0/712805611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de4121450 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.844+0000 7f8693a13740 1 --2- 172.21.15.37:0/712805611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4121d40 0x562de4120a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.844+0000 7f8693a13740 1 -- 172.21.15.37:0/712805611 >> 172.21.15.37:0/712805611 conn(0x562de40e85b0 msgr2=0x562de411bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.844+0000 7f8693a13740 1 -- 172.21.15.37:0/712805611 shutdown_connections 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.844+0000 7f8693a13740 1 -- 172.21.15.37:0/712805611 wait complete. 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.844+0000 7f8693a13740 1 -- start start 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.844+0000 7f8693a13740 1 Processor -- start 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8693a13740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de41275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8693a13740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4121d40 0x562de4127b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8693a13740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de4128070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8693a13740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de412d280 con 0x562de4120fd0 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8693a13740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4041d90 con 0x562de4121d40 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8693a13740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de412d400 con 0x562de412a550 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8693a13740 10 client.0 _ll_register_callbacks cb 0x562de3d99a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8689d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de41275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8689d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de41275f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50098/0 (socket says 172.21.15.37:50098) 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8689d9b640 1 -- 172.21.15.37:0/3974717098 learned_addr learned my addr 172.21.15.37:0/3974717098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f868a59c640 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4121d40 0x562de4127b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8689d9b640 1 -- 172.21.15.37:0/3974717098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 msgr2=0x562de4128070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8689d9b640 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de4128070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8689d9b640 1 -- 172.21.15.37:0/3974717098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4121d40 msgr2=0x562de4127b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8689d9b640 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4121d40 0x562de4127b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8689d9b640 1 -- 172.21.15.37:0/3974717098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562de3e66620 con 0x562de4120fd0 2024-03-26T06:26:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f868a59c640 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4121d40 0x562de4127b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8689d9b640 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de41275f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8680009900 tx=0x7f8680009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c067618f04d1b62f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.845+0000 7f8671ffb640 1 -- 172.21.15.37:0/3974717098 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86800037f0 con 0x562de4120fd0 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8671ffb640 1 -- 172.21.15.37:0/3974717098 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8680003990 con 0x562de4120fd0 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8693a13740 1 -- 172.21.15.37:0/3974717098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f867c01cb00 con 0x562de4120fd0 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8671ffb640 1 -- 172.21.15.37:0/3974717098 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8680053050 con 0x562de4120fd0 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8693a13740 10 client.15795 fetch_fsmap learned FSMap version 32 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8693a13740 1 -- 172.21.15.37:0/3974717098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562de41395c0 con 0x562de4120fd0 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8671ffb640 1 -- 172.21.15.37:0/3974717098 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f867c01cb00 con 0x562de4120fd0 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8671ffb640 1 -- 172.21.15.37:0/3974717098 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f8680034440 con 0x562de4120fd0 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8693a13740 10 client.15795 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8693a13740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:26:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8693a13740 20 client.15795 populate_metadata read hostname 'smithi037' 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8693a13740 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562de413aab0 0x562de413cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8693a13740 4 client.15795 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f8693a13740 1 -- 172.21.15.37:0/3974717098 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562de4092b30 con 0x562de413aab0 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.846+0000 7f868a59c640 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562de413aab0 0x562de413cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.847+0000 7f868a59c640 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562de413aab0 0x562de413cf10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a1a5536c55db4e server_cookie=2c2b4c16ec747441 in_seq=0 out_seq=0 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.847+0000 7f8671ffb640 10 client.15795 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.847+0000 7f8671ffb640 1 -- 172.21.15.37:0/3974717098 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f867c026a90 con 0x562de413aab0 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.847+0000 7f8671ffb640 10 client.15795 handle_command_reply: tid=0 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 1 client.15795 shutdown 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 1 -- 172.21.15.37:0/3974717098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 msgr2=0x562de41275f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de41275f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8680009900 tx=0x7f8680009880 comp rx=0 tx=0).stop 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 1 -- 172.21.15.37:0/3974717098 shutdown_connections 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562de413aab0 0x562de413cf10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de4128070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4121d40 0x562de4127b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8671ffb640 0 client.15795 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 1 --2- 172.21.15.37:0/3974717098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4120fd0 0x562de41275f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 1 -- 172.21.15.37:0/3974717098 >> 172.21.15.37:0/3974717098 conn(0x562de40e85b0 msgr2=0x562de411da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 1 -- 172.21.15.37:0/3974717098 shutdown_connections 2024-03-26T06:26:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 1 -- 172.21.15.37:0/3974717098 wait complete. 2024-03-26T06:26:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.848+0000 7f8693a13740 20 client.15795 trim_cache size 0 max 16384 2024-03-26T06:26:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.861+0000 7f8693a13740 1 Processor -- start 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.861+0000 7f8693a13740 1 -- start start 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.861+0000 7f8693a13740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4120fd0 0x562de4129d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.861+0000 7f8693a13740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de4121d40 0x562de4131670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8693a13740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de412a550 0x562de4131bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8693a13740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1acbf0 con 0x562de412a550 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8693a13740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1ae6c0 con 0x562de4120fd0 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8693a13740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1ad250 con 0x562de4121d40 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8689d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de412a550 0x562de4131bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8689d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de412a550 0x562de4131bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50104/0 (socket says 172.21.15.37:50104) 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8689d9b640 1 -- 172.21.15.37:0/2484077910 learned_addr learned my addr 172.21.15.37:0/2484077910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8689d9b640 1 --2- 172.21.15.37:0/2484077910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4120fd0 0x562de4129d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f868a59c640 1 --2- 172.21.15.37:0/2484077910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de4121d40 0x562de4131670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8689d9b640 1 -- 172.21.15.37:0/2484077910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de4121d40 msgr2=0x562de4131670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8689d9b640 1 --2- 172.21.15.37:0/2484077910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de4121d40 0x562de4131670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8689d9b640 1 -- 172.21.15.37:0/2484077910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4120fd0 msgr2=0x562de4129d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8689d9b640 1 --2- 172.21.15.37:0/2484077910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4120fd0 0x562de4129d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8689d9b640 1 -- 172.21.15.37:0/2484077910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562de4142100 con 0x562de412a550 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f868a59c640 1 --2- 172.21.15.37:0/2484077910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de4121d40 0x562de4131670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8689d9b640 1 --2- 172.21.15.37:0/2484077910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de412a550 0x562de4131bb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f868002c750 tx=0x7f8680000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80b48b6b97c17ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8673fff640 1 -- 172.21.15.37:0/2484077910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680034440 con 0x562de412a550 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8673fff640 1 -- 172.21.15.37:0/2484077910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8680047070 con 0x562de412a550 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.862+0000 7f8673fff640 1 -- 172.21.15.37:0/2484077910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868004c650 con 0x562de412a550 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.863+0000 7f8693a13740 1 -- 172.21.15.37:0/2484077910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de412a550 msgr2=0x562de4131bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.863+0000 7f8693a13740 1 --2- 172.21.15.37:0/2484077910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de412a550 0x562de4131bb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f868002c750 tx=0x7f8680000bc0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.863+0000 7f8693a13740 1 -- 172.21.15.37:0/2484077910 shutdown_connections 2024-03-26T06:26:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.863+0000 7f8693a13740 1 --2- 172.21.15.37:0/2484077910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de412a550 0x562de4131bb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.863+0000 7f8693a13740 1 --2- 172.21.15.37:0/2484077910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de4121d40 0x562de4131670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.863+0000 7f8693a13740 1 --2- 172.21.15.37:0/2484077910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4120fd0 0x562de4129d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.863+0000 7f8693a13740 1 -- 172.21.15.37:0/2484077910 >> 172.21.15.37:0/2484077910 conn(0x562de40e85b0 msgr2=0x562de411b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.863+0000 7f8693a13740 1 -- 172.21.15.37:0/2484077910 shutdown_connections 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.863+0000 7f8693a13740 1 -- 172.21.15.37:0/2484077910 wait complete. 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.863+0000 7f8693a13740 1 -- start start 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.863+0000 7f8693a13740 1 Processor -- start 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8693a13740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4120fd0 0x562de41241b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8693a13740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4121d40 0x562de41246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8693a13740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de4124c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8693a13740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1adfa0 con 0x562de4121d40 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8693a13740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1ad940 con 0x562de4120fd0 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8693a13740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1aae90 con 0x562de412a550 2024-03-26T06:26:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8693a13740 10 client.0 _ll_register_callbacks cb 0x7f867c00b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f868a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4121d40 0x562de41246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f868a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4121d40 0x562de41246f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50118/0 (socket says 172.21.15.37:50118) 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f868a59c640 1 -- 172.21.15.37:0/4075761765 learned_addr learned my addr 172.21.15.37:0/4075761765 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8689d9b640 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4120fd0 0x562de41241b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8689d9b640 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de4124c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f868a59c640 1 -- 172.21.15.37:0/4075761765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 msgr2=0x562de4124c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f868a59c640 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de4124c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f868a59c640 1 -- 172.21.15.37:0/4075761765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4120fd0 msgr2=0x562de41241b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f868a59c640 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4120fd0 0x562de41241b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f868a59c640 1 -- 172.21.15.37:0/4075761765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562de4190ff0 con 0x562de4121d40 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8689d9b640 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de4124c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f868a59c640 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4121d40 0x562de41246f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f867c010ca0 tx=0x7f867c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be18682bf05f70e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8671ffb640 1 -- 172.21.15.37:0/4075761765 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c011a50 con 0x562de4121d40 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8671ffb640 1 -- 172.21.15.37:0/4075761765 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f867c011bf0 con 0x562de4121d40 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.864+0000 7f8693a13740 1 -- 172.21.15.37:0/4075761765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8680047070 con 0x562de4121d40 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8671ffb640 1 -- 172.21.15.37:0/4075761765 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f867c011df0 con 0x562de4121d40 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8693a13740 10 client.15801 fetch_fsmap learned FSMap version 32 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8693a13740 1 -- 172.21.15.37:0/4075761765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562de41922d0 con 0x562de4121d40 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8671ffb640 1 -- 172.21.15.37:0/4075761765 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8680047070 con 0x562de4121d40 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8671ffb640 1 -- 172.21.15.37:0/4075761765 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f867c026540 con 0x562de4121d40 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8693a13740 10 client.15801 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8693a13740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8693a13740 20 client.15801 populate_metadata read hostname 'smithi037' 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8693a13740 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562de4192820 0x562de4194c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8693a13740 4 client.15801 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8693a13740 1 -- 172.21.15.37:0/4075761765 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562de408ce80 con 0x562de4192820 2024-03-26T06:26:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.865+0000 7f8689d9b640 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562de4192820 0x562de4194c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.866+0000 7f8689d9b640 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562de4192820 0x562de4194c40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5876f492f18b4cd5 server_cookie=e5b7af5765d7bd8 in_seq=0 out_seq=0 2024-03-26T06:26:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.866+0000 7f8671ffb640 10 client.15801 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8671ffb640 1 -- 172.21.15.37:0/4075761765 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562de408ce80 con 0x562de4192820 2024-03-26T06:26:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8671ffb640 10 client.15801 handle_command_reply: tid=0 2024-03-26T06:26:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8693a13740 1 client.15801 shutdown 2024-03-26T06:26:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8693a13740 1 -- 172.21.15.37:0/4075761765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4121d40 msgr2=0x562de41246f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8693a13740 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4121d40 0x562de41246f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f867c010ca0 tx=0x7f867c00bf20 comp rx=0 tx=0).stop 2024-03-26T06:26:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8693a13740 1 -- 172.21.15.37:0/4075761765 shutdown_connections 2024-03-26T06:26:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8693a13740 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562de4192820 0x562de4194c40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8693a13740 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562de412a550 0x562de4124c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8693a13740 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562de4121d40 0x562de41246f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8671ffb640 0 client.15801 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8693a13740 1 --2- 172.21.15.37:0/4075761765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562de4120fd0 0x562de41241b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8693a13740 1 -- 172.21.15.37:0/4075761765 >> 172.21.15.37:0/4075761765 conn(0x562de40e85b0 msgr2=0x562de4125e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8693a13740 1 -- 172.21.15.37:0/4075761765 shutdown_connections 2024-03-26T06:26:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.891+0000 7f8693a13740 1 -- 172.21.15.37:0/4075761765 wait complete. 2024-03-26T06:26:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8693a13740 20 client.15801 trim_cache size 0 max 16384 2024-03-26T06:26:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:26:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:26:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:26:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 -- 172.21.15.37:0/1157505824 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f86740a2a10 msgr2=0x7f86740a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f86740a2a10 0x7f86740a4ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f86840cc7d0 tx=0x7f8684005440 comp rx=0 tx=0).stop 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 -- 172.21.15.37:0/1157505824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f868c105190 msgr2=0x7f868c1a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f868c105190 0x7f868c1a3a10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f868400e960 tx=0x7f868400ee20 comp rx=0 tx=0).stop 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 -- 172.21.15.37:0/1157505824 shutdown_connections 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f86740a2a10 0x7f86740a4ed0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f868c105190 0x7f868c1a3a10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f868c100e00 0x7f868c198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 --2- 172.21.15.37:0/1157505824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f868c100440 0x7f868c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 -- 172.21.15.37:0/1157505824 >> 172.21.15.37:0/1157505824 conn(0x7f868c077040 msgr2=0x7f868c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 -- 172.21.15.37:0/1157505824 shutdown_connections 2024-03-26T06:26:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:44.892+0000 7f8690e2b640 1 -- 172.21.15.37:0/1157505824 wait complete. 2024-03-26T06:26:44.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:44 smithi149 ceph-mon[36681]: pgmap v115: 97 pgs: 97 active+clean; 40 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 711 KiB/s wr, 77 op/s 2024-03-26T06:26:44.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:44 smithi037 ceph-mon[31974]: pgmap v115: 97 pgs: 97 active+clean; 40 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 711 KiB/s wr, 77 op/s 2024-03-26T06:26:45.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78877 0.94 MB/sec execute 20 sec latency 4146.577 ms 2024-03-26T06:26:46.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 79250 0.97 MB/sec execute 21 sec latency 4954.915 ms 2024-03-26T06:26:46.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:46 smithi066 ceph-mon[36504]: pgmap v116: 97 pgs: 97 active+clean; 37 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 744 KiB/s wr, 91 op/s 2024-03-26T06:26:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:46 smithi149 ceph-mon[36681]: pgmap v116: 97 pgs: 97 active+clean; 37 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 744 KiB/s wr, 91 op/s 2024-03-26T06:26:46.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:46 smithi037 ceph-mon[31974]: pgmap v116: 97 pgs: 97 active+clean; 37 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 744 KiB/s wr, 91 op/s 2024-03-26T06:26:47.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 81910 1.69 MB/sec execute 22 sec latency 10.270 ms 2024-03-26T06:26:48.494 INFO:tasks.workunit.client.0.smithi037.stdout: 1 82426 1.65 MB/sec execute 23 sec latency 522.012 ms 2024-03-26T06:26:48.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:48 smithi066 ceph-mon[36504]: pgmap v117: 97 pgs: 97 active+clean; 37 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 313 KiB/s wr, 35 op/s 2024-03-26T06:26:48.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:48 smithi149 ceph-mon[36681]: pgmap v117: 97 pgs: 97 active+clean; 37 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 313 KiB/s wr, 35 op/s 2024-03-26T06:26:48.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:48 smithi037 ceph-mon[31974]: pgmap v117: 97 pgs: 97 active+clean; 37 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 313 KiB/s wr, 35 op/s 2024-03-26T06:26:49.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 82426 1.58 MB/sec execute 24 sec latency 1522.057 ms 2024-03-26T06:26:50.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 82426 1.52 MB/sec execute 25 sec latency 2522.119 ms 2024-03-26T06:26:50.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:50 smithi066 ceph-mon[36504]: pgmap v118: 97 pgs: 97 active+clean; 37 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 313 KiB/s wr, 35 op/s 2024-03-26T06:26:50.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:50 smithi149 ceph-mon[36681]: pgmap v118: 97 pgs: 97 active+clean; 37 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 313 KiB/s wr, 35 op/s 2024-03-26T06:26:50.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:50 smithi037 ceph-mon[31974]: pgmap v118: 97 pgs: 97 active+clean; 37 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 313 KiB/s wr, 35 op/s 2024-03-26T06:26:51.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 82778 1.52 MB/sec execute 26 sec latency 3330.212 ms 2024-03-26T06:26:52.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 85425 2.09 MB/sec execute 27 sec latency 13.701 ms 2024-03-26T06:26:52.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:52 smithi066 ceph-mon[36504]: pgmap v119: 97 pgs: 97 active+clean; 35 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 550 KiB/s wr, 83 op/s 2024-03-26T06:26:52.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:52 smithi149 ceph-mon[36681]: pgmap v119: 97 pgs: 97 active+clean; 35 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 550 KiB/s wr, 83 op/s 2024-03-26T06:26:52.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:52 smithi037 ceph-mon[31974]: pgmap v119: 97 pgs: 97 active+clean; 35 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 550 KiB/s wr, 83 op/s 2024-03-26T06:26:53.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 85899 2.03 MB/sec execute 28 sec latency 523.120 ms 2024-03-26T06:26:54.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 85899 1.96 MB/sec execute 29 sec latency 1523.173 ms 2024-03-26T06:26:54.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:54 smithi066 ceph-mon[36504]: pgmap v120: 97 pgs: 97 active+clean; 35 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 271 KiB/s wr, 63 op/s 2024-03-26T06:26:54.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:54 smithi149 ceph-mon[36681]: pgmap v120: 97 pgs: 97 active+clean; 35 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 271 KiB/s wr, 63 op/s 2024-03-26T06:26:54.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:54 smithi037 ceph-mon[31974]: pgmap v120: 97 pgs: 97 active+clean; 35 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 271 KiB/s wr, 63 op/s 2024-03-26T06:26:55.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 85899 1.90 MB/sec execute 30 sec latency 2523.276 ms 2024-03-26T06:26:55.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:55 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:26:55.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:55 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:26:55.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:55 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:26:56.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 85967 1.84 MB/sec execute 31 sec latency 3331.302 ms 2024-03-26T06:26:56.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:56 smithi066 ceph-mon[36504]: pgmap v121: 97 pgs: 97 active+clean; 37 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 501 KiB/s wr, 107 op/s 2024-03-26T06:26:56.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:56 smithi149 ceph-mon[36681]: pgmap v121: 97 pgs: 97 active+clean; 37 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 501 KiB/s wr, 107 op/s 2024-03-26T06:26:56.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:56 smithi037 ceph-mon[31974]: pgmap v121: 97 pgs: 97 active+clean; 37 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 501 KiB/s wr, 107 op/s 2024-03-26T06:26:57.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 85967 1.78 MB/sec execute 32 sec latency 1148.814 ms 2024-03-26T06:26:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.990+0000 7ff5f21cd640 1 -- 172.21.15.37:0/623810616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5ec101510 msgr2=0x7ff5ec1018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.990+0000 7ff5f21cd640 1 --2- 172.21.15.37:0/623810616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5ec101510 0x7ff5ec1018f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009900 tx=0x7ff5dc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:26:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.990+0000 7ff5f21cd640 1 -- 172.21.15.37:0/623810616 shutdown_connections 2024-03-26T06:26:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.990+0000 7ff5f21cd640 1 --2- 172.21.15.37:0/623810616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec1065e0 0x7ff5ec10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.990+0000 7ff5f21cd640 1 --2- 172.21.15.37:0/623810616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ec101ec0 0x7ff5ec105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.990+0000 7ff5f21cd640 1 --2- 172.21.15.37:0/623810616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5ec101510 0x7ff5ec1018f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.990+0000 7ff5f21cd640 1 -- 172.21.15.37:0/623810616 >> 172.21.15.37:0/623810616 conn(0x7ff5ec0fcdb0 msgr2=0x7ff5ec0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.990+0000 7ff5f21cd640 1 -- 172.21.15.37:0/623810616 shutdown_connections 2024-03-26T06:26:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.990+0000 7ff5f21cd640 1 -- 172.21.15.37:0/623810616 wait complete. 2024-03-26T06:26:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.991+0000 7ff5f21cd640 1 Processor -- start 2024-03-26T06:26:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.991+0000 7ff5f21cd640 1 -- start start 2024-03-26T06:26:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.991+0000 7ff5f21cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ec101510 0x7ff5ec19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.991+0000 7ff5f21cd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec1065e0 0x7ff5ec19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.991+0000 7ff5f21cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5ec199a90 0x7ff5ec199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.991+0000 7ff5f21cd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec111670 con 0x7ff5ec199a90 2024-03-26T06:26:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.991+0000 7ff5f21cd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec1114f0 con 0x7ff5ec101510 2024-03-26T06:26:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.991+0000 7ff5f21cd640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec1117f0 con 0x7ff5ec1065e0 2024-03-26T06:26:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.991+0000 7ff5ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5ec199a90 0x7ff5ec199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5ec199a90 0x7ff5ec199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49632/0 (socket says 172.21.15.37:49632) 2024-03-26T06:26:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5ebfff640 1 -- 172.21.15.37:0/3197966196 learned_addr learned my addr 172.21.15.37:0/3197966196 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec1065e0 0x7ff5ec19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ec101510 0x7ff5ec19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5ebfff640 1 -- 172.21.15.37:0/3197966196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec1065e0 msgr2=0x7ff5ec19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5ebfff640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec1065e0 0x7ff5ec19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5ebfff640 1 -- 172.21.15.37:0/3197966196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ec101510 msgr2=0x7ff5ec19f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5ebfff640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ec101510 0x7ff5ec19f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5ebfff640 1 -- 172.21.15.37:0/3197966196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec19a5b0 con 0x7ff5ec199a90 2024-03-26T06:26:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5ebfff640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5ec199a90 0x7ff5ec199f40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400cce0 tx=0x7ff5e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5e8ff9640 1 -- 172.21.15.37:0/3197966196 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400f450 con 0x7ff5ec199a90 2024-03-26T06:26:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5f21cd640 1 -- 172.21.15.37:0/3197966196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ec19a8a0 con 0x7ff5ec199a90 2024-03-26T06:26:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5e8ff9640 1 -- 172.21.15.37:0/3197966196 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5e400fad0 con 0x7ff5ec199a90 2024-03-26T06:26:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5e8ff9640 1 -- 172.21.15.37:0/3197966196 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4018c10 con 0x7ff5ec199a90 2024-03-26T06:26:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.992+0000 7ff5f21cd640 1 -- 172.21.15.37:0/3197966196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ec1a6aa0 con 0x7ff5ec199a90 2024-03-26T06:26:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.995+0000 7ff5e8ff9640 1 -- 172.21.15.37:0/3197966196 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff5e400f5f0 con 0x7ff5ec199a90 2024-03-26T06:26:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.995+0000 7ff5f4dae740 1 Processor -- start 2024-03-26T06:26:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.995+0000 7ff5f4dae740 1 -- start start 2024-03-26T06:26:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.995+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d19d40 0x563122d18a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.995+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d19450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.995+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d22550 0x563122d24940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.995+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122c7d5f0 con 0x563122d18fd0 2024-03-26T06:26:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.995+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122c26930 con 0x563122d22550 2024-03-26T06:26:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.995+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122c79690 con 0x563122d19d40 2024-03-26T06:26:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.995+0000 7ff5e8ff9640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff5d40a2a10 0x7ff5d40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.995+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d19450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d19450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49640/0 (socket says 172.21.15.37:49640) 2024-03-26T06:26:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5eb7fe640 1 -- 172.21.15.37:0/3831002246 learned_addr learned my addr 172.21.15.37:0/3831002246 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/3831002246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d19d40 0x563122d18a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5ebfff640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff5d40a2a10 0x7ff5d40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/3831002246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d22550 0x563122d24940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5e8ff9640 1 -- 172.21.15.37:0/3197966196 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff5e40cf3f0 con 0x7ff5ec199a90 2024-03-26T06:26:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5eb7fe640 1 -- 172.21.15.37:0/3831002246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d19d40 msgr2=0x563122d18a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/3831002246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d19d40 0x563122d18a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5eb7fe640 1 -- 172.21.15.37:0/3831002246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d22550 msgr2=0x563122d24940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/3831002246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d22550 0x563122d24940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5eb7fe640 1 -- 172.21.15.37:0/3831002246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563122992210 con 0x563122d18fd0 2024-03-26T06:26:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5ebfff640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff5d40a2a10 0x7ff5d40a4ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40cd170 tx=0x7ff5e406d6b0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/3831002246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d19450 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc02c170 tx=0x7ff5dc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3db3102678aaec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5d1ffb640 1 -- 172.21.15.37:0/3831002246 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc042070 con 0x563122d18fd0 2024-03-26T06:26:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5d1ffb640 1 -- 172.21.15.37:0/3831002246 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5dc02ca80 con 0x563122d18fd0 2024-03-26T06:26:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.996+0000 7ff5d1ffb640 1 -- 172.21.15.37:0/3831002246 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc004c90 con 0x563122d18fd0 2024-03-26T06:26:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.997+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3831002246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 msgr2=0x563122d19450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.997+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/3831002246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d19450 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc02c170 tx=0x7ff5dc005c00 comp rx=0 tx=0).stop 2024-03-26T06:26:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.997+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3831002246 shutdown_connections 2024-03-26T06:26:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.997+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/3831002246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d22550 0x563122d24940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.997+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/3831002246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d19450 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.997+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/3831002246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d19d40 0x563122d18a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.997+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3831002246 >> 172.21.15.37:0/3831002246 conn(0x563122ce05b0 msgr2=0x563122d13cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.997+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3831002246 shutdown_connections 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.997+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3831002246 wait complete. 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.997+0000 7ff5f4dae740 1 -- start start 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5f4dae740 1 Processor -- start 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d1f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 0x563122d1fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d22550 0x563122d20070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122d25220 con 0x563122d18fd0 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122c39d90 con 0x563122d19d40 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563122d253a0 con 0x563122d22550 2024-03-26T06:26:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5f4dae740 10 client.0 _ll_register_callbacks cb 0x563122991a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5eaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d1f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5eaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d1f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49644/0 (socket says 172.21.15.37:49644) 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5eaffd640 1 -- 172.21.15.37:0/3992381733 learned_addr learned my addr 172.21.15.37:0/3992381733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 0x563122d1fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5eaffd640 1 -- 172.21.15.37:0/3992381733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d22550 msgr2=0x563122d20070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d22550 0x563122d20070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5eaffd640 1 -- 172.21.15.37:0/3992381733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 msgr2=0x563122d1fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 0x563122d1fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5eaffd640 1 -- 172.21.15.37:0/3992381733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563122a5e620 con 0x563122d18fd0 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.998+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 0x563122d1fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d1f5f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec19b280 tx=0x7ff5e00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23af80011f66399 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5caffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5caffd640 1 -- 172.21.15.37:0/3992381733 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0005020 con 0x563122d18fd0 2024-03-26T06:26:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5caffd640 1 -- 172.21.15.37:0/3992381733 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff5e00051c0 con 0x563122d18fd0 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3992381733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5dc02ca80 con 0x563122d18fd0 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5caffd640 1 -- 172.21.15.37:0/3992381733 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5e00053c0 con 0x563122d18fd0 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5f4dae740 10 client.15813 fetch_fsmap learned FSMap version 33 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3992381733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563122d315c0 con 0x563122d18fd0 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5caffd640 1 -- 172.21.15.37:0/3992381733 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5dc02ca80 con 0x563122d18fd0 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5caffd640 1 -- 172.21.15.37:0/3992381733 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff5e0012950 con 0x563122d18fd0 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5f4dae740 10 client.15813 fetch_fsmap finished waiting for FSMap version 33 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5f4dae740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5f4dae740 20 client.15813 populate_metadata read hostname 'smithi037' 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:57.999+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563122d32a30 0x563122d34e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.000+0000 7ff5f4dae740 4 client.15813 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:26:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.000+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3992381733 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563122c8ab30 con 0x563122d32a30 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.000+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563122d32a30 0x563122d34e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.000+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563122d32a30 0x563122d34e20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f14de3f69c48774 server_cookie=d2dbb8e4e32409f9 in_seq=0 out_seq=0 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.000+0000 7ff5caffd640 10 client.15813 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5caffd640 1 -- 172.21.15.37:0/3992381733 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563122c8ab30 con 0x563122d32a30 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5caffd640 10 client.15813 handle_command_reply: tid=0 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5f4dae740 1 client.15813 shutdown 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3992381733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 msgr2=0x563122d1f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d1f5f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec19b280 tx=0x7ff5e00039c0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3992381733 shutdown_connections 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563122d32a30 0x563122d34e20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d22550 0x563122d20070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 0x563122d1fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5caffd640 0 client.15813 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/3992381733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d18fd0 0x563122d1f5f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3992381733 >> 172.21.15.37:0/3992381733 conn(0x563122ce05b0 msgr2=0x563122d15a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3992381733 shutdown_connections 2024-03-26T06:26:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.001+0000 7ff5f4dae740 1 -- 172.21.15.37:0/3992381733 wait complete. 2024-03-26T06:26:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.002+0000 7ff5f4dae740 20 client.15813 trim_cache size 0 max 16384 2024-03-26T06:26:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5f4dae740 1 Processor -- start 2024-03-26T06:26:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5f4dae740 1 -- start start 2024-03-26T06:26:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d18fd0 0x563122d21d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 0x563122d29610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d22550 0x563122d29b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec1aeca0 con 0x563122d22550 2024-03-26T06:26:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec1b0740 con 0x563122d19d40 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec1af300 con 0x563122d18fd0 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5eaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d22550 0x563122d29b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5eaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d22550 0x563122d29b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49652/0 (socket says 172.21.15.37:49652) 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5eaffd640 1 -- 172.21.15.37:0/1335232870 learned_addr learned my addr 172.21.15.37:0/1335232870 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/1335232870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d18fd0 0x563122d21d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/1335232870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 0x563122d29610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.015+0000 7ff5eaffd640 1 -- 172.21.15.37:0/1335232870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d18fd0 msgr2=0x563122d21d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/1335232870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d18fd0 0x563122d21d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5eaffd640 1 -- 172.21.15.37:0/1335232870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 msgr2=0x563122d29610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/1335232870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 0x563122d29610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5eaffd640 1 -- 172.21.15.37:0/1335232870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563122d182a0 con 0x563122d22550 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/1335232870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 0x563122d29610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:26:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/1335232870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d22550 0x563122d29b50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff5e001f4c0 tx=0x7ff5e000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af020c24a2e7403 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5d17fa640 1 -- 172.21.15.37:0/1335232870 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0005850 con 0x563122d22550 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5d17fa640 1 -- 172.21.15.37:0/1335232870 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5e002f070 con 0x563122d22550 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5d17fa640 1 -- 172.21.15.37:0/1335232870 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0004730 con 0x563122d22550 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5f4dae740 1 -- 172.21.15.37:0/1335232870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d22550 msgr2=0x563122d29b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/1335232870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d22550 0x563122d29b50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff5e001f4c0 tx=0x7ff5e000bf30 comp rx=0 tx=0).stop 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5f4dae740 1 -- 172.21.15.37:0/1335232870 shutdown_connections 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.016+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/1335232870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d22550 0x563122d29b50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/1335232870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d19d40 0x563122d29610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/1335232870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d18fd0 0x563122d21d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 -- 172.21.15.37:0/1335232870 >> 172.21.15.37:0/1335232870 conn(0x563122ce05b0 msgr2=0x563122d137f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 -- 172.21.15.37:0/1335232870 shutdown_connections 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 -- 172.21.15.37:0/1335232870 wait complete. 2024-03-26T06:26:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 -- start start 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 Processor -- start 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d18fd0 0x563122d1c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d19d40 0x563122d1c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d22550 0x563122d1cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec1b0050 con 0x563122d19d40 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec1af9f0 con 0x563122d22550 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec1acf40 con 0x563122d18fd0 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5f4dae740 10 client.0 _ll_register_callbacks cb 0x7ff5dc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.017+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d19d40 0x563122d1c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d19d40 0x563122d1c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49656/0 (socket says 172.21.15.37:49656) 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5eb7fe640 1 -- 172.21.15.37:0/303441630 learned_addr learned my addr 172.21.15.37:0/303441630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d18fd0 0x563122d1c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d22550 0x563122d1cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5eb7fe640 1 -- 172.21.15.37:0/303441630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d18fd0 msgr2=0x563122d1c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d18fd0 0x563122d1c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5eb7fe640 1 -- 172.21.15.37:0/303441630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d22550 msgr2=0x563122d1cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d22550 0x563122d1cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5eb7fe640 1 -- 172.21.15.37:0/303441630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563122d88f00 con 0x563122d19d40 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5eb7fe640 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d19d40 0x563122d1c750 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc02c170 tx=0x7ff5dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f1365e908c6c00 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5caffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5caffd640 1 -- 172.21.15.37:0/303441630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc051020 con 0x563122d19d40 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5caffd640 1 -- 172.21.15.37:0/303441630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff5dc02c900 con 0x563122d19d40 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5f4dae740 1 -- 172.21.15.37:0/303441630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5e002f070 con 0x563122d19d40 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5caffd640 1 -- 172.21.15.37:0/303441630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5dc05c050 con 0x563122d19d40 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5f4dae740 10 client.15819 fetch_fsmap learned FSMap version 33 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.018+0000 7ff5f4dae740 1 -- 172.21.15.37:0/303441630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563122d8a1f0 con 0x563122d19d40 2024-03-26T06:26:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.019+0000 7ff5caffd640 1 -- 172.21.15.37:0/303441630 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5e002f070 con 0x563122d19d40 2024-03-26T06:26:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.019+0000 7ff5caffd640 1 -- 172.21.15.37:0/303441630 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff5dc04d2b0 con 0x563122d19d40 2024-03-26T06:26:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.019+0000 7ff5f4dae740 10 client.15819 fetch_fsmap finished waiting for FSMap version 33 2024-03-26T06:26:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.019+0000 7ff5f4dae740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:26:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.019+0000 7ff5f4dae740 20 client.15819 populate_metadata read hostname 'smithi037' 2024-03-26T06:26:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.019+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563122d8a670 0x563122d8ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:26:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.019+0000 7ff5f4dae740 4 client.15819 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:26:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.019+0000 7ff5f4dae740 1 -- 172.21.15.37:0/303441630 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563122c84e80 con 0x563122d8a670 2024-03-26T06:26:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.019+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563122d8a670 0x563122d8ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:26:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.019+0000 7ff5eaffd640 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563122d8a670 0x563122d8ca90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1862fb3db355742 server_cookie=6b1653e66164ea38 in_seq=0 out_seq=0 2024-03-26T06:26:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.019+0000 7ff5caffd640 10 client.15819 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.053+0000 7ff5caffd640 1 -- 172.21.15.37:0/303441630 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563122c84e80 con 0x563122d8a670 2024-03-26T06:26:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.053+0000 7ff5caffd640 10 client.15819 handle_command_reply: tid=0 2024-03-26T06:26:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.053+0000 7ff5f4dae740 1 client.15819 shutdown 2024-03-26T06:26:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f4dae740 1 -- 172.21.15.37:0/303441630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d19d40 msgr2=0x563122d1c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d19d40 0x563122d1c750 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc02c170 tx=0x7ff5dc004ad0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f4dae740 1 -- 172.21.15.37:0/303441630 shutdown_connections 2024-03-26T06:26:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563122d8a670 0x563122d8ca90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563122d22550 0x563122d1cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563122d19d40 0x563122d1c750 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5caffd640 0 client.15819 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:26:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f4dae740 1 --2- 172.21.15.37:0/303441630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563122d18fd0 0x563122d1c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f4dae740 1 -- 172.21.15.37:0/303441630 >> 172.21.15.37:0/303441630 conn(0x563122ce05b0 msgr2=0x563122d1de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f4dae740 1 -- 172.21.15.37:0/303441630 shutdown_connections 2024-03-26T06:26:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f4dae740 1 -- 172.21.15.37:0/303441630 wait complete. 2024-03-26T06:26:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f4dae740 20 client.15819 trim_cache size 0 max 16384 2024-03-26T06:26:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:26:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:26:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:26:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:26:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f21cd640 1 -- 172.21.15.37:0/3197966196 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff5d40a2a10 msgr2=0x7ff5d40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f21cd640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff5d40a2a10 0x7ff5d40a4ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40cd170 tx=0x7ff5e406d6b0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f21cd640 1 -- 172.21.15.37:0/3197966196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5ec199a90 msgr2=0x7ff5ec199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:26:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f21cd640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5ec199a90 0x7ff5ec199f40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400cce0 tx=0x7ff5e400a620 comp rx=0 tx=0).stop 2024-03-26T06:26:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f21cd640 1 -- 172.21.15.37:0/3197966196 shutdown_connections 2024-03-26T06:26:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.054+0000 7ff5f21cd640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff5d40a2a10 0x7ff5d40a4ed0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.055+0000 7ff5f21cd640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5ec199a90 0x7ff5ec199f40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.055+0000 7ff5f21cd640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec1065e0 0x7ff5ec19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.055+0000 7ff5f21cd640 1 --2- 172.21.15.37:0/3197966196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ec101510 0x7ff5ec19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:26:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.055+0000 7ff5f21cd640 1 -- 172.21.15.37:0/3197966196 >> 172.21.15.37:0/3197966196 conn(0x7ff5ec0fcdb0 msgr2=0x7ff5ec10c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:26:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.055+0000 7ff5f21cd640 1 -- 172.21.15.37:0/3197966196 shutdown_connections 2024-03-26T06:26:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:26:58.055+0000 7ff5f21cd640 1 -- 172.21.15.37:0/3197966196 wait complete. 2024-03-26T06:26:58.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 85967 1.73 MB/sec execute 33 sec latency 2148.875 ms 2024-03-26T06:26:58.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:26:58 smithi066 ceph-mon[36504]: pgmap v122: 97 pgs: 97 active+clean; 37 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 468 KiB/s wr, 92 op/s 2024-03-26T06:26:58.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:26:58 smithi149 ceph-mon[36681]: pgmap v122: 97 pgs: 97 active+clean; 37 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 468 KiB/s wr, 92 op/s 2024-03-26T06:26:58.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:26:58 smithi037 ceph-mon[31974]: pgmap v122: 97 pgs: 97 active+clean; 37 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 468 KiB/s wr, 92 op/s 2024-03-26T06:26:59.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 85967 1.68 MB/sec execute 34 sec latency 3148.971 ms 2024-03-26T06:27:00.495 INFO:tasks.workunit.client.0.smithi037.stdout: 1 85967 1.63 MB/sec execute 35 sec latency 4149.046 ms 2024-03-26T06:27:00.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:00 smithi066 ceph-mon[36504]: pgmap v123: 97 pgs: 97 active+clean; 37 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 468 KiB/s wr, 92 op/s 2024-03-26T06:27:00.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:00 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:27:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:00 smithi149 ceph-mon[36681]: pgmap v123: 97 pgs: 97 active+clean; 37 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 468 KiB/s wr, 92 op/s 2024-03-26T06:27:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:00 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:27:00.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:00 smithi037 ceph-mon[31974]: pgmap v123: 97 pgs: 97 active+clean; 37 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 468 KiB/s wr, 92 op/s 2024-03-26T06:27:00.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:00 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:27:01.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 87773 1.80 MB/sec execute 36 sec latency 4162.888 ms 2024-03-26T06:27:02.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 89447 2.05 MB/sec execute 37 sec latency 18.710 ms 2024-03-26T06:27:02.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:02 smithi066 ceph-mon[36504]: pgmap v124: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 508 KiB/s wr, 113 op/s 2024-03-26T06:27:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:02 smithi149 ceph-mon[36681]: pgmap v124: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 508 KiB/s wr, 113 op/s 2024-03-26T06:27:02.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:02 smithi037 ceph-mon[31974]: pgmap v124: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 508 KiB/s wr, 113 op/s 2024-03-26T06:27:03.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 89448 2.00 MB/sec execute 38 sec latency 993.706 ms 2024-03-26T06:27:04.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 89448 1.95 MB/sec execute 39 sec latency 1993.817 ms 2024-03-26T06:27:04.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:04 smithi066 ceph-mon[36504]: pgmap v125: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 270 KiB/s wr, 65 op/s 2024-03-26T06:27:04.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:04 smithi149 ceph-mon[36681]: pgmap v125: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 270 KiB/s wr, 65 op/s 2024-03-26T06:27:04.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:04 smithi037 ceph-mon[31974]: pgmap v125: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 270 KiB/s wr, 65 op/s 2024-03-26T06:27:05.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 89448 1.90 MB/sec execute 40 sec latency 2993.907 ms 2024-03-26T06:27:06.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 89794 1.89 MB/sec execute 41 sec latency 3802.550 ms 2024-03-26T06:27:06.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:06 smithi066 ceph-mon[36504]: pgmap v126: 97 pgs: 97 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 629 KiB/s wr, 143 op/s 2024-03-26T06:27:06.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:06 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:27:06.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:06 smithi149 ceph-mon[36681]: pgmap v126: 97 pgs: 97 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 629 KiB/s wr, 143 op/s 2024-03-26T06:27:06.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:06 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:27:06.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:06 smithi037 ceph-mon[31974]: pgmap v126: 97 pgs: 97 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 629 KiB/s wr, 143 op/s 2024-03-26T06:27:06.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:06 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:27:07.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 92432 2.23 MB/sec execute 42 sec latency 14.954 ms 2024-03-26T06:27:07.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:07 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:27:07.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:07 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:27:07.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:07 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:27:08.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 93840 2.33 MB/sec execute 43 sec latency 15.063 ms 2024-03-26T06:27:08.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:08 smithi066 ceph-mon[36504]: pgmap v127: 97 pgs: 97 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 398 KiB/s wr, 99 op/s 2024-03-26T06:27:08.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:08 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:27:08.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:08 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:27:08.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:08 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:27:08.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:08 smithi149 ceph-mon[36681]: pgmap v127: 97 pgs: 97 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 398 KiB/s wr, 99 op/s 2024-03-26T06:27:08.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:08 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:27:08.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:08 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:27:08.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:08 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:27:08.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:08 smithi037 ceph-mon[31974]: pgmap v127: 97 pgs: 97 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 398 KiB/s wr, 99 op/s 2024-03-26T06:27:08.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:08 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:27:08.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:08 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:27:08.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:08 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:27:09.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 96132 2.58 MB/sec execute 44 sec latency 11.056 ms 2024-03-26T06:27:10.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 96483 2.54 MB/sec execute 45 sec latency 646.708 ms 2024-03-26T06:27:10.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:10 smithi066 ceph-mon[36504]: pgmap v128: 97 pgs: 97 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 398 KiB/s wr, 99 op/s 2024-03-26T06:27:10.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:10 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:27:10.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:10 smithi149 ceph-mon[36681]: pgmap v128: 97 pgs: 97 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 398 KiB/s wr, 99 op/s 2024-03-26T06:27:10.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:10 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:27:10.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:10 smithi037 ceph-mon[31974]: pgmap v128: 97 pgs: 97 active+clean; 40 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 398 KiB/s wr, 99 op/s 2024-03-26T06:27:10.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:10 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:27:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.151+0000 7f0d34f29640 1 -- 172.21.15.37:0/3457807733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d30105170 msgr2=0x7f0d30105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.151+0000 7f0d34f29640 1 --2- 172.21.15.37:0/3457807733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d30105170 0x7f0d30105550 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009ca0 tx=0x7f0d2002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.152+0000 7f0d34f29640 1 -- 172.21.15.37:0/3457807733 shutdown_connections 2024-03-26T06:27:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.152+0000 7f0d34f29640 1 --2- 172.21.15.37:0/3457807733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d30100de0 0x7f0d3010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.152+0000 7f0d34f29640 1 --2- 172.21.15.37:0/3457807733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d30100440 0x7f0d301008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.152+0000 7f0d34f29640 1 --2- 172.21.15.37:0/3457807733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d30105170 0x7f0d30105550 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.152+0000 7f0d34f29640 1 -- 172.21.15.37:0/3457807733 >> 172.21.15.37:0/3457807733 conn(0x7f0d30076fc0 msgr2=0x7f0d300773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.152+0000 7f0d34f29640 1 -- 172.21.15.37:0/3457807733 shutdown_connections 2024-03-26T06:27:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.152+0000 7f0d34f29640 1 -- 172.21.15.37:0/3457807733 wait complete. 2024-03-26T06:27:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.152+0000 7f0d34f29640 1 Processor -- start 2024-03-26T06:27:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.152+0000 7f0d34f29640 1 -- start start 2024-03-26T06:27:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d34f29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d30100440 0x7f0d30198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d34f29640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d30100de0 0x7f0d30198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d34f29640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d30105170 0x7f0d301a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d34f29640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30110640 con 0x7f0d30100440 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d34f29640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301104c0 con 0x7f0d30100de0 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d34f29640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301107c0 con 0x7f0d30105170 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d30100440 0x7f0d30198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d30100440 0x7f0d30198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60140/0 (socket says 172.21.15.37:60140) 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2e59c640 1 -- 172.21.15.37:0/1965219124 learned_addr learned my addr 172.21.15.37:0/1965219124 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2ed9d640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d30105170 0x7f0d301a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d30100de0 0x7f0d30198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2e59c640 1 -- 172.21.15.37:0/1965219124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d30105170 msgr2=0x7f0d301a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2e59c640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d30105170 0x7f0d301a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2e59c640 1 -- 172.21.15.37:0/1965219124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d30100de0 msgr2=0x7f0d30198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2e59c640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d30100de0 0x7f0d30198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2e59c640 1 -- 172.21.15.37:0/1965219124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d301a3f00 con 0x7f0d30100440 2024-03-26T06:27:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2ed9d640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d30105170 0x7f0d301a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:27:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.153+0000 7f0d2e59c640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d30100440 0x7f0d30198940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0d20005e30 tx=0x7f0d20032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.154+0000 7f0d1f7fe640 1 -- 172.21.15.37:0/1965219124 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20039040 con 0x7f0d30100440 2024-03-26T06:27:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.154+0000 7f0d34f29640 1 -- 172.21.15.37:0/1965219124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d301a40d0 con 0x7f0d30100440 2024-03-26T06:27:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.154+0000 7f0d1f7fe640 1 -- 172.21.15.37:0/1965219124 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d20044070 con 0x7f0d30100440 2024-03-26T06:27:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.154+0000 7f0d1f7fe640 1 -- 172.21.15.37:0/1965219124 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d200042a0 con 0x7f0d30100440 2024-03-26T06:27:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.154+0000 7f0d34f29640 1 -- 172.21.15.37:0/1965219124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d301a4590 con 0x7f0d30100440 2024-03-26T06:27:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.156+0000 7f0d1f7fe640 1 -- 172.21.15.37:0/1965219124 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0d20004440 con 0x7f0d30100440 2024-03-26T06:27:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.156+0000 7f0d37be6740 1 Processor -- start 2024-03-26T06:27:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d37be6740 1 -- start start 2024-03-26T06:27:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a01d40 0x55cf56a00a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a00fd0 0x55cf56a01450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a0c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d37be6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf569655f0 con 0x55cf56a0a550 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d1f7fe640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0d180a2a10 0x7f0d180a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d37be6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5690e930 con 0x55cf56a00fd0 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d37be6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf56961690 con 0x55cf56a01d40 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d2ed9d640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0d180a2a10 0x7f0d180a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a0c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d2e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a00fd0 0x55cf56a01450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a0c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60156/0 (socket says 172.21.15.37:60156) 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/3013430087 learned_addr learned my addr 172.21.15.37:0/3013430087 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d2ed9d640 1 --2- 172.21.15.37:0/3013430087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a01d40 0x55cf56a00a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.157+0000 7f0d1f7fe640 1 -- 172.21.15.37:0/1965219124 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0d200f1150 con 0x7f0d30100440 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.158+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/3013430087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a01d40 msgr2=0x55cf56a00a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.158+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/3013430087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a01d40 0x55cf56a00a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.158+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/3013430087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a00fd0 msgr2=0x55cf56a01450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.158+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/3013430087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a00fd0 0x55cf56a01450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.158+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/3013430087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf5667a210 con 0x55cf56a0a550 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.158+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/3013430087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a0c940 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400dc80 tx=0x7f0d2400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2ad4b5db3cf7e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.158+0000 7f0d2ed9d640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0d180a2a10 0x7f0d180a4ed0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d28005d80 tx=0x7f0d28005d10 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.158+0000 7f0d1cff9640 1 -- 172.21.15.37:0/3013430087 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24015070 con 0x55cf56a0a550 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.158+0000 7f0d1cff9640 1 -- 172.21.15.37:0/3013430087 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d2401a940 con 0x55cf56a0a550 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.158+0000 7f0d1cff9640 1 -- 172.21.15.37:0/3013430087 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24011860 con 0x55cf56a0a550 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.159+0000 7f0d37be6740 1 -- 172.21.15.37:0/3013430087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 msgr2=0x55cf56a0c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.159+0000 7f0d37be6740 1 --2- 172.21.15.37:0/3013430087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a0c940 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400dc80 tx=0x7f0d2400f730 comp rx=0 tx=0).stop 2024-03-26T06:27:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.159+0000 7f0d37be6740 1 -- 172.21.15.37:0/3013430087 shutdown_connections 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.159+0000 7f0d37be6740 1 --2- 172.21.15.37:0/3013430087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a0c940 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.159+0000 7f0d37be6740 1 --2- 172.21.15.37:0/3013430087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a00fd0 0x55cf56a01450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.159+0000 7f0d37be6740 1 --2- 172.21.15.37:0/3013430087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a01d40 0x55cf56a00a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.159+0000 7f0d37be6740 1 -- 172.21.15.37:0/3013430087 >> 172.21.15.37:0/3013430087 conn(0x55cf569c85b0 msgr2=0x55cf569fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.159+0000 7f0d37be6740 1 -- 172.21.15.37:0/3013430087 shutdown_connections 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.159+0000 7f0d37be6740 1 -- 172.21.15.37:0/3013430087 wait complete. 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.159+0000 7f0d37be6740 1 -- start start 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.159+0000 7f0d37be6740 1 Processor -- start 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a00fd0 0x55cf56a075c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a07b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a0a550 0x55cf56a08040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d37be6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf56a0d280 con 0x55cf56a01d40 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d37be6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf56921d90 con 0x55cf56a00fd0 2024-03-26T06:27:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d37be6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf56a0d400 con 0x55cf56a0a550 2024-03-26T06:27:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d37be6740 10 client.0 _ll_register_callbacks cb 0x55cf56679a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:27:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a07b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a07b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60158/0 (socket says 172.21.15.37:60158) 2024-03-26T06:27:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/1000216661 learned_addr learned my addr 172.21.15.37:0/1000216661 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2e59c640 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a00fd0 0x55cf56a075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2ed9d640 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a0a550 0x55cf56a08040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/1000216661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a0a550 msgr2=0x55cf56a08040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a0a550 0x55cf56a08040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/1000216661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a00fd0 msgr2=0x55cf56a075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a00fd0 0x55cf56a075c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/1000216661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf56746620 con 0x55cf56a01d40 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2ed9d640 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a0a550 0x55cf56a08040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a07b00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0d24011480 tx=0x7f0d2400dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f35ec9cfdac143b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.160+0000 7f0d15ffb640 1 -- 172.21.15.37:0/1000216661 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24028060 con 0x55cf56a01d40 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d15ffb640 1 -- 172.21.15.37:0/1000216661 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0d2401a730 con 0x55cf56a01d40 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d37be6740 1 -- 172.21.15.37:0/1000216661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d2401a940 con 0x55cf56a01d40 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d15ffb640 1 -- 172.21.15.37:0/1000216661 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d24015070 con 0x55cf56a01d40 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d37be6740 10 client.15831 fetch_fsmap learned FSMap version 36 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d37be6740 1 -- 172.21.15.37:0/1000216661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cf56a195b0 con 0x55cf56a01d40 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d15ffb640 1 -- 172.21.15.37:0/1000216661 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d2401a940 con 0x55cf56a01d40 2024-03-26T06:27:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d15ffb640 1 -- 172.21.15.37:0/1000216661 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0d24004810 con 0x55cf56a01d40 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d37be6740 10 client.15831 fetch_fsmap finished waiting for FSMap version 36 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d37be6740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d37be6740 20 client.15831 populate_metadata read hostname 'smithi037' 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d37be6740 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cf56a1a960 0x55cf56a1cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d37be6740 4 client.15831 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d37be6740 1 -- 172.21.15.37:0/1000216661 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf56972b30 con 0x55cf56a1a960 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.161+0000 7f0d2e59c640 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cf56a1a960 0x55cf56a1cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.162+0000 7f0d2e59c640 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cf56a1a960 0x55cf56a1cd50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf3b07fd1c44d6a server_cookie=b2c715f28dca342b in_seq=0 out_seq=0 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.162+0000 7f0d15ffb640 10 client.15831 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.162+0000 7f0d15ffb640 1 -- 172.21.15.37:0/1000216661 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cf56972b30 con 0x55cf56a1a960 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.162+0000 7f0d15ffb640 10 client.15831 handle_command_reply: tid=0 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.162+0000 7f0d37be6740 1 client.15831 shutdown 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d37be6740 1 -- 172.21.15.37:0/1000216661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 msgr2=0x55cf56a07b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d37be6740 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a07b00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0d24011480 tx=0x7f0d2400dee0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d37be6740 1 -- 172.21.15.37:0/1000216661 shutdown_connections 2024-03-26T06:27:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d37be6740 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cf56a1a960 0x55cf56a1cd50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d37be6740 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a0a550 0x55cf56a08040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d37be6740 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a07b00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d15ffb640 0 client.15831 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d37be6740 1 --2- 172.21.15.37:0/1000216661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a00fd0 0x55cf56a075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d37be6740 1 -- 172.21.15.37:0/1000216661 >> 172.21.15.37:0/1000216661 conn(0x55cf569c85b0 msgr2=0x55cf569fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d37be6740 1 -- 172.21.15.37:0/1000216661 shutdown_connections 2024-03-26T06:27:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d37be6740 1 -- 172.21.15.37:0/1000216661 wait complete. 2024-03-26T06:27:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.163+0000 7f0d37be6740 20 client.15831 trim_cache size 0 max 16384 2024-03-26T06:27:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.176+0000 7f0d37be6740 1 Processor -- start 2024-03-26T06:27:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.176+0000 7f0d37be6740 1 -- start start 2024-03-26T06:27:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.176+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a00fd0 0x55cf56a09cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.176+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a01d40 0x55cf56a11670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.176+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a11bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.176+0000 7f0d37be6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301acbb0 con 0x55cf56a0a550 2024-03-26T06:27:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.176+0000 7f0d37be6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301ae590 con 0x55cf56a01d40 2024-03-26T06:27:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.176+0000 7f0d37be6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301ad210 con 0x55cf56a00fd0 2024-03-26T06:27:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.176+0000 7f0d2ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a11bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d2ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a11bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60168/0 (socket says 172.21.15.37:60168) 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d2ed9d640 1 -- 172.21.15.37:0/3482614423 learned_addr learned my addr 172.21.15.37:0/3482614423 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d2e59c640 1 --2- 172.21.15.37:0/3482614423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a00fd0 0x55cf56a09cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d2ed9d640 1 -- 172.21.15.37:0/3482614423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a00fd0 msgr2=0x55cf56a09cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d2ed9d640 1 --2- 172.21.15.37:0/3482614423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a00fd0 0x55cf56a09cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/3482614423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a01d40 0x55cf56a11670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d2ed9d640 1 -- 172.21.15.37:0/3482614423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a01d40 msgr2=0x55cf56a11670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d2ed9d640 1 --2- 172.21.15.37:0/3482614423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a01d40 0x55cf56a11670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d2ed9d640 1 -- 172.21.15.37:0/3482614423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf56a002a0 con 0x55cf56a0a550 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/3482614423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a01d40 0x55cf56a11670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d2ed9d640 1 --2- 172.21.15.37:0/3482614423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a11bb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800c5e0 tx=0x7f0d2800b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf40240fbb1b234 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d17fff640 1 -- 172.21.15.37:0/3482614423 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28026030 con 0x55cf56a0a550 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d17fff640 1 -- 172.21.15.37:0/3482614423 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d28014250 con 0x55cf56a0a550 2024-03-26T06:27:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.177+0000 7f0d17fff640 1 -- 172.21.15.37:0/3482614423 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28022a80 con 0x55cf56a0a550 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 -- 172.21.15.37:0/3482614423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 msgr2=0x55cf56a11bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 --2- 172.21.15.37:0/3482614423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a11bb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800c5e0 tx=0x7f0d2800b000 comp rx=0 tx=0).stop 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 -- 172.21.15.37:0/3482614423 shutdown_connections 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 --2- 172.21.15.37:0/3482614423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a0a550 0x55cf56a11bb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 --2- 172.21.15.37:0/3482614423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a01d40 0x55cf56a11670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 --2- 172.21.15.37:0/3482614423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a00fd0 0x55cf56a09cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 -- 172.21.15.37:0/3482614423 >> 172.21.15.37:0/3482614423 conn(0x55cf569c85b0 msgr2=0x55cf569fb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 -- 172.21.15.37:0/3482614423 shutdown_connections 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 -- 172.21.15.37:0/3482614423 wait complete. 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 -- start start 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 Processor -- start 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a00fd0 0x55cf56a11970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a03f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a0a550 0x55cf56a044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301adf30 con 0x55cf56a01d40 2024-03-26T06:27:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301ad8d0 con 0x55cf56a0a550 2024-03-26T06:27:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301aae50 con 0x55cf56a00fd0 2024-03-26T06:27:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.178+0000 7f0d37be6740 10 client.0 _ll_register_callbacks cb 0x7f0d2400d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:27:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a03f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a03f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60178/0 (socket says 172.21.15.37:60178) 2024-03-26T06:27:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d2ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a0a550 0x55cf56a044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d2e59c640 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a00fd0 0x55cf56a11970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/321185072 learned_addr learned my addr 172.21.15.37:0/321185072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/321185072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a00fd0 msgr2=0x55cf56a11970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a00fd0 0x55cf56a11970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/321185072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a0a550 msgr2=0x55cf56a044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a0a550 0x55cf56a044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d2dd9b640 1 -- 172.21.15.37:0/321185072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf56a04d00 con 0x55cf56a01d40 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d2dd9b640 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a03f90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0d24000c00 tx=0x7f0d240115f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d354611d465ce1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d15ffb640 1 -- 172.21.15.37:0/321185072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24028070 con 0x55cf56a01d40 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d15ffb640 1 -- 172.21.15.37:0/321185072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0d24004240 con 0x55cf56a01d40 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d37be6740 1 -- 172.21.15.37:0/321185072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d28014250 con 0x55cf56a01d40 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d15ffb640 1 -- 172.21.15.37:0/321185072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d2400d500 con 0x55cf56a01d40 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d37be6740 10 client.15837 fetch_fsmap learned FSMap version 36 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.179+0000 7f0d37be6740 1 -- 172.21.15.37:0/321185072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cf56a719a0 con 0x55cf56a01d40 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.180+0000 7f0d15ffb640 1 -- 172.21.15.37:0/321185072 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d28014250 con 0x55cf56a01d40 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.180+0000 7f0d15ffb640 1 -- 172.21.15.37:0/321185072 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0d2402b040 con 0x55cf56a01d40 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.180+0000 7f0d37be6740 10 client.15837 fetch_fsmap finished waiting for FSMap version 36 2024-03-26T06:27:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.180+0000 7f0d37be6740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:27:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.180+0000 7f0d37be6740 20 client.15837 populate_metadata read hostname 'smithi037' 2024-03-26T06:27:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.180+0000 7f0d37be6740 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cf56a71fc0 0x55cf56a743e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.180+0000 7f0d37be6740 4 client.15837 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:27:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.180+0000 7f0d37be6740 1 -- 172.21.15.37:0/321185072 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cf5696ce80 con 0x55cf56a71fc0 2024-03-26T06:27:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.180+0000 7f0d2e59c640 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cf56a71fc0 0x55cf56a743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.180+0000 7f0d2e59c640 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cf56a71fc0 0x55cf56a743e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a71c068a25753a7e server_cookie=37ce9676d8b71ff1 in_seq=0 out_seq=0 2024-03-26T06:27:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.180+0000 7f0d15ffb640 10 client.15837 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d15ffb640 1 -- 172.21.15.37:0/321185072 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cf5696ce80 con 0x55cf56a71fc0 2024-03-26T06:27:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d15ffb640 10 client.15837 handle_command_reply: tid=0 2024-03-26T06:27:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 1 client.15837 shutdown 2024-03-26T06:27:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 1 -- 172.21.15.37:0/321185072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 msgr2=0x55cf56a03f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a03f90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0d24000c00 tx=0x7f0d240115f0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 1 -- 172.21.15.37:0/321185072 shutdown_connections 2024-03-26T06:27:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cf56a71fc0 0x55cf56a743e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cf56a0a550 0x55cf56a044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf56a01d40 0x55cf56a03f90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d15ffb640 0 client.15837 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 1 --2- 172.21.15.37:0/321185072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf56a00fd0 0x55cf56a11970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 1 -- 172.21.15.37:0/321185072 >> 172.21.15.37:0/321185072 conn(0x55cf569c85b0 msgr2=0x55cf56a05ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 1 -- 172.21.15.37:0/321185072 shutdown_connections 2024-03-26T06:27:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 1 -- 172.21.15.37:0/321185072 wait complete. 2024-03-26T06:27:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d37be6740 20 client.15837 trim_cache size 0 max 16384 2024-03-26T06:27:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:27:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:27:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:27:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:27:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d34f29640 1 -- 172.21.15.37:0/1965219124 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0d180a2a10 msgr2=0x7f0d180a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d34f29640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0d180a2a10 0x7f0d180a4ed0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d28005d80 tx=0x7f0d28005d10 comp rx=0 tx=0).stop 2024-03-26T06:27:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d34f29640 1 -- 172.21.15.37:0/1965219124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d30100440 msgr2=0x7f0d30198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d34f29640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d30100440 0x7f0d30198940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0d20005e30 tx=0x7f0d20032f20 comp rx=0 tx=0).stop 2024-03-26T06:27:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d34f29640 1 -- 172.21.15.37:0/1965219124 shutdown_connections 2024-03-26T06:27:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d34f29640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0d180a2a10 0x7f0d180a4ed0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d34f29640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d30105170 0x7f0d301a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d34f29640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d30100de0 0x7f0d30198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d34f29640 1 --2- 172.21.15.37:0/1965219124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d30100440 0x7f0d30198940 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.220+0000 7f0d34f29640 1 -- 172.21.15.37:0/1965219124 >> 172.21.15.37:0/1965219124 conn(0x7f0d30076fc0 msgr2=0x7f0d300fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.221+0000 7f0d34f29640 1 -- 172.21.15.37:0/1965219124 shutdown_connections 2024-03-26T06:27:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:11.221+0000 7f0d34f29640 1 -- 172.21.15.37:0/1965219124 wait complete. 2024-03-26T06:27:11.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 96796 2.50 MB/sec execute 46 sec latency 1452.143 ms 2024-03-26T06:27:12.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 99495 2.80 MB/sec execute 47 sec latency 11.112 ms 2024-03-26T06:27:12.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:12 smithi066 ceph-mon[36504]: pgmap v129: 97 pgs: 97 active+clean; 46 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-03-26T06:27:12.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:12 smithi149 ceph-mon[36681]: pgmap v129: 97 pgs: 97 active+clean; 46 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-03-26T06:27:12.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:12 smithi037 ceph-mon[31974]: pgmap v129: 97 pgs: 97 active+clean; 46 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-03-26T06:27:13.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 100084 2.78 MB/sec execute 48 sec latency 346.583 ms 2024-03-26T06:27:14.496 INFO:tasks.workunit.client.0.smithi037.stdout: 1 100084 2.72 MB/sec execute 49 sec latency 1346.690 ms 2024-03-26T06:27:14.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:14 smithi066 ceph-mon[36504]: pgmap v130: 97 pgs: 97 active+clean; 46 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-03-26T06:27:14.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:14 smithi149 ceph-mon[36681]: pgmap v130: 97 pgs: 97 active+clean; 46 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-03-26T06:27:14.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:14 smithi037 ceph-mon[31974]: pgmap v130: 97 pgs: 97 active+clean; 46 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-03-26T06:27:15.497 INFO:tasks.workunit.client.0.smithi037.stdout: 1 100084 2.67 MB/sec execute 50 sec latency 2346.799 ms 2024-03-26T06:27:16.497 INFO:tasks.workunit.client.0.smithi037.stdout: 1 100434 2.65 MB/sec execute 51 sec latency 3153.610 ms 2024-03-26T06:27:16.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:16 smithi066 ceph-mon[36504]: pgmap v131: 97 pgs: 97 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-03-26T06:27:16.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:16 smithi149 ceph-mon[36681]: pgmap v131: 97 pgs: 97 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-03-26T06:27:16.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:16 smithi037 ceph-mon[31974]: pgmap v131: 97 pgs: 97 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-03-26T06:27:17.497 INFO:tasks.workunit.client.0.smithi037.stdout: 1 103128 2.92 MB/sec execute 52 sec latency 9.848 ms 2024-03-26T06:27:18.497 INFO:tasks.workunit.client.0.smithi037.stdout: 1 104686 2.97 MB/sec execute 53 sec latency 10.641 ms 2024-03-26T06:27:18.847 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:18 smithi066 ceph-mon[36504]: pgmap v132: 97 pgs: 97 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-03-26T06:27:18.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:18 smithi149 ceph-mon[36681]: pgmap v132: 97 pgs: 97 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-03-26T06:27:18.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:18 smithi037 ceph-mon[31974]: pgmap v132: 97 pgs: 97 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-03-26T06:27:19.497 INFO:tasks.workunit.client.0.smithi037.stdout: 1 106850 3.17 MB/sec execute 54 sec latency 14.360 ms 2024-03-26T06:27:20.497 INFO:tasks.workunit.client.0.smithi037.stdout: 1 109032 3.26 MB/sec execute 55 sec latency 28.781 ms 2024-03-26T06:27:20.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:20 smithi066 ceph-mon[36504]: pgmap v133: 97 pgs: 97 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-03-26T06:27:20.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:20 smithi149 ceph-mon[36681]: pgmap v133: 97 pgs: 97 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-03-26T06:27:20.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:20 smithi037 ceph-mon[31974]: pgmap v133: 97 pgs: 97 active+clean; 44 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-03-26T06:27:21.497 INFO:tasks.workunit.client.0.smithi037.stdout: 1 110603 3.40 MB/sec execute 56 sec latency 11.847 ms 2024-03-26T06:27:22.497 INFO:tasks.workunit.client.0.smithi037.stdout: 1 113171 3.57 MB/sec execute 57 sec latency 13.342 ms 2024-03-26T06:27:22.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:22 smithi149 ceph-mon[36681]: pgmap v134: 97 pgs: 97 active+clean; 49 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-03-26T06:27:22.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:22 smithi037 ceph-mon[31974]: pgmap v134: 97 pgs: 97 active+clean; 49 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-03-26T06:27:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:22 smithi066 ceph-mon[36504]: pgmap v134: 97 pgs: 97 active+clean; 49 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-03-26T06:27:23.497 INFO:tasks.workunit.client.0.smithi037.stdout: 1 114266 3.62 MB/sec execute 58 sec latency 175.079 ms 2024-03-26T06:27:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.319+0000 7fb395820640 1 -- 172.21.15.37:0/1534051478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb390100440 msgr2=0x7fb3901008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.319+0000 7fb395820640 1 --2- 172.21.15.37:0/1534051478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb390100440 0x7fb3901008c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb378009900 tx=0x7fb37802bc80 comp rx=0 tx=0).stop 2024-03-26T06:27:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.319+0000 7fb395820640 1 -- 172.21.15.37:0/1534051478 shutdown_connections 2024-03-26T06:27:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.319+0000 7fb395820640 1 --2- 172.21.15.37:0/1534051478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb390100e00 0x7fb39010d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.319+0000 7fb395820640 1 --2- 172.21.15.37:0/1534051478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb390100440 0x7fb3901008c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.319+0000 7fb395820640 1 --2- 172.21.15.37:0/1534051478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3901062d0 0x7fb3901066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.319+0000 7fb395820640 1 -- 172.21.15.37:0/1534051478 >> 172.21.15.37:0/1534051478 conn(0x7fb390077040 msgr2=0x7fb390077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.319+0000 7fb395820640 1 -- 172.21.15.37:0/1534051478 shutdown_connections 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.319+0000 7fb395820640 1 -- 172.21.15.37:0/1534051478 wait complete. 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.319+0000 7fb395820640 1 Processor -- start 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb395820640 1 -- start start 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb395820640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb390100440 0x7fb39019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb395820640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb390100e00 0x7fb39019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb395820640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3901062d0 0x7fb390197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb395820640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb390111730 con 0x7fb390100e00 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb395820640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901115b0 con 0x7fb3901062d0 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb395820640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901118b0 con 0x7fb390100440 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb38e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb390100e00 0x7fb39019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb38e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb390100e00 0x7fb39019dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32966/0 (socket says 172.21.15.37:32966) 2024-03-26T06:27:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb38e7fc640 1 -- 172.21.15.37:0/2214358363 learned_addr learned my addr 172.21.15.37:0/2214358363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb38f7fe640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3901062d0 0x7fb390197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb38effd640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb390100440 0x7fb39019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb38e7fc640 1 -- 172.21.15.37:0/2214358363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb390100440 msgr2=0x7fb39019d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb390100440 0x7fb39019d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb38e7fc640 1 -- 172.21.15.37:0/2214358363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3901062d0 msgr2=0x7fb390197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3901062d0 0x7fb390197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.320+0000 7fb38e7fc640 1 -- 172.21.15.37:0/2214358363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb390198110 con 0x7fb390100e00 2024-03-26T06:27:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.321+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb390100e00 0x7fb39019dcd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb378006010 tx=0x7fb37802cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.321+0000 7fb39481e640 1 -- 172.21.15.37:0/2214358363 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378041070 con 0x7fb390100e00 2024-03-26T06:27:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.321+0000 7fb39481e640 1 -- 172.21.15.37:0/2214358363 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3780054e0 con 0x7fb390100e00 2024-03-26T06:27:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.321+0000 7fb395820640 1 -- 172.21.15.37:0/2214358363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3901983a0 con 0x7fb390100e00 2024-03-26T06:27:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.321+0000 7fb39481e640 1 -- 172.21.15.37:0/2214358363 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378034440 con 0x7fb390100e00 2024-03-26T06:27:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.321+0000 7fb395820640 1 -- 172.21.15.37:0/2214358363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3901a4650 con 0x7fb390100e00 2024-03-26T06:27:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.323+0000 7fb39481e640 1 -- 172.21.15.37:0/2214358363 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb378044070 con 0x7fb390100e00 2024-03-26T06:27:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.323+0000 7fb398404740 1 Processor -- start 2024-03-26T06:27:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.323+0000 7fb398404740 1 -- start start 2024-03-26T06:27:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb39481e640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb37c0a2a10 0x7fb37c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb398404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c107d40 0x55b71c106a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb398404740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c106fd0 0x55b71c107450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb398404740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 0x55b71c112940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb398404740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71c06b5f0 con 0x55b71c106fd0 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb398404740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71c014930 con 0x55b71c110550 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb398404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71c067690 con 0x55b71c107d40 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38f7fe640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb37c0a2a10 0x7fb37c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb39481e640 1 -- 172.21.15.37:0/2214358363 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fb3780f0920 con 0x7fb390100e00 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c106fd0 0x55b71c107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c106fd0 0x55b71c107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32972/0 (socket says 172.21.15.37:32972) 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38e7fc640 1 -- 172.21.15.37:0/1892546824 learned_addr learned my addr 172.21.15.37:0/1892546824 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38effd640 1 --2- 172.21.15.37:0/1892546824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c107d40 0x55b71c106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38e7fc640 1 -- 172.21.15.37:0/1892546824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c107d40 msgr2=0x55b71c106a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/1892546824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c107d40 0x55b71c106a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38e7fc640 1 -- 172.21.15.37:0/1892546824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 msgr2=0x55b71c112940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38f7fe640 1 --2- 172.21.15.37:0/1892546824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 0x55b71c112940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/1892546824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 0x55b71c112940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38e7fc640 1 -- 172.21.15.37:0/1892546824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b71bd80210 con 0x55b71c106fd0 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38effd640 1 --2- 172.21.15.37:0/1892546824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c107d40 0x55b71c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38f7fe640 1 --2- 172.21.15.37:0/1892546824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 0x55b71c112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38f7fe640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb37c0a2a10 0x7fb37c0a4ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb38800b580 tx=0x7fb38800b9d0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.324+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/1892546824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c106fd0 0x55b71c107450 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb3780c5330 tx=0x7fb3780b9420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae3869693a64503 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb385ffb640 1 -- 172.21.15.37:0/1892546824 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37804b040 con 0x55b71c106fd0 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb385ffb640 1 -- 172.21.15.37:0/1892546824 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb378005690 con 0x55b71c106fd0 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb385ffb640 1 -- 172.21.15.37:0/1892546824 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378004c40 con 0x55b71c106fd0 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb398404740 1 -- 172.21.15.37:0/1892546824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c106fd0 msgr2=0x55b71c107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb398404740 1 --2- 172.21.15.37:0/1892546824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c106fd0 0x55b71c107450 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb3780c5330 tx=0x7fb3780b9420 comp rx=0 tx=0).stop 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb398404740 1 -- 172.21.15.37:0/1892546824 shutdown_connections 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb398404740 1 --2- 172.21.15.37:0/1892546824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 0x55b71c112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb398404740 1 --2- 172.21.15.37:0/1892546824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c106fd0 0x55b71c107450 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb398404740 1 --2- 172.21.15.37:0/1892546824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c107d40 0x55b71c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb398404740 1 -- 172.21.15.37:0/1892546824 >> 172.21.15.37:0/1892546824 conn(0x55b71c0ce5b0 msgr2=0x55b71c101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb398404740 1 -- 172.21.15.37:0/1892546824 shutdown_connections 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.325+0000 7fb398404740 1 -- 172.21.15.37:0/1892546824 wait complete. 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb398404740 1 -- start start 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb398404740 1 Processor -- start 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb398404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c106fd0 0x55b71c10d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb398404740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c10db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb398404740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 0x55b71c10e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb398404740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71c113040 con 0x55b71c107d40 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb398404740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71c027d90 con 0x55b71c110550 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb398404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71c1131c0 con 0x55b71c106fd0 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb398404740 10 client.0 _ll_register_callbacks cb 0x55b71bd7fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:27:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb38e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c10db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb38effd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c106fd0 0x55b71c10d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb38e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c10db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32974/0 (socket says 172.21.15.37:32974) 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.326+0000 7fb38e7fc640 1 -- 172.21.15.37:0/1921649244 learned_addr learned my addr 172.21.15.37:0/1921649244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb38f7fe640 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 0x55b71c10e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb38e7fc640 1 -- 172.21.15.37:0/1921649244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c106fd0 msgr2=0x55b71c10d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c106fd0 0x55b71c10d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb38e7fc640 1 -- 172.21.15.37:0/1921649244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 msgr2=0x55b71c10e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 0x55b71c10e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb38e7fc640 1 -- 172.21.15.37:0/1921649244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b71be4c620 con 0x55b71c107d40 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb38f7fe640 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 0x55b71c10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c10db30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb378004650 tx=0x7fb3780bcd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06581cbcec5590a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb36effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:27:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb36effd640 1 -- 172.21.15.37:0/1921649244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37804b070 con 0x55b71c107d40 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb36effd640 1 -- 172.21.15.37:0/1921649244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fb3780a6160 con 0x55b71c107d40 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb398404740 1 -- 172.21.15.37:0/1921649244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b71c078b30 con 0x55b71c107d40 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb36effd640 1 -- 172.21.15.37:0/1921649244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3780bd7c0 con 0x55b71c107d40 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb398404740 10 client.15849 fetch_fsmap learned FSMap version 36 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb398404740 1 -- 172.21.15.37:0/1921649244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b71c113040 con 0x55b71c107d40 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.327+0000 7fb36effd640 1 -- 172.21.15.37:0/1921649244 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b71c078b30 con 0x55b71c107d40 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.328+0000 7fb36effd640 1 -- 172.21.15.37:0/1921649244 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb3780b3a90 con 0x55b71c107d40 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.328+0000 7fb398404740 10 client.15849 fetch_fsmap finished waiting for FSMap version 36 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.328+0000 7fb398404740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.328+0000 7fb398404740 20 client.15849 populate_metadata read hostname 'smithi037' 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.328+0000 7fb398404740 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b71c1203d0 0x55b71c1227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.328+0000 7fb398404740 4 client.15849 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.328+0000 7fb398404740 1 -- 172.21.15.37:0/1921649244 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b71c072e80 con 0x55b71c1203d0 2024-03-26T06:27:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.328+0000 7fb38effd640 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b71c1203d0 0x55b71c1227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb36effd640 10 client.15849 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb38effd640 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b71c1203d0 0x55b71c1227c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c2e9447fb727d3 server_cookie=d5293142b220f182 in_seq=0 out_seq=0 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb36effd640 1 -- 172.21.15.37:0/1921649244 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b71c072e80 con 0x55b71c1203d0 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb36effd640 10 client.15849 handle_command_reply: tid=0 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb398404740 1 client.15849 shutdown 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb398404740 1 -- 172.21.15.37:0/1921649244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 msgr2=0x55b71c10db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb398404740 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c10db30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb378004650 tx=0x7fb3780bcd00 comp rx=0 tx=0).stop 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb398404740 1 -- 172.21.15.37:0/1921649244 shutdown_connections 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb398404740 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b71c1203d0 0x55b71c1227c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb398404740 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c110550 0x55b71c10e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb398404740 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c10db30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb36effd640 0 client.15849 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb398404740 1 --2- 172.21.15.37:0/1921649244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c106fd0 0x55b71c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.329+0000 7fb398404740 1 -- 172.21.15.37:0/1921649244 >> 172.21.15.37:0/1921649244 conn(0x55b71c0ce5b0 msgr2=0x55b71c103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.330+0000 7fb398404740 1 -- 172.21.15.37:0/1921649244 shutdown_connections 2024-03-26T06:27:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.330+0000 7fb398404740 1 -- 172.21.15.37:0/1921649244 wait complete. 2024-03-26T06:27:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.330+0000 7fb398404740 20 client.15849 trim_cache size 0 max 16384 2024-03-26T06:27:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.343+0000 7fb398404740 1 Processor -- start 2024-03-26T06:27:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.343+0000 7fb398404740 1 -- start start 2024-03-26T06:27:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.343+0000 7fb398404740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c106fd0 0x55b71c10fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.343+0000 7fb398404740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c1172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.343+0000 7fb398404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c110550 0x55b71c117830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.343+0000 7fb398404740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901aca50 con 0x55b71c107d40 2024-03-26T06:27:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.343+0000 7fb398404740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901ae430 con 0x55b71c106fd0 2024-03-26T06:27:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.343+0000 7fb398404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901ad0b0 con 0x55b71c110550 2024-03-26T06:27:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.343+0000 7fb38e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c1172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.343+0000 7fb38e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c1172f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32982/0 (socket says 172.21.15.37:32982) 2024-03-26T06:27:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.343+0000 7fb38e7fc640 1 -- 172.21.15.37:0/3917830927 learned_addr learned my addr 172.21.15.37:0/3917830927 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb38f7fe640 1 --2- 172.21.15.37:0/3917830927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c110550 0x55b71c117830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb38effd640 1 --2- 172.21.15.37:0/3917830927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c106fd0 0x55b71c10fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb38e7fc640 1 -- 172.21.15.37:0/3917830927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c110550 msgr2=0x55b71c117830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/3917830927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c110550 0x55b71c117830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb38e7fc640 1 -- 172.21.15.37:0/3917830927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c106fd0 msgr2=0x55b71c10fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:24.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/3917830927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c106fd0 0x55b71c10fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb38e7fc640 1 -- 172.21.15.37:0/3917830927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b71c122b30 con 0x55b71c107d40 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb38effd640 1 --2- 172.21.15.37:0/3917830927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c106fd0 0x55b71c10fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/3917830927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c1172f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb3780b9520 tx=0x7fb37802bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd3e8682330109a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb3857fa640 1 -- 172.21.15.37:0/3917830927 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37804b070 con 0x55b71c107d40 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb3857fa640 1 -- 172.21.15.37:0/3917830927 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3780dd070 con 0x55b71c107d40 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.344+0000 7fb3857fa640 1 -- 172.21.15.37:0/3917830927 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3780a63c0 con 0x55b71c107d40 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.345+0000 7fb398404740 1 -- 172.21.15.37:0/3917830927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 msgr2=0x55b71c1172f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.345+0000 7fb398404740 1 --2- 172.21.15.37:0/3917830927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c1172f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb3780b9520 tx=0x7fb37802bc80 comp rx=0 tx=0).stop 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.345+0000 7fb398404740 1 -- 172.21.15.37:0/3917830927 shutdown_connections 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.345+0000 7fb398404740 1 --2- 172.21.15.37:0/3917830927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c110550 0x55b71c117830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.345+0000 7fb398404740 1 --2- 172.21.15.37:0/3917830927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c107d40 0x55b71c1172f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.345+0000 7fb398404740 1 --2- 172.21.15.37:0/3917830927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c106fd0 0x55b71c10fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.345+0000 7fb398404740 1 -- 172.21.15.37:0/3917830927 >> 172.21.15.37:0/3917830927 conn(0x55b71c0ce5b0 msgr2=0x55b71c10bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.345+0000 7fb398404740 1 -- 172.21.15.37:0/3917830927 shutdown_connections 2024-03-26T06:27:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.345+0000 7fb398404740 1 -- 172.21.15.37:0/3917830927 wait complete. 2024-03-26T06:27:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.345+0000 7fb398404740 1 -- start start 2024-03-26T06:27:24.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb398404740 1 Processor -- start 2024-03-26T06:27:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb398404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c106fd0 0x55b71c10a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb398404740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c107d40 0x55b71c10a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb398404740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c110550 0x55b71c10ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb398404740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901addd0 con 0x55b71c110550 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb398404740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901ad770 con 0x55b71c107d40 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb398404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901aacf0 con 0x55b71c106fd0 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb398404740 10 client.0 _ll_register_callbacks cb 0x7fb378004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb38f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c110550 0x55b71c10ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb38f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c110550 0x55b71c10ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32992/0 (socket says 172.21.15.37:32992) 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb38f7fe640 1 -- 172.21.15.37:0/3414160045 learned_addr learned my addr 172.21.15.37:0/3414160045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb38effd640 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c106fd0 0x55b71c10a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c107d40 0x55b71c10a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb38f7fe640 1 -- 172.21.15.37:0/3414160045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c106fd0 msgr2=0x55b71c10a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb38f7fe640 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c106fd0 0x55b71c10a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb38f7fe640 1 -- 172.21.15.37:0/3414160045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c107d40 msgr2=0x55b71c10a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb38f7fe640 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c107d40 0x55b71c10a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb38f7fe640 1 -- 172.21.15.37:0/3414160045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b71c176880 con 0x55b71c110550 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.346+0000 7fb38e7fc640 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c107d40 0x55b71c10a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb38f7fe640 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c110550 0x55b71c10ae00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb388004030 tx=0x7fb388004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c39eeab0de13f81 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb36effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb36effd640 1 -- 172.21.15.37:0/3414160045 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388005020 con 0x55b71c110550 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb36effd640 1 -- 172.21.15.37:0/3414160045 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fb3880051c0 con 0x55b71c110550 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb398404740 1 -- 172.21.15.37:0/3414160045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3780dd070 con 0x55b71c110550 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb36effd640 1 -- 172.21.15.37:0/3414160045 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3880053c0 con 0x55b71c110550 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb398404740 10 client.15855 fetch_fsmap learned FSMap version 36 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb398404740 1 -- 172.21.15.37:0/3414160045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b71c177b90 con 0x55b71c110550 2024-03-26T06:27:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb36effd640 1 -- 172.21.15.37:0/3414160045 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3780dd070 con 0x55b71c110550 2024-03-26T06:27:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb36effd640 1 -- 172.21.15.37:0/3414160045 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb38802a760 con 0x55b71c110550 2024-03-26T06:27:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb398404740 10 client.15855 fetch_fsmap finished waiting for FSMap version 36 2024-03-26T06:27:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb398404740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:27:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb398404740 20 client.15855 populate_metadata read hostname 'smithi037' 2024-03-26T06:27:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb398404740 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b71c1782f0 0x55b71c17a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb398404740 4 client.15855 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:27:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.347+0000 7fb398404740 1 -- 172.21.15.37:0/3414160045 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b71bff1710 con 0x55b71c1782f0 2024-03-26T06:27:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.348+0000 7fb38effd640 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b71c1782f0 0x55b71c17a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.348+0000 7fb38effd640 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b71c1782f0 0x55b71c17a710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebfe94b3fdb2dc3d server_cookie=788febb8ff3b9263 in_seq=0 out_seq=0 2024-03-26T06:27:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.348+0000 7fb36effd640 10 client.15855 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb36effd640 1 -- 172.21.15.37:0/3414160045 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b71bff1710 con 0x55b71c1782f0 2024-03-26T06:27:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb36effd640 10 client.15855 handle_command_reply: tid=0 2024-03-26T06:27:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb398404740 1 client.15855 shutdown 2024-03-26T06:27:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb398404740 1 -- 172.21.15.37:0/3414160045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c110550 msgr2=0x55b71c10ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb398404740 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c110550 0x55b71c10ae00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb388004030 tx=0x7fb388004730 comp rx=0 tx=0).stop 2024-03-26T06:27:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb398404740 1 -- 172.21.15.37:0/3414160045 shutdown_connections 2024-03-26T06:27:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb398404740 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b71c1782f0 0x55b71c17a710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb398404740 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b71c110550 0x55b71c10ae00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb36effd640 0 client.15855 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb398404740 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b71c107d40 0x55b71c10a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb398404740 1 --2- 172.21.15.37:0/3414160045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b71c106fd0 0x55b71c10a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.392+0000 7fb398404740 1 -- 172.21.15.37:0/3414160045 >> 172.21.15.37:0/3414160045 conn(0x55b71c0ce5b0 msgr2=0x55b71c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb398404740 1 -- 172.21.15.37:0/3414160045 shutdown_connections 2024-03-26T06:27:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb398404740 1 -- 172.21.15.37:0/3414160045 wait complete. 2024-03-26T06:27:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb398404740 20 client.15855 trim_cache size 0 max 16384 2024-03-26T06:27:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:27:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:27:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:27:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:27:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 -- 172.21.15.37:0/2214358363 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb37c0a2a10 msgr2=0x7fb37c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb37c0a2a10 0x7fb37c0a4ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb38800b580 tx=0x7fb38800b9d0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 -- 172.21.15.37:0/2214358363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb390100e00 msgr2=0x7fb39019dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb390100e00 0x7fb39019dcd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb378006010 tx=0x7fb37802cbf0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 -- 172.21.15.37:0/2214358363 shutdown_connections 2024-03-26T06:27:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb37c0a2a10 0x7fb37c0a4ed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3901062d0 0x7fb390197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb390100e00 0x7fb39019dcd0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 --2- 172.21.15.37:0/2214358363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb390100440 0x7fb39019d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 -- 172.21.15.37:0/2214358363 >> 172.21.15.37:0/2214358363 conn(0x7fb390077040 msgr2=0x7fb3900ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 -- 172.21.15.37:0/2214358363 shutdown_connections 2024-03-26T06:27:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:24.393+0000 7fb395820640 1 -- 172.21.15.37:0/2214358363 wait complete. 2024-03-26T06:27:24.497 INFO:tasks.workunit.client.0.smithi037.stdout: 1 114266 3.56 MB/sec execute 59 sec latency 1175.202 ms 2024-03-26T06:27:24.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:24 smithi149 ceph-mon[36681]: pgmap v135: 97 pgs: 97 active+clean; 49 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 808 KiB/s wr, 146 op/s 2024-03-26T06:27:24.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:24 smithi037 ceph-mon[31974]: pgmap v135: 97 pgs: 97 active+clean; 49 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 808 KiB/s wr, 146 op/s 2024-03-26T06:27:25.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:24 smithi066 ceph-mon[36504]: pgmap v135: 97 pgs: 97 active+clean; 49 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 808 KiB/s wr, 146 op/s 2024-03-26T06:27:25.497 INFO:tasks.workunit.client.0.smithi037.stdout: 1 114266 3.50 MB/sec execute 60 sec latency 2175.275 ms 2024-03-26T06:27:26.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 114334 3.44 MB/sec execute 61 sec latency 2982.642 ms 2024-03-26T06:27:26.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:26 smithi149 ceph-mon[36681]: pgmap v136: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 257 op/s 2024-03-26T06:27:26.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:26 smithi037 ceph-mon[31974]: pgmap v136: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 257 op/s 2024-03-26T06:27:27.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:26 smithi066 ceph-mon[36504]: pgmap v136: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 257 op/s 2024-03-26T06:27:27.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 114334 3.39 MB/sec execute 62 sec latency 1146.519 ms 2024-03-26T06:27:28.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 114334 3.33 MB/sec execute 63 sec latency 2146.637 ms 2024-03-26T06:27:28.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:28 smithi149 ceph-mon[36681]: pgmap v137: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-03-26T06:27:28.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:28 smithi037 ceph-mon[31974]: pgmap v137: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-03-26T06:27:29.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:28 smithi066 ceph-mon[36504]: pgmap v137: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-03-26T06:27:29.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 114334 3.28 MB/sec execute 64 sec latency 3146.673 ms 2024-03-26T06:27:30.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 114334 3.23 MB/sec execute 65 sec latency 4146.713 ms 2024-03-26T06:27:30.900 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:30 smithi066 ceph-mon[36504]: pgmap v138: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-03-26T06:27:30.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:30 smithi149 ceph-mon[36681]: pgmap v138: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-03-26T06:27:30.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:30 smithi037 ceph-mon[31974]: pgmap v138: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-03-26T06:27:31.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 114698 3.21 MB/sec execute 66 sec latency 4950.579 ms 2024-03-26T06:27:32.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 117150 3.39 MB/sec execute 67 sec latency 10.113 ms 2024-03-26T06:27:32.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:32 smithi149 ceph-mon[36681]: pgmap v139: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-03-26T06:27:32.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:32 smithi037 ceph-mon[31974]: pgmap v139: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-03-26T06:27:33.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:32 smithi066 ceph-mon[36504]: pgmap v139: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-03-26T06:27:33.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 118490 3.44 MB/sec execute 68 sec latency 35.805 ms 2024-03-26T06:27:34.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 120951 3.59 MB/sec execute 69 sec latency 10.109 ms 2024-03-26T06:27:34.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:34 smithi149 ceph-mon[36681]: pgmap v140: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-03-26T06:27:34.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:34 smithi037 ceph-mon[31974]: pgmap v140: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-03-26T06:27:35.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:34 smithi066 ceph-mon[36504]: pgmap v140: 97 pgs: 97 active+clean; 49 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-03-26T06:27:35.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 123035 3.64 MB/sec execute 70 sec latency 10.448 ms 2024-03-26T06:27:36.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 124638 3.76 MB/sec execute 71 sec latency 26.072 ms 2024-03-26T06:27:36.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:36 smithi149 ceph-mon[36681]: pgmap v141: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-03-26T06:27:36.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:36 smithi037 ceph-mon[31974]: pgmap v141: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-03-26T06:27:37.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:36 smithi066 ceph-mon[36504]: pgmap v141: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-03-26T06:27:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.483+0000 7f51b2e20640 1 -- 172.21.15.37:0/775111117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac1065e0 msgr2=0x7f51ac10e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.483+0000 7f51b2e20640 1 --2- 172.21.15.37:0/775111117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac1065e0 0x7f51ac10e1b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009ca0 tx=0x7f51a402c0f0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.483+0000 7f51b2e20640 1 -- 172.21.15.37:0/775111117 shutdown_connections 2024-03-26T06:27:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.483+0000 7f51b2e20640 1 --2- 172.21.15.37:0/775111117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac1065e0 0x7f51ac10e1b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.483+0000 7f51b2e20640 1 --2- 172.21.15.37:0/775111117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac101ec0 0x7f51ac105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.483+0000 7f51b2e20640 1 --2- 172.21.15.37:0/775111117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51ac101510 0x7f51ac1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.483+0000 7f51b2e20640 1 -- 172.21.15.37:0/775111117 >> 172.21.15.37:0/775111117 conn(0x7f51ac0fcdb0 msgr2=0x7f51ac0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.483+0000 7f51b2e20640 1 -- 172.21.15.37:0/775111117 shutdown_connections 2024-03-26T06:27:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.483+0000 7f51b2e20640 1 -- 172.21.15.37:0/775111117 wait complete. 2024-03-26T06:27:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.483+0000 7f51b2e20640 1 Processor -- start 2024-03-26T06:27:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51b2e20640 1 -- start start 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51b2e20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51ac101510 0x7f51ac19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51b2e20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac1065e0 0x7f51ac19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51b2e20640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac197920 0x7f51ac197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51b2e20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1116a0 con 0x7f51ac1065e0 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51b2e20640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac111520 con 0x7f51ac101510 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51b2e20640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac111820 con 0x7f51ac197920 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac1065e0 0x7f51ac19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51b0bbc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51ac101510 0x7f51ac19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac1065e0 0x7f51ac19dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60890/0 (socket says 172.21.15.37:60890) 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51abfff640 1 -- 172.21.15.37:0/3788387002 learned_addr learned my addr 172.21.15.37:0/3788387002 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51b13bd640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac197920 0x7f51ac197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51abfff640 1 -- 172.21.15.37:0/3788387002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac197920 msgr2=0x7f51ac197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51abfff640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac197920 0x7f51ac197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51abfff640 1 -- 172.21.15.37:0/3788387002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51ac101510 msgr2=0x7f51ac19d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51abfff640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51ac101510 0x7f51ac19d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51abfff640 1 -- 172.21.15.37:0/3788387002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac198440 con 0x7f51ac1065e0 2024-03-26T06:27:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51ac101510 0x7f51ac19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:27:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51abfff640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac1065e0 0x7f51ac19dcb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f519800e9f0 tx=0x7f519800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51a9ffb640 1 -- 172.21.15.37:0/3788387002 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519800ccc0 con 0x7f51ac1065e0 2024-03-26T06:27:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51a9ffb640 1 -- 172.21.15.37:0/3788387002 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5198005500 con 0x7f51ac1065e0 2024-03-26T06:27:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.484+0000 7f51b2e20640 1 -- 172.21.15.37:0/3788387002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac198730 con 0x7f51ac1065e0 2024-03-26T06:27:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.485+0000 7f51a9ffb640 1 -- 172.21.15.37:0/3788387002 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198010690 con 0x7f51ac1065e0 2024-03-26T06:27:37.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.485+0000 7f51b2e20640 1 -- 172.21.15.37:0/3788387002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac073820 con 0x7f51ac1065e0 2024-03-26T06:27:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.487+0000 7f51a9ffb640 1 -- 172.21.15.37:0/3788387002 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f51980046c0 con 0x7f51ac1065e0 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b5a02740 1 Processor -- start 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b5a02740 1 -- start start 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51a9ffb640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f519c0a2a10 0x7f519c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6efd40 0x56365c6eea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6eefd0 0x56365c6ef450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6f8550 0x56365c6fa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b5a02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c6535f0 con 0x56365c6eefd0 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b5a02740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c5fc930 con 0x56365c6f8550 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b5a02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c64f690 con 0x56365c6efd40 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b13bd640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f519c0a2a10 0x7f519c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6eefd0 0x56365c6ef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b0bbc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6efd40 0x56365c6eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b0bbc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6efd40 0x56365c6eea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:38728/0 (socket says 172.21.15.37:38728) 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6eefd0 0x56365c6ef450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60902/0 (socket says 172.21.15.37:60902) 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b0bbc640 1 -- 172.21.15.37:0/2788847301 learned_addr learned my addr 172.21.15.37:0/2788847301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b13bd640 1 --2- 172.21.15.37:0/2788847301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6f8550 0x56365c6fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51a9ffb640 1 -- 172.21.15.37:0/3788387002 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f51980cf110 con 0x7f51ac1065e0 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b0bbc640 1 -- 172.21.15.37:0/2788847301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6f8550 msgr2=0x56365c6fa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/2788847301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6f8550 0x56365c6fa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b0bbc640 1 -- 172.21.15.37:0/2788847301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6eefd0 msgr2=0x56365c6ef450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/2788847301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6eefd0 0x56365c6ef450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.488+0000 7f51b0bbc640 1 -- 172.21.15.37:0/2788847301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56365c368210 con 0x56365c6efd40 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.489+0000 7f51b13bd640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f519c0a2a10 0x7f519c0a4ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f51a4004af0 tx=0x7f51a40323c0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.489+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/2788847301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6efd40 0x56365c6eea90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f51a00096f0 tx=0x7f51a0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73b26d163f67caed server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.489+0000 7f5196ffd640 1 -- 172.21.15.37:0/2788847301 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a001e030 con 0x56365c6efd40 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.489+0000 7f5196ffd640 1 -- 172.21.15.37:0/2788847301 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51a0005500 con 0x56365c6efd40 2024-03-26T06:27:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.489+0000 7f5196ffd640 1 -- 172.21.15.37:0/2788847301 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a000b820 con 0x56365c6efd40 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.490+0000 7f51b5a02740 1 -- 172.21.15.37:0/2788847301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6efd40 msgr2=0x56365c6eea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.490+0000 7f51b5a02740 1 --2- 172.21.15.37:0/2788847301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6efd40 0x56365c6eea90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f51a00096f0 tx=0x7f51a0005f90 comp rx=0 tx=0).stop 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.490+0000 7f51b5a02740 1 -- 172.21.15.37:0/2788847301 shutdown_connections 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.490+0000 7f51b5a02740 1 --2- 172.21.15.37:0/2788847301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6f8550 0x56365c6fa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.490+0000 7f51b5a02740 1 --2- 172.21.15.37:0/2788847301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6eefd0 0x56365c6ef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.490+0000 7f51b5a02740 1 --2- 172.21.15.37:0/2788847301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6efd40 0x56365c6eea90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.490+0000 7f51b5a02740 1 -- 172.21.15.37:0/2788847301 >> 172.21.15.37:0/2788847301 conn(0x56365c6b65b0 msgr2=0x56365c6e9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.490+0000 7f51b5a02740 1 -- 172.21.15.37:0/2788847301 shutdown_connections 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.490+0000 7f51b5a02740 1 -- 172.21.15.37:0/2788847301 wait complete. 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.490+0000 7f51b5a02740 1 -- start start 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.490+0000 7f51b5a02740 1 Processor -- start 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6eefd0 0x56365c6f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6f8550 0x56365c6f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51b5a02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c6fb280 con 0x56365c6efd40 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51b5a02740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c60fd90 con 0x56365c6eefd0 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51b5a02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365c6fb400 con 0x56365c6f8550 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51b5a02740 10 client.0 _ll_register_callbacks cb 0x56365c367a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60904/0 (socket says 172.21.15.37:60904) 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51abfff640 1 -- 172.21.15.37:0/194935648 learned_addr learned my addr 172.21.15.37:0/194935648 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51b13bd640 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6f8550 0x56365c6f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6eefd0 0x56365c6f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51abfff640 1 -- 172.21.15.37:0/194935648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6f8550 msgr2=0x56365c6f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51abfff640 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6f8550 0x56365c6f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51abfff640 1 -- 172.21.15.37:0/194935648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6eefd0 msgr2=0x56365c6f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51abfff640 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6eefd0 0x56365c6f55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51abfff640 1 -- 172.21.15.37:0/194935648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56365c434620 con 0x56365c6efd40 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51abfff640 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f5b00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f51980cc8f0 tx=0x7f519809b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f94f37d39a8b15c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f518bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f518bfff640 1 -- 172.21.15.37:0/194935648 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519809b7a0 con 0x56365c6efd40 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f518bfff640 1 -- 172.21.15.37:0/194935648 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f5198097e10 con 0x56365c6efd40 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.491+0000 7f51b5a02740 1 -- 172.21.15.37:0/194935648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51a0005500 con 0x56365c6efd40 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f518bfff640 1 -- 172.21.15.37:0/194935648 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51980c0080 con 0x56365c6efd40 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f51b5a02740 10 client.15867 fetch_fsmap learned FSMap version 36 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f51b5a02740 1 -- 172.21.15.37:0/194935648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56365c707610 con 0x56365c6efd40 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f518bfff640 1 -- 172.21.15.37:0/194935648 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51a0005500 con 0x56365c6efd40 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f518bfff640 1 -- 172.21.15.37:0/194935648 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5198084ce0 con 0x56365c6efd40 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f51b5a02740 10 client.15867 fetch_fsmap finished waiting for FSMap version 36 2024-03-26T06:27:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f51b5a02740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f51b5a02740 20 client.15867 populate_metadata read hostname 'smithi037' 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f51b5a02740 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56365c708a80 0x56365c70ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f51b5a02740 4 client.15867 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f51b5a02740 1 -- 172.21.15.37:0/194935648 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56365c660b30 con 0x56365c708a80 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.492+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56365c708a80 0x56365c70ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56365c708a80 0x56365c70ae70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d61d5114e8ac5b server_cookie=d2823d46ff0473f1 in_seq=0 out_seq=0 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f518bfff640 10 client.15867 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f518bfff640 1 -- 172.21.15.37:0/194935648 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56365c660b30 con 0x56365c708a80 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f518bfff640 10 client.15867 handle_command_reply: tid=0 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f51b5a02740 1 client.15867 shutdown 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f51b5a02740 1 -- 172.21.15.37:0/194935648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 msgr2=0x56365c6f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f51b5a02740 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f5b00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f51980cc8f0 tx=0x7f519809b630 comp rx=0 tx=0).stop 2024-03-26T06:27:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f51b5a02740 1 -- 172.21.15.37:0/194935648 shutdown_connections 2024-03-26T06:27:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f51b5a02740 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56365c708a80 0x56365c70ae70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f51b5a02740 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6f8550 0x56365c6f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f51b5a02740 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f5b00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f51b5a02740 1 --2- 172.21.15.37:0/194935648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6eefd0 0x56365c6f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f51b5a02740 1 -- 172.21.15.37:0/194935648 >> 172.21.15.37:0/194935648 conn(0x56365c6b65b0 msgr2=0x56365c6eba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f518bfff640 0 client.15867 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.493+0000 7f51b5a02740 1 -- 172.21.15.37:0/194935648 shutdown_connections 2024-03-26T06:27:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.494+0000 7f51b5a02740 1 -- 172.21.15.37:0/194935648 wait complete. 2024-03-26T06:27:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.494+0000 7f51b5a02740 20 client.15867 trim_cache size 0 max 16384 2024-03-26T06:27:37.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 127101 3.88 MB/sec execute 72 sec latency 12.092 ms 2024-03-26T06:27:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.507+0000 7f51b5a02740 1 Processor -- start 2024-03-26T06:27:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.507+0000 7f51b5a02740 1 -- start start 2024-03-26T06:27:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.507+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6eefd0 0x56365c6f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.507+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.507+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6f8550 0x56365c6f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.507+0000 7f51b5a02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1aa980 con 0x56365c6efd40 2024-03-26T06:27:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.507+0000 7f51b5a02740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1ac360 con 0x56365c6eefd0 2024-03-26T06:27:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.507+0000 7f51b5a02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1aafe0 con 0x56365c6f8550 2024-03-26T06:27:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51b0bbc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6eefd0 0x56365c6f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60910/0 (socket says 172.21.15.37:60910) 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51abfff640 1 -- 172.21.15.37:0/428143233 learned_addr learned my addr 172.21.15.37:0/428143233 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51b13bd640 1 --2- 172.21.15.37:0/428143233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6f8550 0x56365c6f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51b0bbc640 1 -- 172.21.15.37:0/428143233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6f8550 msgr2=0x56365c6f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/428143233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6f8550 0x56365c6f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51b0bbc640 1 -- 172.21.15.37:0/428143233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 msgr2=0x56365c6f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/428143233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51b0bbc640 1 -- 172.21.15.37:0/428143233 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56365c710080 con 0x56365c6eefd0 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51abfff640 1 --2- 172.21.15.37:0/428143233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/428143233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6eefd0 0x56365c6f7cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x56365c70b7a0 tx=0x7f51a0004dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d0cd546d6db57c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51967fc640 1 -- 172.21.15.37:0/428143233 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0032070 con 0x56365c6eefd0 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51967fc640 1 -- 172.21.15.37:0/428143233 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51a0005850 con 0x56365c6eefd0 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.508+0000 7f51967fc640 1 -- 172.21.15.37:0/428143233 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0004640 con 0x56365c6eefd0 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.509+0000 7f51b5a02740 1 -- 172.21.15.37:0/428143233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6eefd0 msgr2=0x56365c6f7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.509+0000 7f51b5a02740 1 --2- 172.21.15.37:0/428143233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6eefd0 0x56365c6f7cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x56365c70b7a0 tx=0x7f51a0004dd0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.509+0000 7f51b5a02740 1 -- 172.21.15.37:0/428143233 shutdown_connections 2024-03-26T06:27:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.509+0000 7f51b5a02740 1 --2- 172.21.15.37:0/428143233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6f8550 0x56365c6f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.509+0000 7f51b5a02740 1 --2- 172.21.15.37:0/428143233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.509+0000 7f51b5a02740 1 --2- 172.21.15.37:0/428143233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6eefd0 0x56365c6f7cf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.509+0000 7f51b5a02740 1 -- 172.21.15.37:0/428143233 >> 172.21.15.37:0/428143233 conn(0x56365c6b65b0 msgr2=0x56365c6e97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.509+0000 7f51b5a02740 1 -- 172.21.15.37:0/428143233 shutdown_connections 2024-03-26T06:27:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.509+0000 7f51b5a02740 1 -- 172.21.15.37:0/428143233 wait complete. 2024-03-26T06:27:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.509+0000 7f51b5a02740 1 -- start start 2024-03-26T06:27:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.509+0000 7f51b5a02740 1 Processor -- start 2024-03-26T06:27:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6eefd0 0x56365c6ff890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51b5a02740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6f8550 0x56365c6fe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51b5a02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1abd00 con 0x56365c6efd40 2024-03-26T06:27:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51b5a02740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1ab6a0 con 0x56365c6f8550 2024-03-26T06:27:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51b5a02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1a8c20 con 0x56365c6eefd0 2024-03-26T06:27:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51b5a02740 10 client.0 _ll_register_callbacks cb 0x7f51a0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:27:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6fdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60912/0 (socket says 172.21.15.37:60912) 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51abfff640 1 -- 172.21.15.37:0/2758203894 learned_addr learned my addr 172.21.15.37:0/2758203894 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51b13bd640 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6f8550 0x56365c6fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6eefd0 0x56365c6ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51abfff640 1 -- 172.21.15.37:0/2758203894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6eefd0 msgr2=0x56365c6ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51abfff640 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6eefd0 0x56365c6ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51abfff640 1 -- 172.21.15.37:0/2758203894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6f8550 msgr2=0x56365c6fe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51abfff640 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6f8550 0x56365c6fe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51abfff640 1 -- 172.21.15.37:0/2758203894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56365c6fe830 con 0x56365c6efd40 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51b13bd640 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6f8550 0x56365c6fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f51abfff640 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6fdc30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51980973d0 tx=0x7f5198099cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4c50f9ef82163e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.510+0000 7f518bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f518bfff640 1 -- 172.21.15.37:0/2758203894 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519809b100 con 0x56365c6efd40 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f518bfff640 1 -- 172.21.15.37:0/2758203894 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f5198097e10 con 0x56365c6efd40 2024-03-26T06:27:37.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f51b5a02740 1 -- 172.21.15.37:0/2758203894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51a0005850 con 0x56365c6efd40 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f518bfff640 1 -- 172.21.15.37:0/2758203894 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5198097a40 con 0x56365c6efd40 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f51b5a02740 10 client.15873 fetch_fsmap learned FSMap version 36 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f51b5a02740 1 -- 172.21.15.37:0/2758203894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56365c75feb0 con 0x56365c6efd40 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f518bfff640 1 -- 172.21.15.37:0/2758203894 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51a0005850 con 0x56365c6efd40 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f518bfff640 1 -- 172.21.15.37:0/2758203894 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f519809abf0 con 0x56365c6efd40 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f51b5a02740 10 client.15873 fetch_fsmap finished waiting for FSMap version 36 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f51b5a02740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f51b5a02740 20 client.15873 populate_metadata read hostname 'smithi037' 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f51b5a02740 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56365c760580 0x56365c7629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f51b5a02740 4 client.15873 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f51b5a02740 1 -- 172.21.15.37:0/2758203894 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56365c65ae80 con 0x56365c760580 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.511+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56365c760580 0x56365c7629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.512+0000 7f51b0bbc640 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56365c760580 0x56365c7629a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a0c1af61962cbd server_cookie=21f8c5dc9f29ad62 in_seq=0 out_seq=0 2024-03-26T06:27:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.512+0000 7f518bfff640 10 client.15873 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f518bfff640 1 -- 172.21.15.37:0/2758203894 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56365c65ae80 con 0x56365c760580 2024-03-26T06:27:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f518bfff640 10 client.15873 handle_command_reply: tid=0 2024-03-26T06:27:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 1 client.15873 shutdown 2024-03-26T06:27:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 1 -- 172.21.15.37:0/2758203894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 msgr2=0x56365c6fdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6fdc30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51980973d0 tx=0x7f5198099cb0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 1 -- 172.21.15.37:0/2758203894 shutdown_connections 2024-03-26T06:27:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56365c760580 0x56365c7629a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56365c6f8550 0x56365c6fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56365c6efd40 0x56365c6fdc30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f518bfff640 0 client.15873 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 1 --2- 172.21.15.37:0/2758203894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56365c6eefd0 0x56365c6ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 1 -- 172.21.15.37:0/2758203894 >> 172.21.15.37:0/2758203894 conn(0x56365c6b65b0 msgr2=0x56365c6f3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 1 -- 172.21.15.37:0/2758203894 shutdown_connections 2024-03-26T06:27:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 1 -- 172.21.15.37:0/2758203894 wait complete. 2024-03-26T06:27:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.545+0000 7f51b5a02740 20 client.15873 trim_cache size 0 max 16384 2024-03-26T06:27:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:27:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:27:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:27:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:27:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 -- 172.21.15.37:0/3788387002 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f519c0a2a10 msgr2=0x7f519c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f519c0a2a10 0x7f519c0a4ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f51a4004af0 tx=0x7f51a40323c0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 -- 172.21.15.37:0/3788387002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac1065e0 msgr2=0x7f51ac19dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac1065e0 0x7f51ac19dcb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f519800e9f0 tx=0x7f519800eeb0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 -- 172.21.15.37:0/3788387002 shutdown_connections 2024-03-26T06:27:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f519c0a2a10 0x7f519c0a4ed0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac197920 0x7f51ac197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51ac1065e0 0x7f51ac19dcb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 --2- 172.21.15.37:0/3788387002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51ac101510 0x7f51ac19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 -- 172.21.15.37:0/3788387002 >> 172.21.15.37:0/3788387002 conn(0x7f51ac0fcdb0 msgr2=0x7f51ac0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 -- 172.21.15.37:0/3788387002 shutdown_connections 2024-03-26T06:27:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:37.546+0000 7f51b2e20640 1 -- 172.21.15.37:0/3788387002 wait complete. 2024-03-26T06:27:38.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 128449 3.92 MB/sec execute 73 sec latency 67.412 ms 2024-03-26T06:27:38.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:38 smithi149 ceph-mon[36681]: pgmap v142: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 733 KiB/s wr, 95 op/s 2024-03-26T06:27:38.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:38 smithi037 ceph-mon[31974]: pgmap v142: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 733 KiB/s wr, 95 op/s 2024-03-26T06:27:39.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:38 smithi066 ceph-mon[36504]: pgmap v142: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 733 KiB/s wr, 95 op/s 2024-03-26T06:27:39.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 128449 3.87 MB/sec execute 74 sec latency 1067.522 ms 2024-03-26T06:27:40.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 128449 3.82 MB/sec execute 75 sec latency 2067.639 ms 2024-03-26T06:27:40.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:40 smithi149 ceph-mon[36681]: pgmap v143: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 733 KiB/s wr, 95 op/s 2024-03-26T06:27:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:40 smithi037 ceph-mon[31974]: pgmap v143: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 733 KiB/s wr, 95 op/s 2024-03-26T06:27:41.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:40 smithi066 ceph-mon[36504]: pgmap v143: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 733 KiB/s wr, 95 op/s 2024-03-26T06:27:41.498 INFO:tasks.workunit.client.0.smithi037.stdout: 1 128777 3.79 MB/sec execute 76 sec latency 2873.203 ms 2024-03-26T06:27:42.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 131485 3.96 MB/sec execute 77 sec latency 13.583 ms 2024-03-26T06:27:42.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:42 smithi149 ceph-mon[36681]: pgmap v144: 97 pgs: 97 active+clean; 54 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-03-26T06:27:42.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:42 smithi037 ceph-mon[31974]: pgmap v144: 97 pgs: 97 active+clean; 54 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-03-26T06:27:43.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:42 smithi066 ceph-mon[36504]: pgmap v144: 97 pgs: 97 active+clean; 54 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-03-26T06:27:43.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 132968 3.98 MB/sec execute 78 sec latency 34.401 ms 2024-03-26T06:27:44.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 135194 4.11 MB/sec execute 79 sec latency 12.403 ms 2024-03-26T06:27:44.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:44 smithi149 ceph-mon[36681]: pgmap v145: 97 pgs: 97 active+clean; 54 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-03-26T06:27:44.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:44 smithi037 ceph-mon[31974]: pgmap v145: 97 pgs: 97 active+clean; 54 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-03-26T06:27:45.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:44 smithi066 ceph-mon[36504]: pgmap v145: 97 pgs: 97 active+clean; 54 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2024-03-26T06:27:45.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 135487 4.06 MB/sec execute 80 sec latency 702.043 ms 2024-03-26T06:27:46.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 135611 4.01 MB/sec execute 81 sec latency 1514.337 ms 2024-03-26T06:27:46.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:46 smithi149 ceph-mon[36681]: pgmap v146: 97 pgs: 97 active+clean; 58 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-03-26T06:27:46.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:46 smithi037 ceph-mon[31974]: pgmap v146: 97 pgs: 97 active+clean; 58 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-03-26T06:27:47.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:46 smithi066 ceph-mon[36504]: pgmap v146: 97 pgs: 97 active+clean; 58 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-03-26T06:27:47.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 135611 3.96 MB/sec execute 82 sec latency 1103.518 ms 2024-03-26T06:27:48.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 135611 3.91 MB/sec execute 83 sec latency 2103.607 ms 2024-03-26T06:27:48.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:48 smithi149 ceph-mon[36681]: pgmap v147: 97 pgs: 97 active+clean; 58 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-03-26T06:27:48.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:48 smithi037 ceph-mon[31974]: pgmap v147: 97 pgs: 97 active+clean; 58 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-03-26T06:27:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:48 smithi066 ceph-mon[36504]: pgmap v147: 97 pgs: 97 active+clean; 58 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-03-26T06:27:49.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 135611 3.87 MB/sec execute 84 sec latency 3103.703 ms 2024-03-26T06:27:50.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 135611 3.82 MB/sec execute 85 sec latency 4103.748 ms 2024-03-26T06:27:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.645+0000 7fabc9ed1640 1 -- 172.21.15.37:0/2190412042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc4102df0 msgr2=0x7fabc410b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.645+0000 7fabc9ed1640 1 --2- 172.21.15.37:0/2190412042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc4102df0 0x7fabc410b2d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009900 tx=0x7fabb802bc30 comp rx=0 tx=0).stop 2024-03-26T06:27:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.645+0000 7fabc9ed1640 1 -- 172.21.15.37:0/2190412042 shutdown_connections 2024-03-26T06:27:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.645+0000 7fabc9ed1640 1 --2- 172.21.15.37:0/2190412042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc410b940 0x7fabc410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.645+0000 7fabc9ed1640 1 --2- 172.21.15.37:0/2190412042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc4102df0 0x7fabc410b2d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.645+0000 7fabc9ed1640 1 --2- 172.21.15.37:0/2190412042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc41024d0 0x7fabc41028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.645+0000 7fabc9ed1640 1 -- 172.21.15.37:0/2190412042 >> 172.21.15.37:0/2190412042 conn(0x7fabc4077320 msgr2=0x7fabc4077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.645+0000 7fabc9ed1640 1 -- 172.21.15.37:0/2190412042 shutdown_connections 2024-03-26T06:27:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.645+0000 7fabc9ed1640 1 -- 172.21.15.37:0/2190412042 wait complete. 2024-03-26T06:27:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.645+0000 7fabc9ed1640 1 Processor -- start 2024-03-26T06:27:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.645+0000 7fabc9ed1640 1 -- start start 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc9ed1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc41024d0 0x7fabc419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc9ed1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc4102df0 0x7fabc419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc9ed1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc410b940 0x7fabc41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc9ed1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41116d0 con 0x7fabc4102df0 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc9ed1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4111550 con 0x7fabc41024d0 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc9ed1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4111850 con 0x7fabc410b940 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc4102df0 0x7fabc419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc4102df0 0x7fabc419dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54656/0 (socket says 172.21.15.37:54656) 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc2ffd640 1 -- 172.21.15.37:0/3047665452 learned_addr learned my addr 172.21.15.37:0/3047665452 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc37fe640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc41024d0 0x7fabc419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc3fff640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc410b940 0x7fabc41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc2ffd640 1 -- 172.21.15.37:0/3047665452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc410b940 msgr2=0x7fabc41978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc410b940 0x7fabc41978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc2ffd640 1 -- 172.21.15.37:0/3047665452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc41024d0 msgr2=0x7fabc419d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc41024d0 0x7fabc419d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc2ffd640 1 -- 172.21.15.37:0/3047665452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc4198070 con 0x7fabc4102df0 2024-03-26T06:27:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc37fe640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc41024d0 0x7fabc419d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:27:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.646+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc4102df0 0x7fabc419dc30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fabb8006010 tx=0x7fabb802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.647+0000 7fabc0ff9640 1 -- 172.21.15.37:0/3047665452 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8041070 con 0x7fabc4102df0 2024-03-26T06:27:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.647+0000 7fabc0ff9640 1 -- 172.21.15.37:0/3047665452 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabb8004450 con 0x7fabc4102df0 2024-03-26T06:27:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.647+0000 7fabc9ed1640 1 -- 172.21.15.37:0/3047665452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc4198300 con 0x7fabc4102df0 2024-03-26T06:27:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.647+0000 7fabc0ff9640 1 -- 172.21.15.37:0/3047665452 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8034440 con 0x7fabc4102df0 2024-03-26T06:27:50.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.647+0000 7fabc9ed1640 1 -- 172.21.15.37:0/3047665452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc41a45b0 con 0x7fabc4102df0 2024-03-26T06:27:50.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.649+0000 7fabc0ff9640 1 -- 172.21.15.37:0/3047665452 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fabb8044070 con 0x7fabc4102df0 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.649+0000 7fabc0ff9640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fabac0a2a10 0x7fabac0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.649+0000 7fabccb90740 1 Processor -- start 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.649+0000 7fabccb90740 1 -- start start 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc37fe640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fabac0a2a10 0x7fabac0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc0ff9640 1 -- 172.21.15.37:0/3047665452 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fabb80f0a20 con 0x7fabc4102df0 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabccb90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052461d40 0x558052460a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabccb90740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052460fd0 0x558052461450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabccb90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55805246e960 0x558052470d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabccb90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580523c55f0 con 0x558052461d40 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabccb90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805236e930 con 0x55805246e960 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabccb90740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580523c1690 con 0x558052460fd0 2024-03-26T06:27:50.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052461d40 0x558052460a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052461d40 0x558052460a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54658/0 (socket says 172.21.15.37:54658) 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc37fe640 1 -- 172.21.15.37:0/1087889168 learned_addr learned my addr 172.21.15.37:0/1087889168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc37fe640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fabac0a2a10 0x7fabac0a4ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fabb4005fd0 tx=0x7fabb400e720 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/1087889168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052460fd0 0x558052461450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc3fff640 1 --2- 172.21.15.37:0/1087889168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55805246e960 0x558052470d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc37fe640 1 -- 172.21.15.37:0/1087889168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052460fd0 msgr2=0x558052461450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc37fe640 1 --2- 172.21.15.37:0/1087889168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052460fd0 0x558052461450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc37fe640 1 -- 172.21.15.37:0/1087889168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55805246e960 msgr2=0x558052470d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc37fe640 1 --2- 172.21.15.37:0/1087889168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55805246e960 0x558052470d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc37fe640 1 -- 172.21.15.37:0/1087889168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580520da210 con 0x558052461d40 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/1087889168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052460fd0 0x558052461450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.650+0000 7fabc3fff640 1 --2- 172.21.15.37:0/1087889168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55805246e960 0x558052470d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7fabc37fe640 1 --2- 172.21.15.37:0/1087889168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052461d40 0x558052460a90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fabb400af30 tx=0x7fabb4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504285ee3dc4d9b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:50.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7faba9ffb640 1 -- 172.21.15.37:0/1087889168 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb401d840 con 0x558052461d40 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7faba9ffb640 1 -- 172.21.15.37:0/1087889168 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabb4010420 con 0x558052461d40 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7faba9ffb640 1 -- 172.21.15.37:0/1087889168 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb400f620 con 0x558052461d40 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7fabccb90740 1 -- 172.21.15.37:0/1087889168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052461d40 msgr2=0x558052460a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7fabccb90740 1 --2- 172.21.15.37:0/1087889168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052461d40 0x558052460a90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fabb400af30 tx=0x7fabb4016000 comp rx=0 tx=0).stop 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7fabccb90740 1 -- 172.21.15.37:0/1087889168 shutdown_connections 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7fabccb90740 1 --2- 172.21.15.37:0/1087889168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55805246e960 0x558052470d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7fabccb90740 1 --2- 172.21.15.37:0/1087889168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052460fd0 0x558052461450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7fabccb90740 1 --2- 172.21.15.37:0/1087889168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052461d40 0x558052460a90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7fabccb90740 1 -- 172.21.15.37:0/1087889168 >> 172.21.15.37:0/1087889168 conn(0x5580524285b0 msgr2=0x55805245bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.651+0000 7fabccb90740 1 -- 172.21.15.37:0/1087889168 shutdown_connections 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabccb90740 1 -- 172.21.15.37:0/1087889168 wait complete. 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabccb90740 1 -- start start 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabccb90740 1 Processor -- start 2024-03-26T06:27:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabccb90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabccb90740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052461d40 0x55805246bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabccb90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55805246e960 0x55805246c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabccb90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558052471690 con 0x558052460fd0 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabccb90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558052381d90 con 0x55805246e960 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabccb90740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558052471810 con 0x558052461d40 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabccb90740 10 client.0 _ll_register_callbacks cb 0x5580520d9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.652+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54662/0 (socket says 172.21.15.37:54662) 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabc37fe640 1 -- 172.21.15.37:0/3266537435 learned_addr learned my addr 172.21.15.37:0/3266537435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabc3fff640 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55805246e960 0x55805246c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052461d40 0x55805246bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabc37fe640 1 -- 172.21.15.37:0/3266537435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052461d40 msgr2=0x55805246bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabc37fe640 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052461d40 0x55805246bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabc37fe640 1 -- 172.21.15.37:0/3266537435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55805246e960 msgr2=0x55805246c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabc37fe640 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55805246e960 0x55805246c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabc37fe640 1 -- 172.21.15.37:0/3266537435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580521a6620 con 0x558052460fd0 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052461d40 0x55805246bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabc37fe640 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246b980 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fabb401e880 tx=0x7fabb401e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f39dd1502de0587 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7faba2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7faba2ffd640 1 -- 172.21.15.37:0/3266537435 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4034060 con 0x558052460fd0 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7faba2ffd640 1 -- 172.21.15.37:0/3266537435 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fabb401ece0 con 0x558052460fd0 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabccb90740 1 -- 172.21.15.37:0/3266537435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabb4010420 con 0x558052460fd0 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7faba2ffd640 1 -- 172.21.15.37:0/3266537435 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabb4005050 con 0x558052460fd0 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabccb90740 10 client.15885 fetch_fsmap learned FSMap version 36 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.653+0000 7fabccb90740 1 -- 172.21.15.37:0/3266537435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55805247d980 con 0x558052460fd0 2024-03-26T06:27:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.654+0000 7faba2ffd640 1 -- 172.21.15.37:0/3266537435 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabb4010420 con 0x558052460fd0 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.654+0000 7faba2ffd640 1 -- 172.21.15.37:0/3266537435 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fabb4004990 con 0x558052460fd0 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.654+0000 7fabccb90740 10 client.15885 fetch_fsmap finished waiting for FSMap version 36 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.654+0000 7fabccb90740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.654+0000 7fabccb90740 20 client.15885 populate_metadata read hostname 'smithi037' 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.654+0000 7fabccb90740 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55805247ed80 0x558052481170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.654+0000 7fabccb90740 4 client.15885 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.654+0000 7fabccb90740 1 -- 172.21.15.37:0/3266537435 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580523d2b30 con 0x55805247ed80 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.654+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55805247ed80 0x558052481170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7faba2ffd640 10 client.15885 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55805247ed80 0x558052481170 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871c682d9787bbc9 server_cookie=8bd25eae1144c343 in_seq=0 out_seq=0 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7faba2ffd640 1 -- 172.21.15.37:0/3266537435 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5580523d2b30 con 0x55805247ed80 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7faba2ffd640 10 client.15885 handle_command_reply: tid=0 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7fabccb90740 1 client.15885 shutdown 2024-03-26T06:27:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7fabccb90740 1 -- 172.21.15.37:0/3266537435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 msgr2=0x55805246b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7fabccb90740 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246b980 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fabb401e880 tx=0x7fabb401e8b0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7fabccb90740 1 -- 172.21.15.37:0/3266537435 shutdown_connections 2024-03-26T06:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7fabccb90740 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55805247ed80 0x558052481170 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7fabccb90740 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55805246e960 0x55805246c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7fabccb90740 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558052461d40 0x55805246bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7fabccb90740 1 --2- 172.21.15.37:0/3266537435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246b980 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7faba2ffd640 0 client.15885 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.655+0000 7fabccb90740 1 -- 172.21.15.37:0/3266537435 >> 172.21.15.37:0/3266537435 conn(0x5580524285b0 msgr2=0x55805245da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.656+0000 7fabccb90740 1 -- 172.21.15.37:0/3266537435 shutdown_connections 2024-03-26T06:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.656+0000 7fabccb90740 1 -- 172.21.15.37:0/3266537435 wait complete. 2024-03-26T06:27:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.656+0000 7fabccb90740 20 client.15885 trim_cache size 0 max 16384 2024-03-26T06:27:50.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabccb90740 1 Processor -- start 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabccb90740 1 -- start start 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabccb90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabccb90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558052461d40 0x558052468c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabccb90740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55805246e960 0x558052469140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabccb90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41aa8c0 con 0x558052460fd0 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabccb90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41ac2a0 con 0x558052461d40 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabccb90740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41aaf20 con 0x55805246e960 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54674/0 (socket says 172.21.15.37:54674) 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabc37fe640 1 -- 172.21.15.37:0/4178202720 learned_addr learned my addr 172.21.15.37:0/4178202720 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/4178202720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558052461d40 0x558052468c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.669+0000 7fabc3fff640 1 --2- 172.21.15.37:0/4178202720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55805246e960 0x558052469140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.670+0000 7fabc37fe640 1 -- 172.21.15.37:0/4178202720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55805246e960 msgr2=0x558052469140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.670+0000 7fabc37fe640 1 --2- 172.21.15.37:0/4178202720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55805246e960 0x558052469140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.670+0000 7fabc37fe640 1 -- 172.21.15.37:0/4178202720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558052461d40 msgr2=0x558052468c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.670+0000 7fabc37fe640 1 --2- 172.21.15.37:0/4178202720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558052461d40 0x558052468c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.670+0000 7fabc37fe640 1 -- 172.21.15.37:0/4178202720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55805247a9d0 con 0x558052460fd0 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.670+0000 7fabc37fe640 1 --2- 172.21.15.37:0/4178202720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246e0b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fabb400ef40 tx=0x7fabb400af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8754e40b8f3e242f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.670+0000 7faba97fa640 1 -- 172.21.15.37:0/4178202720 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4034040 con 0x558052460fd0 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.670+0000 7faba97fa640 1 -- 172.21.15.37:0/4178202720 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabb4010420 con 0x558052460fd0 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.670+0000 7faba97fa640 1 -- 172.21.15.37:0/4178202720 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40048e0 con 0x558052460fd0 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 -- 172.21.15.37:0/4178202720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 msgr2=0x55805246e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 --2- 172.21.15.37:0/4178202720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246e0b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fabb400ef40 tx=0x7fabb400af00 comp rx=0 tx=0).stop 2024-03-26T06:27:50.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 -- 172.21.15.37:0/4178202720 shutdown_connections 2024-03-26T06:27:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 --2- 172.21.15.37:0/4178202720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55805246e960 0x558052469140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 --2- 172.21.15.37:0/4178202720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558052461d40 0x558052468c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 --2- 172.21.15.37:0/4178202720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x55805246e0b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 -- 172.21.15.37:0/4178202720 >> 172.21.15.37:0/4178202720 conn(0x5580524285b0 msgr2=0x55805246a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 -- 172.21.15.37:0/4178202720 shutdown_connections 2024-03-26T06:27:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 -- 172.21.15.37:0/4178202720 wait complete. 2024-03-26T06:27:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 -- start start 2024-03-26T06:27:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 Processor -- start 2024-03-26T06:27:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x558052474760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.671+0000 7fabccb90740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558052461d40 0x558052474ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabccb90740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55805246e960 0x5580524751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabccb90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41abc40 con 0x558052460fd0 2024-03-26T06:27:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabccb90740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41ab5e0 con 0x558052461d40 2024-03-26T06:27:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabccb90740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41a8b60 con 0x55805246e960 2024-03-26T06:27:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabccb90740 10 client.0 _ll_register_callbacks cb 0x7fabb4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:27:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x558052474760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x558052474760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54682/0 (socket says 172.21.15.37:54682) 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabc37fe640 1 -- 172.21.15.37:0/2951717004 learned_addr learned my addr 172.21.15.37:0/2951717004 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558052461d40 0x558052474ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabc3fff640 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55805246e960 0x5580524751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabc37fe640 1 -- 172.21.15.37:0/2951717004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55805246e960 msgr2=0x5580524751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabc37fe640 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55805246e960 0x5580524751e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabc37fe640 1 -- 172.21.15.37:0/2951717004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558052461d40 msgr2=0x558052474ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabc37fe640 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558052461d40 0x558052474ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabc37fe640 1 -- 172.21.15.37:0/2951717004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580524d5130 con 0x558052460fd0 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabc37fe640 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x558052474760 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fabb4010e50 tx=0x7fabb40255c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e82c2e33b8827ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7faba2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7faba2ffd640 1 -- 172.21.15.37:0/2951717004 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4046070 con 0x558052460fd0 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7faba2ffd640 1 -- 172.21.15.37:0/2951717004 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fabb4004890 con 0x558052460fd0 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabccb90740 1 -- 172.21.15.37:0/2951717004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabb4010420 con 0x558052460fd0 2024-03-26T06:27:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7faba2ffd640 1 -- 172.21.15.37:0/2951717004 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabb4051050 con 0x558052460fd0 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.672+0000 7fabccb90740 10 client.15891 fetch_fsmap learned FSMap version 36 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.673+0000 7fabccb90740 1 -- 172.21.15.37:0/2951717004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5580524d6440 con 0x558052460fd0 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.673+0000 7faba2ffd640 1 -- 172.21.15.37:0/2951717004 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabb4010420 con 0x558052460fd0 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.673+0000 7faba2ffd640 1 -- 172.21.15.37:0/2951717004 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fabb4041070 con 0x558052460fd0 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.673+0000 7fabccb90740 10 client.15891 fetch_fsmap finished waiting for FSMap version 36 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.673+0000 7fabccb90740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.673+0000 7fabccb90740 20 client.15891 populate_metadata read hostname 'smithi037' 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.673+0000 7fabccb90740 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5580524d6a00 0x5580524d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.673+0000 7fabccb90740 4 client.15891 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.673+0000 7fabccb90740 1 -- 172.21.15.37:0/2951717004 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5580523cce80 con 0x5580524d6a00 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.673+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5580524d6a00 0x5580524d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.674+0000 7faba2ffd640 10 client.15891 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.674+0000 7fabc2ffd640 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5580524d6a00 0x5580524d8e20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bdb0ba304471a86 server_cookie=877a710a513f2166 in_seq=0 out_seq=0 2024-03-26T06:27:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.708+0000 7faba2ffd640 1 -- 172.21.15.37:0/2951717004 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5580523cce80 con 0x5580524d6a00 2024-03-26T06:27:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.708+0000 7faba2ffd640 10 client.15891 handle_command_reply: tid=0 2024-03-26T06:27:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.708+0000 7fabccb90740 1 client.15891 shutdown 2024-03-26T06:27:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabccb90740 1 -- 172.21.15.37:0/2951717004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 msgr2=0x558052474760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabccb90740 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x558052474760 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fabb4010e50 tx=0x7fabb40255c0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabccb90740 1 -- 172.21.15.37:0/2951717004 shutdown_connections 2024-03-26T06:27:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabccb90740 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5580524d6a00 0x5580524d8e20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabccb90740 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55805246e960 0x5580524751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabccb90740 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558052461d40 0x558052474ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7faba2ffd640 0 client.15891 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:27:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabccb90740 1 --2- 172.21.15.37:0/2951717004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558052460fd0 0x558052474760 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabccb90740 1 -- 172.21.15.37:0/2951717004 >> 172.21.15.37:0/2951717004 conn(0x5580524285b0 msgr2=0x55805245a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabccb90740 1 -- 172.21.15.37:0/2951717004 shutdown_connections 2024-03-26T06:27:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabccb90740 1 -- 172.21.15.37:0/2951717004 wait complete. 2024-03-26T06:27:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabccb90740 20 client.15891 trim_cache size 0 max 16384 2024-03-26T06:27:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:27:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:27:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:27:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:27:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabc9ed1640 1 -- 172.21.15.37:0/3047665452 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fabac0a2a10 msgr2=0x7fabac0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabc9ed1640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fabac0a2a10 0x7fabac0a4ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fabb4005fd0 tx=0x7fabb400e720 comp rx=0 tx=0).stop 2024-03-26T06:27:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabc9ed1640 1 -- 172.21.15.37:0/3047665452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc4102df0 msgr2=0x7fabc419dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:27:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabc9ed1640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc4102df0 0x7fabc419dc30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fabb8006010 tx=0x7fabb802cc70 comp rx=0 tx=0).stop 2024-03-26T06:27:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabc9ed1640 1 -- 172.21.15.37:0/3047665452 shutdown_connections 2024-03-26T06:27:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabc9ed1640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fabac0a2a10 0x7fabac0a4ed0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabc9ed1640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc410b940 0x7fabc41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabc9ed1640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc4102df0 0x7fabc419dc30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabc9ed1640 1 --2- 172.21.15.37:0/3047665452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fabc41024d0 0x7fabc419d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:27:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.709+0000 7fabc9ed1640 1 -- 172.21.15.37:0/3047665452 >> 172.21.15.37:0/3047665452 conn(0x7fabc4077320 msgr2=0x7fabc40fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:27:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.710+0000 7fabc9ed1640 1 -- 172.21.15.37:0/3047665452 shutdown_connections 2024-03-26T06:27:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:27:50.710+0000 7fabc9ed1640 1 -- 172.21.15.37:0/3047665452 wait complete. 2024-03-26T06:27:50.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:50 smithi149 ceph-mon[36681]: pgmap v148: 97 pgs: 97 active+clean; 58 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-03-26T06:27:50.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:50 smithi037 ceph-mon[31974]: pgmap v148: 97 pgs: 97 active+clean; 58 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-03-26T06:27:51.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:50 smithi066 ceph-mon[36504]: pgmap v148: 97 pgs: 97 active+clean; 58 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-03-26T06:27:51.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 135984 3.80 MB/sec execute 86 sec latency 4920.989 ms 2024-03-26T06:27:52.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 138589 3.94 MB/sec execute 87 sec latency 18.668 ms 2024-03-26T06:27:52.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:52 smithi149 ceph-mon[36681]: pgmap v149: 97 pgs: 97 active+clean; 59 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-03-26T06:27:52.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:52 smithi037 ceph-mon[31974]: pgmap v149: 97 pgs: 97 active+clean; 59 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-03-26T06:27:53.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:52 smithi066 ceph-mon[36504]: pgmap v149: 97 pgs: 97 active+clean; 59 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-03-26T06:27:53.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 139090 3.91 MB/sec execute 88 sec latency 449.527 ms 2024-03-26T06:27:54.499 INFO:tasks.workunit.client.0.smithi037.stdout: 1 139090 3.86 MB/sec execute 89 sec latency 1449.571 ms 2024-03-26T06:27:54.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:54 smithi149 ceph-mon[36681]: pgmap v150: 97 pgs: 97 active+clean; 59 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-03-26T06:27:54.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:54 smithi037 ceph-mon[31974]: pgmap v150: 97 pgs: 97 active+clean; 59 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-03-26T06:27:55.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:54 smithi066 ceph-mon[36504]: pgmap v150: 97 pgs: 97 active+clean; 59 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-03-26T06:27:55.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 139090 3.82 MB/sec execute 90 sec latency 2449.643 ms 2024-03-26T06:27:56.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 139551 3.80 MB/sec execute 91 sec latency 3102.443 ms 2024-03-26T06:27:56.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:56 smithi149 ceph-mon[36681]: pgmap v151: 97 pgs: 97 active+clean; 55 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-03-26T06:27:56.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:56 smithi037 ceph-mon[31974]: pgmap v151: 97 pgs: 97 active+clean; 55 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-03-26T06:27:57.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:56 smithi066 ceph-mon[36504]: pgmap v151: 97 pgs: 97 active+clean; 55 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-03-26T06:27:57.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 142146 3.94 MB/sec execute 92 sec latency 15.316 ms 2024-03-26T06:27:58.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 142577 3.90 MB/sec execute 93 sec latency 550.685 ms 2024-03-26T06:27:58.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:27:58 smithi149 ceph-mon[36681]: pgmap v152: 97 pgs: 97 active+clean; 55 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-03-26T06:27:58.967 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:27:58 smithi037 ceph-mon[31974]: pgmap v152: 97 pgs: 97 active+clean; 55 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-03-26T06:27:59.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:27:58 smithi066 ceph-mon[36504]: pgmap v152: 97 pgs: 97 active+clean; 55 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-03-26T06:27:59.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 142577 3.86 MB/sec execute 94 sec latency 1550.766 ms 2024-03-26T06:28:00.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 142577 3.82 MB/sec execute 95 sec latency 2550.867 ms 2024-03-26T06:28:00.900 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:00 smithi066 ceph-mon[36504]: pgmap v153: 97 pgs: 97 active+clean; 55 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-03-26T06:28:00.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:00 smithi149 ceph-mon[36681]: pgmap v153: 97 pgs: 97 active+clean; 55 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-03-26T06:28:00.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:00 smithi037 ceph-mon[31974]: pgmap v153: 97 pgs: 97 active+clean; 55 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-03-26T06:28:01.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 144288 3.85 MB/sec execute 96 sec latency 2646.697 ms 2024-03-26T06:28:01.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:01 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:01.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:01 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:02.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:01 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:02.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 145916 3.93 MB/sec execute 97 sec latency 10.667 ms 2024-03-26T06:28:02.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:02 smithi149 ceph-mon[36681]: pgmap v154: 97 pgs: 97 active+clean; 57 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-03-26T06:28:02.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:02 smithi037 ceph-mon[31974]: pgmap v154: 97 pgs: 97 active+clean; 57 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-03-26T06:28:03.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:02 smithi066 ceph-mon[36504]: pgmap v154: 97 pgs: 97 active+clean; 57 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-03-26T06:28:03.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 146250 3.90 MB/sec execute 98 sec latency 764.951 ms 2024-03-26T06:28:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.807+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1460288388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c105170 msgr2=0x7f7d5c105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.807+0000 7f7d60d3b640 1 --2- 172.21.15.37:0/1460288388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c105170 0x7f7d5c105550 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009900 tx=0x7f7d4c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:28:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.807+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1460288388 shutdown_connections 2024-03-26T06:28:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.807+0000 7f7d60d3b640 1 --2- 172.21.15.37:0/1460288388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d5c100de0 0x7f7d5c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.807+0000 7f7d60d3b640 1 --2- 172.21.15.37:0/1460288388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c100440 0x7f7d5c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.807+0000 7f7d60d3b640 1 --2- 172.21.15.37:0/1460288388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c105170 0x7f7d5c105550 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.807+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1460288388 >> 172.21.15.37:0/1460288388 conn(0x7f7d5c076fc0 msgr2=0x7f7d5c0773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.807+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1460288388 shutdown_connections 2024-03-26T06:28:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.807+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1460288388 wait complete. 2024-03-26T06:28:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.808+0000 7f7d60d3b640 1 Processor -- start 2024-03-26T06:28:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.808+0000 7f7d60d3b640 1 -- start start 2024-03-26T06:28:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.808+0000 7f7d60d3b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c100440 0x7f7d5c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.808+0000 7f7d60d3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100de0 0x7f7d5c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.808+0000 7f7d60d3b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d5c105170 0x7f7d5c196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.808+0000 7f7d60d3b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c110620 con 0x7f7d5c100de0 2024-03-26T06:28:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.808+0000 7f7d60d3b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1104a0 con 0x7f7d5c100440 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.808+0000 7f7d60d3b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1107a0 con 0x7f7d5c105170 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d59d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100de0 0x7f7d5c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d59d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100de0 0x7f7d5c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43900/0 (socket says 172.21.15.37:43900) 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d59d9b640 1 -- 172.21.15.37:0/1386393390 learned_addr learned my addr 172.21.15.37:0/1386393390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d5a59c640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c100440 0x7f7d5c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d5ad9d640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d5c105170 0x7f7d5c196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d59d9b640 1 -- 172.21.15.37:0/1386393390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d5c105170 msgr2=0x7f7d5c196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d5c105170 0x7f7d5c196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d59d9b640 1 -- 172.21.15.37:0/1386393390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c100440 msgr2=0x7f7d5c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c100440 0x7f7d5c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d59d9b640 1 -- 172.21.15.37:0/1386393390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c197000 con 0x7f7d5c100de0 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100de0 0x7f7d5c19cc10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000e9f0 tx=0x7f7d5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d4b7fe640 1 -- 172.21.15.37:0/1386393390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000cd80 con 0x7f7d5c100de0 2024-03-26T06:28:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d4b7fe640 1 -- 172.21.15.37:0/1386393390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d50010910 con 0x7f7d5c100de0 2024-03-26T06:28:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1386393390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c1972f0 con 0x7f7d5c100de0 2024-03-26T06:28:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d4b7fe640 1 -- 172.21.15.37:0/1386393390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50018990 con 0x7f7d5c100de0 2024-03-26T06:28:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.809+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1386393390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c1a3590 con 0x7f7d5c100de0 2024-03-26T06:28:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.812+0000 7f7d4b7fe640 1 -- 172.21.15.37:0/1386393390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7d50010ac0 con 0x7f7d5c100de0 2024-03-26T06:28:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.812+0000 7f7d639f2740 1 Processor -- start 2024-03-26T06:28:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.812+0000 7f7d639f2740 1 -- start start 2024-03-26T06:28:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.812+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddce10 0x555d8eddbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.812+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8eddc030 0x555d8eddc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.812+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8ede7b50 0x555d8ede9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.812+0000 7f7d639f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8ed3e5f0 con 0x555d8ede7b50 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.812+0000 7f7d639f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8ece7930 con 0x555d8eddce10 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.812+0000 7f7d639f2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8ed3a690 con 0x555d8eddc030 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.812+0000 7f7d4b7fe640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7d440a2a10 0x7f7d440a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d59d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8ede7b50 0x555d8ede9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d5ad9d640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7d440a2a10 0x7f7d440a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d59d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8ede7b50 0x555d8ede9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43910/0 (socket says 172.21.15.37:43910) 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2775669538 learned_addr learned my addr 172.21.15.37:0/2775669538 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d5a59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8eddc030 0x555d8eddc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d5ad9d640 1 --2- 172.21.15.37:0/2775669538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddce10 0x555d8eddbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d4b7fe640 1 -- 172.21.15.37:0/1386393390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7d500cf0d0 con 0x7f7d5c100de0 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2775669538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8eddc030 msgr2=0x555d8eddc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/2775669538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8eddc030 0x555d8eddc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2775669538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddce10 msgr2=0x555d8eddbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/2775669538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddce10 0x555d8eddbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2775669538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d8ea53210 con 0x555d8ede7b50 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d5a59c640 1 --2- 172.21.15.37:0/2775669538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8eddc030 0x555d8eddc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d5ad9d640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7d440a2a10 0x7f7d440a4ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7d54005fd0 tx=0x7f7d54009de0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/2775669538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8ede7b50 0x555d8ede9f40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7d50097d10 tx=0x7f7d5006b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b431ce07968b40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.813+0000 7f7d48ff9640 1 -- 172.21.15.37:0/2775669538 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5009b8b0 con 0x555d8ede7b50 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.814+0000 7f7d48ff9640 1 -- 172.21.15.37:0/2775669538 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d5009f950 con 0x555d8ede7b50 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.814+0000 7f7d48ff9640 1 -- 172.21.15.37:0/2775669538 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50097f20 con 0x555d8ede7b50 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.814+0000 7f7d639f2740 1 -- 172.21.15.37:0/2775669538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8ede7b50 msgr2=0x555d8ede9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.814+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2775669538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8ede7b50 0x555d8ede9f40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7d50097d10 tx=0x7f7d5006b440 comp rx=0 tx=0).stop 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.814+0000 7f7d639f2740 1 -- 172.21.15.37:0/2775669538 shutdown_connections 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.814+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2775669538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8ede7b50 0x555d8ede9f40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.814+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2775669538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8eddc030 0x555d8eddc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.814+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2775669538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddce10 0x555d8eddbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.814+0000 7f7d639f2740 1 -- 172.21.15.37:0/2775669538 >> 172.21.15.37:0/2775669538 conn(0x555d8eda15b0 msgr2=0x555d8edd6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.814+0000 7f7d639f2740 1 -- 172.21.15.37:0/2775669538 shutdown_connections 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.814+0000 7f7d639f2740 1 -- 172.21.15.37:0/2775669538 wait complete. 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d639f2740 1 -- start start 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d639f2740 1 Processor -- start 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddc030 0x555d8ede4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8ede50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 0x555d8ede5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d639f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8edea600 con 0x555d8eddce10 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d639f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8ecfad90 con 0x555d8eddc030 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d639f2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8edea780 con 0x555d8ede7b50 2024-03-26T06:28:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d639f2740 10 client.0 _ll_register_callbacks cb 0x555d8ea52a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d59d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8ede50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d59d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8ede50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43914/0 (socket says 172.21.15.37:43914) 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2268211700 learned_addr learned my addr 172.21.15.37:0/2268211700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d5a59c640 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddc030 0x555d8ede4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.815+0000 7f7d5ad9d640 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 0x555d8ede5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2268211700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 msgr2=0x555d8ede5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 0x555d8ede5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2268211700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddc030 msgr2=0x555d8ede4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddc030 0x555d8ede4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2268211700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d8eb1f620 con 0x555d8eddce10 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8ede50c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7d5009b5b0 tx=0x7f7d500a4010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9605a4c2c0f3cecc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:28:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d41ffb640 1 -- 172.21.15.37:0/2268211700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d500a3b10 con 0x555d8eddce10 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d41ffb640 1 -- 172.21.15.37:0/2268211700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7d500a3cb0 con 0x555d8eddce10 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d639f2740 1 -- 172.21.15.37:0/2268211700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d5009f950 con 0x555d8eddce10 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d41ffb640 1 -- 172.21.15.37:0/2268211700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d500afbe0 con 0x555d8eddce10 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d639f2740 10 client.15903 fetch_fsmap learned FSMap version 37 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d639f2740 1 -- 172.21.15.37:0/2268211700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555d8edf68a0 con 0x555d8eddce10 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d41ffb640 1 -- 172.21.15.37:0/2268211700 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d5009f950 con 0x555d8eddce10 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.816+0000 7f7d41ffb640 1 -- 172.21.15.37:0/2268211700 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 37) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7d50018b30 con 0x555d8eddce10 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.817+0000 7f7d639f2740 10 client.15903 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.817+0000 7f7d639f2740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.817+0000 7f7d639f2740 20 client.15903 populate_metadata read hostname 'smithi037' 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.817+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555d8edf7bf0 0x555d8edf9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.817+0000 7f7d639f2740 4 client.15903 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:28:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.817+0000 7f7d639f2740 1 -- 172.21.15.37:0/2268211700 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d8ed4bb30 con 0x555d8edf7bf0 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.817+0000 7f7d5a59c640 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555d8edf7bf0 0x555d8edf9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.817+0000 7f7d5a59c640 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555d8edf7bf0 0x555d8edf9fe0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b85c1ab66c6228d server_cookie=e1c3a3355f150608 in_seq=0 out_seq=0 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.817+0000 7f7d41ffb640 10 client.15903 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d41ffb640 1 -- 172.21.15.37:0/2268211700 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d8ed4bb30 con 0x555d8edf7bf0 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d41ffb640 10 client.15903 handle_command_reply: tid=0 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 1 client.15903 shutdown 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 1 -- 172.21.15.37:0/2268211700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 msgr2=0x555d8ede50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8ede50c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7d5009b5b0 tx=0x7f7d500a4010 comp rx=0 tx=0).stop 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 1 -- 172.21.15.37:0/2268211700 shutdown_connections 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555d8edf7bf0 0x555d8edf9fe0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 0x555d8ede5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8ede50c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d41ffb640 0 client.15903 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2268211700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddc030 0x555d8ede4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 1 -- 172.21.15.37:0/2268211700 >> 172.21.15.37:0/2268211700 conn(0x555d8eda15b0 msgr2=0x555d8edd8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 1 -- 172.21.15.37:0/2268211700 shutdown_connections 2024-03-26T06:28:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 1 -- 172.21.15.37:0/2268211700 wait complete. 2024-03-26T06:28:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.818+0000 7f7d639f2740 20 client.15903 trim_cache size 0 max 16384 2024-03-26T06:28:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d639f2740 1 Processor -- start 2024-03-26T06:28:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d639f2740 1 -- start start 2024-03-26T06:28:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddc030 0x555d8ede72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8edd3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 0x555d8edee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d639f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1aa980 con 0x555d8eddce10 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d639f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1ac330 con 0x555d8eddc030 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d639f2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1aafe0 con 0x555d8ede7b50 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d59d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8edd3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d59d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8edd3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43920/0 (socket says 172.21.15.37:43920) 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2401376532 learned_addr learned my addr 172.21.15.37:0/2401376532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d5a59c640 1 --2- 172.21.15.37:0/2401376532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddc030 0x555d8ede72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2401376532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 msgr2=0x555d8edee9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/2401376532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 0x555d8edee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2401376532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddc030 msgr2=0x555d8ede72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/2401376532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddc030 0x555d8ede72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d59d9b640 1 -- 172.21.15.37:0/2401376532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d8ede2b20 con 0x555d8eddce10 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.832+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/2401376532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8edd3e90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7d5009b840 tx=0x7f7d50000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cd33f378e8f4b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d43fff640 1 -- 172.21.15.37:0/2401376532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5009b8b0 con 0x555d8eddce10 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d43fff640 1 -- 172.21.15.37:0/2401376532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d5009cb30 con 0x555d8eddce10 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d43fff640 1 -- 172.21.15.37:0/2401376532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5009ad00 con 0x555d8eddce10 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d639f2740 1 -- 172.21.15.37:0/2401376532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 msgr2=0x555d8edd3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2401376532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8edd3e90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7d5009b840 tx=0x7f7d50000bc0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d639f2740 1 -- 172.21.15.37:0/2401376532 shutdown_connections 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2401376532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 0x555d8edee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2401376532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddce10 0x555d8edd3e90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d639f2740 1 --2- 172.21.15.37:0/2401376532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddc030 0x555d8ede72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d639f2740 1 -- 172.21.15.37:0/2401376532 >> 172.21.15.37:0/2401376532 conn(0x555d8eda15b0 msgr2=0x555d8edd35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d639f2740 1 -- 172.21.15.37:0/2401376532 shutdown_connections 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d639f2740 1 -- 172.21.15.37:0/2401376532 wait complete. 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.833+0000 7f7d639f2740 1 -- start start 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d639f2740 1 Processor -- start 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddc030 0x555d8eded6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddce10 0x555d8ededbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d639f2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 0x555d8ee4dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d639f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1abcd0 con 0x555d8eddc030 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d639f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1ab670 con 0x555d8eddce10 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d639f2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1a8c20 con 0x555d8ede7b50 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d639f2740 10 client.0 _ll_register_callbacks cb 0x7f7d50009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d5a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddc030 0x555d8eded6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d5a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddc030 0x555d8eded6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43932/0 (socket says 172.21.15.37:43932) 2024-03-26T06:28:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d5a59c640 1 -- 172.21.15.37:0/1277703398 learned_addr learned my addr 172.21.15.37:0/1277703398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d5a59c640 1 -- 172.21.15.37:0/1277703398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 msgr2=0x555d8ee4dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d5a59c640 1 --2- 172.21.15.37:0/1277703398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 0x555d8ee4dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d5a59c640 1 -- 172.21.15.37:0/1277703398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddce10 msgr2=0x555d8ededbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d5a59c640 1 --2- 172.21.15.37:0/1277703398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddce10 0x555d8ededbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d5a59c640 1 -- 172.21.15.37:0/1277703398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d8edee4d0 con 0x555d8eddc030 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d5a59c640 1 --2- 172.21.15.37:0/1277703398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddc030 0x555d8eded6a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c02c170 tx=0x7f7d4c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff88379611d5ed41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.834+0000 7f7d41ffb640 1 -- 172.21.15.37:0/1277703398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c004030 con 0x555d8eddc030 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d41ffb640 1 -- 172.21.15.37:0/1277703398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7d4c0041d0 con 0x555d8eddc030 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d639f2740 1 -- 172.21.15.37:0/1277703398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d5009cb30 con 0x555d8eddc030 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d41ffb640 1 -- 172.21.15.37:0/1277703398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d4c056050 con 0x555d8eddc030 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d639f2740 10 client.15909 fetch_fsmap learned FSMap version 37 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d639f2740 1 -- 172.21.15.37:0/1277703398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555d8ee4f350 con 0x555d8eddc030 2024-03-26T06:28:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d41ffb640 1 -- 172.21.15.37:0/1277703398 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d5009cb30 con 0x555d8eddc030 2024-03-26T06:28:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d41ffb640 1 -- 172.21.15.37:0/1277703398 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 37) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7d4c047070 con 0x555d8eddc030 2024-03-26T06:28:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d639f2740 10 client.15909 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T06:28:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d639f2740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:28:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d639f2740 20 client.15909 populate_metadata read hostname 'smithi037' 2024-03-26T06:28:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d639f2740 1 --2- 172.21.15.37:0/1277703398 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555d8ee4fa80 0x555d8ee51ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d639f2740 4 client.15909 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:28:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d639f2740 1 -- 172.21.15.37:0/1277703398 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555d8ed45e80 con 0x555d8ee4fa80 2024-03-26T06:28:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.835+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/1277703398 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555d8ee4fa80 0x555d8ee51ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.836+0000 7f7d59d9b640 1 --2- 172.21.15.37:0/1277703398 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555d8ee4fa80 0x555d8ee51ea0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4756bf2204271a server_cookie=59e8db91a5ac9107 in_seq=0 out_seq=0 2024-03-26T06:28:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.836+0000 7f7d41ffb640 10 client.15909 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d41ffb640 1 -- 172.21.15.37:0/1277703398 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555d8ed45e80 con 0x555d8ee4fa80 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d41ffb640 10 client.15909 handle_command_reply: tid=0 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 1 client.15909 shutdown 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 1 -- 172.21.15.37:0/1277703398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddc030 msgr2=0x555d8eded6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 1 --2- 172.21.15.37:0/1277703398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddc030 0x555d8eded6a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c02c170 tx=0x7f7d4c0095f0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 1 -- 172.21.15.37:0/1277703398 shutdown_connections 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 1 --2- 172.21.15.37:0/1277703398 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555d8ee4fa80 0x555d8ee51ea0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 1 --2- 172.21.15.37:0/1277703398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d8ede7b50 0x555d8ee4dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 1 --2- 172.21.15.37:0/1277703398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555d8eddce10 0x555d8ededbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 1 --2- 172.21.15.37:0/1277703398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d8eddc030 0x555d8eded6a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d41ffb640 0 client.15909 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 1 -- 172.21.15.37:0/1277703398 >> 172.21.15.37:0/1277703398 conn(0x555d8eda15b0 msgr2=0x555d8edd6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 1 -- 172.21.15.37:0/1277703398 shutdown_connections 2024-03-26T06:28:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 1 -- 172.21.15.37:0/1277703398 wait complete. 2024-03-26T06:28:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.873+0000 7f7d639f2740 20 client.15909 trim_cache size 0 max 16384 2024-03-26T06:28:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:28:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:28:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:28:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:28:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1386393390 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7d440a2a10 msgr2=0x7f7d440a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7d440a2a10 0x7f7d440a4ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7d54005fd0 tx=0x7f7d54009de0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1386393390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100de0 msgr2=0x7f7d5c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100de0 0x7f7d5c19cc10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000e9f0 tx=0x7f7d5000eeb0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1386393390 shutdown_connections 2024-03-26T06:28:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7d440a2a10 0x7f7d440a4ed0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d5c105170 0x7f7d5c196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100de0 0x7f7d5c19cc10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 --2- 172.21.15.37:0/1386393390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7d5c100440 0x7f7d5c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1386393390 >> 172.21.15.37:0/1386393390 conn(0x7f7d5c076fc0 msgr2=0x7f7d5c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1386393390 shutdown_connections 2024-03-26T06:28:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:03.874+0000 7f7d60d3b640 1 -- 172.21.15.37:0/1386393390 wait complete. 2024-03-26T06:28:04.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 146250 3.86 MB/sec execute 99 sec latency 1765.036 ms 2024-03-26T06:28:04.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:04 smithi037 ceph-mon[31974]: pgmap v155: 97 pgs: 97 active+clean; 57 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-03-26T06:28:05.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:04 smithi066 ceph-mon[36504]: pgmap v155: 97 pgs: 97 active+clean; 57 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-03-26T06:28:05.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:04 smithi149 ceph-mon[36681]: pgmap v155: 97 pgs: 97 active+clean; 57 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-03-26T06:28:05.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 146250 3.82 MB/sec execute 100 sec latency 2765.156 ms 2024-03-26T06:28:06.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 146505 3.80 MB/sec execute 101 sec latency 3569.957 ms 2024-03-26T06:28:06.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:06 smithi037 ceph-mon[31974]: pgmap v156: 97 pgs: 97 active+clean; 56 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-03-26T06:28:07.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:06 smithi066 ceph-mon[36504]: pgmap v156: 97 pgs: 97 active+clean; 56 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-03-26T06:28:07.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:06 smithi149 ceph-mon[36681]: pgmap v156: 97 pgs: 97 active+clean; 56 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-03-26T06:28:07.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 149148 3.92 MB/sec execute 102 sec latency 14.757 ms 2024-03-26T06:28:08.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 149667 3.89 MB/sec execute 103 sec latency 453.960 ms 2024-03-26T06:28:08.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:08 smithi037 ceph-mon[31974]: pgmap v157: 97 pgs: 97 active+clean; 56 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 942 KiB/s wr, 155 op/s 2024-03-26T06:28:08.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:08 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:28:09.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:08 smithi066 ceph-mon[36504]: pgmap v157: 97 pgs: 97 active+clean; 56 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 942 KiB/s wr, 155 op/s 2024-03-26T06:28:09.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:08 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:28:09.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:08 smithi149 ceph-mon[36681]: pgmap v157: 97 pgs: 97 active+clean; 56 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 942 KiB/s wr, 155 op/s 2024-03-26T06:28:09.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:08 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:28:09.500 INFO:tasks.workunit.client.0.smithi037.stdout: 1 149667 3.86 MB/sec execute 104 sec latency 1454.098 ms 2024-03-26T06:28:09.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:09 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:28:09.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:09 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:28:09.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:09 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:28:10.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:09 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:28:10.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:09 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:28:10.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:09 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:28:10.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:09 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:28:10.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:09 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:28:10.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:09 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:28:10.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 149667 3.82 MB/sec execute 105 sec latency 2454.192 ms 2024-03-26T06:28:10.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:10 smithi037 ceph-mon[31974]: pgmap v158: 97 pgs: 97 active+clean; 56 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 942 KiB/s wr, 155 op/s 2024-03-26T06:28:11.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:10 smithi066 ceph-mon[36504]: pgmap v158: 97 pgs: 97 active+clean; 56 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 942 KiB/s wr, 155 op/s 2024-03-26T06:28:11.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:10 smithi149 ceph-mon[36681]: pgmap v158: 97 pgs: 97 active+clean; 56 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 942 KiB/s wr, 155 op/s 2024-03-26T06:28:11.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 149979 3.79 MB/sec execute 106 sec latency 3259.600 ms 2024-03-26T06:28:12.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 152665 3.91 MB/sec execute 107 sec latency 14.291 ms 2024-03-26T06:28:12.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:12 smithi037 ceph-mon[31974]: pgmap v159: 97 pgs: 97 active+clean; 57 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-03-26T06:28:13.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:12 smithi066 ceph-mon[36504]: pgmap v159: 97 pgs: 97 active+clean; 57 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-03-26T06:28:13.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:12 smithi149 ceph-mon[36681]: pgmap v159: 97 pgs: 97 active+clean; 57 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-03-26T06:28:13.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 153341 3.89 MB/sec execute 108 sec latency 353.530 ms 2024-03-26T06:28:14.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 153341 3.86 MB/sec execute 109 sec latency 1353.649 ms 2024-03-26T06:28:14.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:14 smithi037 ceph-mon[31974]: pgmap v160: 97 pgs: 97 active+clean; 57 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-03-26T06:28:15.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:14 smithi066 ceph-mon[36504]: pgmap v160: 97 pgs: 97 active+clean; 57 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-03-26T06:28:15.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:14 smithi149 ceph-mon[36681]: pgmap v160: 97 pgs: 97 active+clean; 57 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-03-26T06:28:15.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 153341 3.82 MB/sec execute 110 sec latency 2353.742 ms 2024-03-26T06:28:16.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 153729 3.80 MB/sec execute 111 sec latency 3157.222 ms 2024-03-26T06:28:16.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:16 smithi037 ceph-mon[31974]: pgmap v161: 97 pgs: 97 active+clean; 59 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-03-26T06:28:16.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:16 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.965+0000 7f60ee259640 1 -- 172.21.15.37:0/3955105433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81065d0 msgr2=0x7f60e810e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.965+0000 7f60ee259640 1 --2- 172.21.15.37:0/3955105433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81065d0 0x7f60e810e1a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009ca0 tx=0x7f60e002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.965+0000 7f60ee259640 1 -- 172.21.15.37:0/3955105433 shutdown_connections 2024-03-26T06:28:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.965+0000 7f60ee259640 1 --2- 172.21.15.37:0/3955105433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81065d0 0x7f60e810e1a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.965+0000 7f60ee259640 1 --2- 172.21.15.37:0/3955105433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e8101eb0 0x7f60e8105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.965+0000 7f60ee259640 1 --2- 172.21.15.37:0/3955105433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e8101500 0x7f60e81018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.965+0000 7f60ee259640 1 -- 172.21.15.37:0/3955105433 >> 172.21.15.37:0/3955105433 conn(0x7f60e80fcd80 msgr2=0x7f60e80ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.965+0000 7f60ee259640 1 -- 172.21.15.37:0/3955105433 shutdown_connections 2024-03-26T06:28:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.965+0000 7f60ee259640 1 -- 172.21.15.37:0/3955105433 wait complete. 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60ee259640 1 Processor -- start 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60ee259640 1 -- start start 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60ee259640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e8101500 0x7f60e80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60ee259640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81065d0 0x7f60e8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60ee259640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e8074070 0x7f60e81a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60ee259640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e8111670 con 0x7f60e81065d0 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60ee259640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e81114f0 con 0x7f60e8101500 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60ee259640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e81117f0 con 0x7f60e8074070 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e8101500 0x7f60e80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60e6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81065d0 0x7f60e8073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e8101500 0x7f60e80735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:59640/0 (socket says 172.21.15.37:59640) 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60e77fe640 1 -- 172.21.15.37:0/4082465791 learned_addr learned my addr 172.21.15.37:0/4082465791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.966+0000 7f60e7fff640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e8074070 0x7f60e81a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60e77fe640 1 -- 172.21.15.37:0/4082465791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e8074070 msgr2=0x7f60e81a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60e77fe640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e8074070 0x7f60e81a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60e77fe640 1 -- 172.21.15.37:0/4082465791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81065d0 msgr2=0x7f60e8073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60e77fe640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81065d0 0x7f60e8073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60e77fe640 1 -- 172.21.15.37:0/4082465791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e81a6340 con 0x7f60e8101500 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60e6ffd640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81065d0 0x7f60e8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60e77fe640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e8101500 0x7f60e80735f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e9f0 tx=0x7f60dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60e4ff9640 1 -- 172.21.15.37:0/4082465791 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00cd80 con 0x7f60e8101500 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60ee259640 1 -- 172.21.15.37:0/4082465791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e81a6630 con 0x7f60e8101500 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60ee259640 1 -- 172.21.15.37:0/4082465791 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e81a6bd0 con 0x7f60e8101500 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60e4ff9640 1 -- 172.21.15.37:0/4082465791 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60dc010910 con 0x7f60e8101500 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.967+0000 7f60e4ff9640 1 -- 172.21.15.37:0/4082465791 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc018980 con 0x7f60e8101500 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60e4ff9640 1 -- 172.21.15.37:0/4082465791 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f60dc010ac0 con 0x7f60e8101500 2024-03-26T06:28:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60f0e38740 1 Processor -- start 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60f0e38740 1 -- start start 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f7fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f80450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f8b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60f0e38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4ee45f0 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60f0e38740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4e8d930 con 0x55ecb4f89550 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60f0e38740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4ee0690 con 0x55ecb4f80d40 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f80450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f80450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38970/0 (socket says 172.21.15.37:38970) 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60e77fe640 1 -- 172.21.15.37:0/119730664 learned_addr learned my addr 172.21.15.37:0/119730664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60e7fff640 1 --2- 172.21.15.37:0/119730664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f7fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.970+0000 7f60e6ffd640 1 --2- 172.21.15.37:0/119730664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f8b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60e77fe640 1 -- 172.21.15.37:0/119730664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 msgr2=0x55ecb4f7fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60e77fe640 1 --2- 172.21.15.37:0/119730664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f7fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60e4ff9640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f60d00a2a40 0x7f60d00a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60e77fe640 1 -- 172.21.15.37:0/119730664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 msgr2=0x55ecb4f8b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60e77fe640 1 --2- 172.21.15.37:0/119730664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f8b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60e77fe640 1 -- 172.21.15.37:0/119730664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb4bf9210 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60e7fff640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f60d00a2a40 0x7f60d00a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60e4ff9640 1 -- 172.21.15.37:0/4082465791 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f60dc0cf0b0 con 0x7f60e8101500 2024-03-26T06:28:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60e77fe640 1 --2- 172.21.15.37:0/119730664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f80450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0cc7a0 tx=0x7f60dc0aae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47743ecd23bee18 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60cdffb640 1 -- 172.21.15.37:0/119730664 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc026030 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60cdffb640 1 -- 172.21.15.37:0/119730664 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60dc010db0 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.971+0000 7f60cdffb640 1 -- 172.21.15.37:0/119730664 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc09b150 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.972+0000 7f60f0e38740 1 -- 172.21.15.37:0/119730664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 msgr2=0x55ecb4f80450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.972+0000 7f60e7fff640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f60d00a2a40 0x7f60d00a4f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f60e0004af0 tx=0x7f60e0005c50 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.972+0000 7f60f0e38740 1 --2- 172.21.15.37:0/119730664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f80450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0cc7a0 tx=0x7f60dc0aae40 comp rx=0 tx=0).stop 2024-03-26T06:28:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.972+0000 7f60f0e38740 1 -- 172.21.15.37:0/119730664 shutdown_connections 2024-03-26T06:28:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.972+0000 7f60f0e38740 1 --2- 172.21.15.37:0/119730664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f8b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.972+0000 7f60f0e38740 1 --2- 172.21.15.37:0/119730664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f80450 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.972+0000 7f60f0e38740 1 --2- 172.21.15.37:0/119730664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f7fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.972+0000 7f60f0e38740 1 -- 172.21.15.37:0/119730664 >> 172.21.15.37:0/119730664 conn(0x55ecb4f475b0 msgr2=0x55ecb4f7acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.972+0000 7f60f0e38740 1 -- 172.21.15.37:0/119730664 shutdown_connections 2024-03-26T06:28:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.972+0000 7f60f0e38740 1 -- 172.21.15.37:0/119730664 wait complete. 2024-03-26T06:28:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.972+0000 7f60f0e38740 1 -- start start 2024-03-26T06:28:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60f0e38740 1 Processor -- start 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f86530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f86a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f86fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60f0e38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4f8c280 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60f0e38740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4ea0d90 con 0x55ecb4f89550 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60f0e38740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb4f8c400 con 0x55ecb4f80d40 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60f0e38740 10 client.0 _ll_register_callbacks cb 0x55ecb4bf8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f86530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f86530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38982/0 (socket says 172.21.15.37:38982) 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60e77fe640 1 -- 172.21.15.37:0/1924577505 learned_addr learned my addr 172.21.15.37:0/1924577505 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60e6ffd640 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f86a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60e7fff640 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f86fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60e77fe640 1 -- 172.21.15.37:0/1924577505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 msgr2=0x55ecb4f86a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60e77fe640 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f86a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60e77fe640 1 -- 172.21.15.37:0/1924577505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 msgr2=0x55ecb4f86fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60e77fe640 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f86fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60e77fe640 1 -- 172.21.15.37:0/1924577505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecb4cc5620 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60e77fe640 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f86530 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0cc8e0 tx=0x7f60dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c1d97fc8566974 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.973+0000 7f60c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1924577505 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc09b150 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1924577505 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f60dc097540 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60f0e38740 1 -- 172.21.15.37:0/1924577505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60dc010db0 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1924577505 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60dc09b5c0 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60f0e38740 10 client.15921 fetch_fsmap learned FSMap version 38 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60f0e38740 1 -- 172.21.15.37:0/1924577505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55ecb4f98590 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1924577505 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60dc010db0 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1924577505 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60dc09ba50 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60f0e38740 10 client.15921 fetch_fsmap finished waiting for FSMap version 38 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60f0e38740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60f0e38740 20 client.15921 populate_metadata read hostname 'smithi037' 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ecb4f998e0 0x55ecb4f9bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60f0e38740 4 client.15921 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60f0e38740 1 -- 172.21.15.37:0/1924577505 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecb4ef1b30 con 0x55ecb4f998e0 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.974+0000 7f60e6ffd640 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ecb4f998e0 0x55ecb4f9bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.975+0000 7f60c6ffd640 10 client.15921 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.975+0000 7f60e6ffd640 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ecb4f998e0 0x55ecb4f9bcd0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606f2123be1669ec server_cookie=a470edd4525af125 in_seq=0 out_seq=0 2024-03-26T06:28:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.975+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1924577505 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ecb4ef1b30 con 0x55ecb4f998e0 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.975+0000 7f60c6ffd640 10 client.15921 handle_command_reply: tid=0 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.975+0000 7f60f0e38740 1 client.15921 shutdown 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.975+0000 7f60f0e38740 1 -- 172.21.15.37:0/1924577505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 msgr2=0x55ecb4f86530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.975+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f86530 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0cc8e0 tx=0x7f60dc00cf90 comp rx=0 tx=0).stop 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.975+0000 7f60f0e38740 1 -- 172.21.15.37:0/1924577505 shutdown_connections 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.975+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ecb4f998e0 0x55ecb4f9bcd0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.975+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f86fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.976+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f86a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.975+0000 7f60c6ffd640 0 client.15921 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.976+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1924577505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f86530 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.976+0000 7f60f0e38740 1 -- 172.21.15.37:0/1924577505 >> 172.21.15.37:0/1924577505 conn(0x55ecb4f475b0 msgr2=0x55ecb4f7ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.976+0000 7f60f0e38740 1 -- 172.21.15.37:0/1924577505 shutdown_connections 2024-03-26T06:28:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.976+0000 7f60f0e38740 1 -- 172.21.15.37:0/1924577505 wait complete. 2024-03-26T06:28:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.976+0000 7f60f0e38740 20 client.15921 trim_cache size 0 max 16384 2024-03-26T06:28:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60f0e38740 1 Processor -- start 2024-03-26T06:28:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60f0e38740 1 -- start start 2024-03-26T06:28:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f85e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f9dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f89550 0x55ecb4f9e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60f0e38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d400e3c0 con 0x55ecb4f89550 2024-03-26T06:28:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60f0e38740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d400fd40 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60f0e38740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d400ea20 con 0x55ecb4f80d40 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60e7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f89550 0x55ecb4f9e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f85e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60e7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f89550 0x55ecb4f9e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38984/0 (socket says 172.21.15.37:38984) 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60e7fff640 1 -- 172.21.15.37:0/2670669792 learned_addr learned my addr 172.21.15.37:0/2670669792 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60e6ffd640 1 --2- 172.21.15.37:0/2670669792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f9dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60e7fff640 1 -- 172.21.15.37:0/2670669792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 msgr2=0x55ecb4f9dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60e7fff640 1 --2- 172.21.15.37:0/2670669792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f9dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60e7fff640 1 -- 172.21.15.37:0/2670669792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f7ffd0 msgr2=0x55ecb4f85e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60e7fff640 1 --2- 172.21.15.37:0/2670669792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f85e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60e7fff640 1 -- 172.21.15.37:0/2670669792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb4f7be80 con 0x55ecb4f89550 2024-03-26T06:28:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.989+0000 7f60e7fff640 1 --2- 172.21.15.37:0/2670669792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f89550 0x55ecb4f9e220 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009ca0 tx=0x7f60e0041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c554d01f7107a282 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60cd7fa640 1 -- 172.21.15.37:0/2670669792 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0039040 con 0x55ecb4f89550 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60cd7fa640 1 -- 172.21.15.37:0/2670669792 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60e0003e50 con 0x55ecb4f89550 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60cd7fa640 1 -- 172.21.15.37:0/2670669792 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0004830 con 0x55ecb4f89550 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60f0e38740 1 -- 172.21.15.37:0/2670669792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f89550 msgr2=0x55ecb4f9e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60f0e38740 1 --2- 172.21.15.37:0/2670669792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f89550 0x55ecb4f9e220 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009ca0 tx=0x7f60e0041000 comp rx=0 tx=0).stop 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60f0e38740 1 -- 172.21.15.37:0/2670669792 shutdown_connections 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60f0e38740 1 --2- 172.21.15.37:0/2670669792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f89550 0x55ecb4f9e220 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60f0e38740 1 --2- 172.21.15.37:0/2670669792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f9dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60f0e38740 1 --2- 172.21.15.37:0/2670669792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f85e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60f0e38740 1 -- 172.21.15.37:0/2670669792 >> 172.21.15.37:0/2670669792 conn(0x55ecb4f475b0 msgr2=0x55ecb4f94fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60f0e38740 1 -- 172.21.15.37:0/2670669792 shutdown_connections 2024-03-26T06:28:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.990+0000 7f60f0e38740 1 -- 172.21.15.37:0/2670669792 wait complete. 2024-03-26T06:28:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60f0e38740 1 -- start start 2024-03-26T06:28:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60f0e38740 1 Processor -- start 2024-03-26T06:28:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f9e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f9c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60f0e38740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f9cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60f0e38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d400f6e0 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60f0e38740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d400f080 con 0x55ecb4f89550 2024-03-26T06:28:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60f0e38740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d400c660 con 0x55ecb4f80d40 2024-03-26T06:28:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60f0e38740 10 client.0 _ll_register_callbacks cb 0x7f60dc0d2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f9e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f9e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38994/0 (socket says 172.21.15.37:38994) 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60e77fe640 1 -- 172.21.15.37:0/1428573273 learned_addr learned my addr 172.21.15.37:0/1428573273 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60e7fff640 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f9cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.991+0000 7f60e6ffd640 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f9c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60e77fe640 1 -- 172.21.15.37:0/1428573273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 msgr2=0x55ecb4f9c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60e77fe640 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f9c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60e77fe640 1 -- 172.21.15.37:0/1428573273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 msgr2=0x55ecb4f9cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60e77fe640 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f9cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60e77fe640 1 -- 172.21.15.37:0/1428573273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecb4f7f2a0 con 0x55ecb4f7ffd0 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60e6ffd640 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f9c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60e77fe640 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f9e140 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f60dc09f610 tx=0x7f60dc0d2e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379e2c8e04864765 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1428573273 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0d6050 con 0x55ecb4f7ffd0 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1428573273 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f60dc0a3c50 con 0x55ecb4f7ffd0 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60f0e38740 1 -- 172.21.15.37:0/1428573273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60e0003e50 con 0x55ecb4f7ffd0 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1428573273 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60dc09b150 con 0x55ecb4f7ffd0 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60f0e38740 10 client.15927 fetch_fsmap learned FSMap version 38 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60f0e38740 1 -- 172.21.15.37:0/1428573273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55ecb4ff0960 con 0x55ecb4f7ffd0 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1428573273 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60e0003e50 con 0x55ecb4f7ffd0 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1428573273 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60dc09f180 con 0x55ecb4f7ffd0 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60f0e38740 10 client.15927 fetch_fsmap finished waiting for FSMap version 38 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60f0e38740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.992+0000 7f60f0e38740 20 client.15927 populate_metadata read hostname 'smithi037' 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.993+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ecb4ff0e40 0x55ecb4ff3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.993+0000 7f60f0e38740 4 client.15927 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:28:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.993+0000 7f60f0e38740 1 -- 172.21.15.37:0/1428573273 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ecb4eebe80 con 0x55ecb4ff0e40 2024-03-26T06:28:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.993+0000 7f60e6ffd640 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ecb4ff0e40 0x55ecb4ff3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.993+0000 7f60c6ffd640 10 client.15927 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:16.993+0000 7f60e6ffd640 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ecb4ff0e40 0x55ecb4ff3230 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d0dbaf82ca5f742 server_cookie=52f65fd40e4391a in_seq=0 out_seq=0 2024-03-26T06:28:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60c6ffd640 1 -- 172.21.15.37:0/1428573273 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ecb4eebe80 con 0x55ecb4ff0e40 2024-03-26T06:28:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60c6ffd640 10 client.15927 handle_command_reply: tid=0 2024-03-26T06:28:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 1 client.15927 shutdown 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 1 -- 172.21.15.37:0/1428573273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 msgr2=0x55ecb4f9e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f9e140 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f60dc09f610 tx=0x7f60dc0d2e90 comp rx=0 tx=0).stop 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 1 -- 172.21.15.37:0/1428573273 shutdown_connections 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ecb4ff0e40 0x55ecb4ff3230 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ecb4f89550 0x55ecb4f9cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ecb4f80d40 0x55ecb4f9c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 1 --2- 172.21.15.37:0/1428573273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecb4f7ffd0 0x55ecb4f9e140 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60c6ffd640 0 client.15927 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 1 -- 172.21.15.37:0/1428573273 >> 172.21.15.37:0/1428573273 conn(0x55ecb4f475b0 msgr2=0x55ecb4f7c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 1 -- 172.21.15.37:0/1428573273 shutdown_connections 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 1 -- 172.21.15.37:0/1428573273 wait complete. 2024-03-26T06:28:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.025+0000 7f60f0e38740 20 client.15927 trim_cache size 0 max 16384 2024-03-26T06:28:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:28:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:28:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:28:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:28:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 -- 172.21.15.37:0/4082465791 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f60d00a2a40 msgr2=0x7f60d00a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f60d00a2a40 0x7f60d00a4f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f60e0004af0 tx=0x7f60e0005c50 comp rx=0 tx=0).stop 2024-03-26T06:28:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 -- 172.21.15.37:0/4082465791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e8101500 msgr2=0x7f60e80735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e8101500 0x7f60e80735f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e9f0 tx=0x7f60dc00eeb0 comp rx=0 tx=0).stop 2024-03-26T06:28:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 -- 172.21.15.37:0/4082465791 shutdown_connections 2024-03-26T06:28:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f60d00a2a40 0x7f60d00a4f00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e8074070 0x7f60e81a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81065d0 0x7f60e8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 --2- 172.21.15.37:0/4082465791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60e8101500 0x7f60e80735f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 -- 172.21.15.37:0/4082465791 >> 172.21.15.37:0/4082465791 conn(0x7f60e80fcd80 msgr2=0x7f60e80fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 -- 172.21.15.37:0/4082465791 shutdown_connections 2024-03-26T06:28:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:17.026+0000 7f60ee259640 1 -- 172.21.15.37:0/4082465791 wait complete. 2024-03-26T06:28:17.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:16 smithi066 ceph-mon[36504]: pgmap v161: 97 pgs: 97 active+clean; 59 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-03-26T06:28:17.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:16 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:17.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:16 smithi149 ceph-mon[36681]: pgmap v161: 97 pgs: 97 active+clean; 59 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-03-26T06:28:17.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:16 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:17.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 156237 3.91 MB/sec execute 112 sec latency 26.084 ms 2024-03-26T06:28:18.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 157702 3.93 MB/sec execute 113 sec latency 16.244 ms 2024-03-26T06:28:18.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:18 smithi037 ceph-mon[31974]: pgmap v162: 97 pgs: 97 active+clean; 59 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-03-26T06:28:19.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:18 smithi066 ceph-mon[36504]: pgmap v162: 97 pgs: 97 active+clean; 59 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-03-26T06:28:19.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:18 smithi149 ceph-mon[36681]: pgmap v162: 97 pgs: 97 active+clean; 59 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-03-26T06:28:19.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 159959 4.02 MB/sec execute 114 sec latency 12.873 ms 2024-03-26T06:28:20.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 162057 4.05 MB/sec execute 115 sec latency 14.622 ms 2024-03-26T06:28:20.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:20 smithi037 ceph-mon[31974]: pgmap v163: 97 pgs: 97 active+clean; 59 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-03-26T06:28:21.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:20 smithi066 ceph-mon[36504]: pgmap v163: 97 pgs: 97 active+clean; 59 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-03-26T06:28:21.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:20 smithi149 ceph-mon[36681]: pgmap v163: 97 pgs: 97 active+clean; 59 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-03-26T06:28:21.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 163655 4.11 MB/sec execute 116 sec latency 11.558 ms 2024-03-26T06:28:22.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 165551 4.14 MB/sec execute 117 sec latency 18.785 ms 2024-03-26T06:28:22.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:22 smithi037 ceph-mon[31974]: pgmap v164: 97 pgs: 97 active+clean; 61 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 237 op/s 2024-03-26T06:28:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:22 smithi066 ceph-mon[36504]: pgmap v164: 97 pgs: 97 active+clean; 61 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 237 op/s 2024-03-26T06:28:23.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:22 smithi149 ceph-mon[36681]: pgmap v164: 97 pgs: 97 active+clean; 61 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 237 op/s 2024-03-26T06:28:23.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 167176 4.21 MB/sec execute 118 sec latency 11.310 ms 2024-03-26T06:28:24.501 INFO:tasks.workunit.client.0.smithi037.stdout: 1 167455 4.17 MB/sec execute 119 sec latency 777.244 ms 2024-03-26T06:28:24.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:24 smithi037 ceph-mon[31974]: pgmap v165: 97 pgs: 97 active+clean; 61 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-03-26T06:28:25.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:24 smithi066 ceph-mon[36504]: pgmap v165: 97 pgs: 97 active+clean; 61 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-03-26T06:28:25.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:24 smithi149 ceph-mon[36681]: pgmap v165: 97 pgs: 97 active+clean; 61 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-03-26T06:28:25.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 167455 4.14 MB/sec execute 120 sec latency 1777.336 ms 2024-03-26T06:28:26.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 167804 4.12 MB/sec execute 121 sec latency 2583.934 ms 2024-03-26T06:28:26.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:26 smithi037 ceph-mon[31974]: pgmap v166: 97 pgs: 97 active+clean; 66 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 262 op/s 2024-03-26T06:28:27.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:26 smithi066 ceph-mon[36504]: pgmap v166: 97 pgs: 97 active+clean; 66 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 262 op/s 2024-03-26T06:28:27.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:26 smithi149 ceph-mon[36681]: pgmap v166: 97 pgs: 97 active+clean; 66 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 262 op/s 2024-03-26T06:28:27.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 170057 4.19 MB/sec execute 122 sec latency 34.532 ms 2024-03-26T06:28:28.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 171408 4.21 MB/sec execute 123 sec latency 20.196 ms 2024-03-26T06:28:28.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:28 smithi149 ceph-mon[36681]: pgmap v167: 97 pgs: 97 active+clean; 66 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-03-26T06:28:29.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:28 smithi066 ceph-mon[36504]: pgmap v167: 97 pgs: 97 active+clean; 66 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-03-26T06:28:29.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:28 smithi037 ceph-mon[31974]: pgmap v167: 97 pgs: 97 active+clean; 66 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-03-26T06:28:29.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 173296 4.26 MB/sec execute 124 sec latency 21.346 ms 2024-03-26T06:28:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.116+0000 7f540c3cd640 1 -- 172.21.15.37:0/2348204331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5404100440 msgr2=0x7f54041008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.116+0000 7f540c3cd640 1 --2- 172.21.15.37:0/2348204331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5404100440 0x7f54041008c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009900 tx=0x7f53f402bc30 comp rx=0 tx=0).stop 2024-03-26T06:28:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.116+0000 7f540c3cd640 1 -- 172.21.15.37:0/2348204331 shutdown_connections 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.116+0000 7f540c3cd640 1 --2- 172.21.15.37:0/2348204331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5404100e00 0x7f540410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.116+0000 7f540c3cd640 1 --2- 172.21.15.37:0/2348204331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5404100440 0x7f54041008c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.116+0000 7f540c3cd640 1 --2- 172.21.15.37:0/2348204331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5404105190 0x7f5404105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.116+0000 7f540c3cd640 1 -- 172.21.15.37:0/2348204331 >> 172.21.15.37:0/2348204331 conn(0x7f5404077040 msgr2=0x7f5404077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.116+0000 7f540c3cd640 1 -- 172.21.15.37:0/2348204331 shutdown_connections 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.116+0000 7f540c3cd640 1 -- 172.21.15.37:0/2348204331 wait complete. 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540c3cd640 1 Processor -- start 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540c3cd640 1 -- start start 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540c3cd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5404100440 0x7f540419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540c3cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5404100e00 0x7f540419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540c3cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5404105190 0x7f5404197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540c3cd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5404110650 con 0x7f5404105190 2024-03-26T06:28:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540c3cd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041104d0 con 0x7f5404100e00 2024-03-26T06:28:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540c3cd640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041107d0 con 0x7f5404100440 2024-03-26T06:28:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540a96a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5404105190 0x7f5404197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540a96a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5404105190 0x7f5404197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58554/0 (socket says 172.21.15.37:58554) 2024-03-26T06:28:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540a96a640 1 -- 172.21.15.37:0/1800045465 learned_addr learned my addr 172.21.15.37:0/1800045465 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f5409968640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5404100e00 0x7f540419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540a169640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5404100440 0x7f540419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540a96a640 1 -- 172.21.15.37:0/1800045465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5404100440 msgr2=0x7f540419d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540a96a640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5404100440 0x7f540419d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540a96a640 1 -- 172.21.15.37:0/1800045465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5404100e00 msgr2=0x7f540419dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540a96a640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5404100e00 0x7f540419dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.117+0000 7f540a96a640 1 -- 172.21.15.37:0/1800045465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54041980e0 con 0x7f5404105190 2024-03-26T06:28:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.118+0000 7f540a96a640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5404105190 0x7f5404197960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00ea60 tx=0x7f53fc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.118+0000 7f53fb7fe640 1 -- 172.21.15.37:0/1800045465 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00ce40 con 0x7f5404105190 2024-03-26T06:28:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.118+0000 7f53fb7fe640 1 -- 172.21.15.37:0/1800045465 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53fc005020 con 0x7f5404105190 2024-03-26T06:28:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.118+0000 7f540c3cd640 1 -- 172.21.15.37:0/1800045465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54041983d0 con 0x7f5404105190 2024-03-26T06:28:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.118+0000 7f53fb7fe640 1 -- 172.21.15.37:0/1800045465 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc010690 con 0x7f5404105190 2024-03-26T06:28:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.118+0000 7f540c3cd640 1 -- 172.21.15.37:0/1800045465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54041a4670 con 0x7f5404105190 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.120+0000 7f53fb7fe640 1 -- 172.21.15.37:0/1800045465 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f53fc005520 con 0x7f5404105190 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.120+0000 7f540efab740 1 Processor -- start 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540efab740 1 -- start start 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f53fb7fe640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53f00a2a10 0x7f53f00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540efab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd905aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540efab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd905b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540efab740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd9066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540efab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd8fbf5f0 con 0x55cbd905bd40 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540efab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd8f68930 con 0x55cbd905afd0 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540efab740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd8fbb690 con 0x55cbd9064550 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f5409968640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53f00a2a10 0x7f53f00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f53fb7fe640 1 -- 172.21.15.37:0/1800045465 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f53fc0cf400 con 0x7f5404105190 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f5409968640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd905b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540a169640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd905aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f5409968640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd905b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:56986/0 (socket says 172.21.15.37:56986) 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f5409968640 1 -- 172.21.15.37:0/1821092451 learned_addr learned my addr 172.21.15.37:0/1821092451 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540a96a640 1 --2- 172.21.15.37:0/1821092451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd9066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540a169640 1 -- 172.21.15.37:0/1821092451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 msgr2=0x55cbd9066940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540a169640 1 --2- 172.21.15.37:0/1821092451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd9066940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540a169640 1 -- 172.21.15.37:0/1821092451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 msgr2=0x55cbd905b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540a169640 1 --2- 172.21.15.37:0/1821092451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd905b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540a169640 1 -- 172.21.15.37:0/1821092451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbd8cd4210 con 0x55cbd905bd40 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540a96a640 1 --2- 172.21.15.37:0/1821092451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd9066940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f5409968640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53f00a2a10 0x7f53f00a4ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f53f4005ec0 tx=0x7f53f402c5d0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.121+0000 7f540a169640 1 --2- 172.21.15.37:0/1821092451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd905aa90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5400009b40 tx=0x7f54000113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8c67fab9cbeee7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.122+0000 7f53f8bfe640 1 -- 172.21.15.37:0/1821092451 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400011d90 con 0x55cbd905bd40 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.122+0000 7f53f8bfe640 1 -- 172.21.15.37:0/1821092451 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5400012930 con 0x55cbd905bd40 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.122+0000 7f53f8bfe640 1 -- 172.21.15.37:0/1821092451 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540000f9f0 con 0x55cbd905bd40 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.122+0000 7f540efab740 1 -- 172.21.15.37:0/1821092451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 msgr2=0x55cbd905aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.122+0000 7f540efab740 1 --2- 172.21.15.37:0/1821092451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd905aa90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5400009b40 tx=0x7f54000113e0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.122+0000 7f540efab740 1 -- 172.21.15.37:0/1821092451 shutdown_connections 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.122+0000 7f540efab740 1 --2- 172.21.15.37:0/1821092451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd9066940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.122+0000 7f540efab740 1 --2- 172.21.15.37:0/1821092451 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd905b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.122+0000 7f540efab740 1 --2- 172.21.15.37:0/1821092451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd905aa90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.122+0000 7f540efab740 1 -- 172.21.15.37:0/1821092451 >> 172.21.15.37:0/1821092451 conn(0x55cbd90225b0 msgr2=0x55cbd9055cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540efab740 1 -- 172.21.15.37:0/1821092451 shutdown_connections 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540efab740 1 -- 172.21.15.37:0/1821092451 wait complete. 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540efab740 1 -- start start 2024-03-26T06:28:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540efab740 1 Processor -- start 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540efab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd9061560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540efab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9061aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540efab740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd9061fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540efab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd9067280 con 0x55cbd905bd40 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540efab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd8f7bd90 con 0x55cbd905afd0 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540efab740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd9067400 con 0x55cbd9064550 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540efab740 10 client.0 _ll_register_callbacks cb 0x55cbd8cd3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f5409968640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9061aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f5409968640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9061aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58584/0 (socket says 172.21.15.37:58584) 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f5409968640 1 -- 172.21.15.37:0/1971350900 learned_addr learned my addr 172.21.15.37:0/1971350900 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540a169640 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd9061560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.123+0000 7f540a96a640 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd9061fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f5409968640 1 -- 172.21.15.37:0/1971350900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 msgr2=0x55cbd9061fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f5409968640 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd9061fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f5409968640 1 -- 172.21.15.37:0/1971350900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 msgr2=0x55cbd9061560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f5409968640 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd9061560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f5409968640 1 -- 172.21.15.37:0/1971350900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbd8da0620 con 0x55cbd905bd40 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f5409968640 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9061aa0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009230 tx=0x7f53f402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d43055f0146e73 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f53edffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f53edffb640 1 -- 172.21.15.37:0/1971350900 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4053060 con 0x55cbd905bd40 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f540efab740 1 -- 172.21.15.37:0/1971350900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5400012930 con 0x55cbd905bd40 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f53edffb640 1 -- 172.21.15.37:0/1971350900 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f53f4004ec0 con 0x55cbd905bd40 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f53edffb640 1 -- 172.21.15.37:0/1971350900 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53f4004c40 con 0x55cbd905bd40 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f540efab740 10 client.15939 fetch_fsmap learned FSMap version 38 2024-03-26T06:28:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f540efab740 1 -- 172.21.15.37:0/1971350900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55cbd9073580 con 0x55cbd905bd40 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.124+0000 7f53edffb640 1 -- 172.21.15.37:0/1971350900 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5400012930 con 0x55cbd905bd40 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.125+0000 7f53edffb640 1 -- 172.21.15.37:0/1971350900 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f53f4043af0 con 0x55cbd905bd40 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.125+0000 7f540efab740 10 client.15939 fetch_fsmap finished waiting for FSMap version 38 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.125+0000 7f540efab740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.125+0000 7f540efab740 20 client.15939 populate_metadata read hostname 'smithi037' 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.125+0000 7f540efab740 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbd90749f0 0x55cbd9076de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.125+0000 7f540efab740 4 client.15939 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.125+0000 7f540efab740 1 -- 172.21.15.37:0/1971350900 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbd8fccb30 con 0x55cbd90749f0 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.125+0000 7f540a169640 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbd90749f0 0x55cbd9076de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.125+0000 7f540a169640 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbd90749f0 0x55cbd9076de0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19e380f3e6c07ebd server_cookie=a242b2f888df98f6 in_seq=0 out_seq=0 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.125+0000 7f53edffb640 10 client.15939 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f53edffb640 1 -- 172.21.15.37:0/1971350900 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbd8fccb30 con 0x55cbd90749f0 2024-03-26T06:28:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f53edffb640 10 client.15939 handle_command_reply: tid=0 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 1 client.15939 shutdown 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 1 -- 172.21.15.37:0/1971350900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 msgr2=0x55cbd9061aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9061aa0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009230 tx=0x7f53f402bc30 comp rx=0 tx=0).stop 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 1 -- 172.21.15.37:0/1971350900 shutdown_connections 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbd90749f0 0x55cbd9076de0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd9061fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9061aa0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f53edffb640 0 client.15939 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 1 --2- 172.21.15.37:0/1971350900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd9061560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 1 -- 172.21.15.37:0/1971350900 >> 172.21.15.37:0/1971350900 conn(0x55cbd90225b0 msgr2=0x55cbd9057a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 1 -- 172.21.15.37:0/1971350900 shutdown_connections 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 1 -- 172.21.15.37:0/1971350900 wait complete. 2024-03-26T06:28:30.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.126+0000 7f540efab740 20 client.15939 trim_cache size 0 max 16384 2024-03-26T06:28:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.140+0000 7f540efab740 1 Processor -- start 2024-03-26T06:28:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.140+0000 7f540efab740 1 -- start start 2024-03-26T06:28:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.140+0000 7f540efab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd905fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.140+0000 7f540efab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9060370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.140+0000 7f540efab740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd905e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.140+0000 7f540efab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041acaf0 con 0x55cbd905bd40 2024-03-26T06:28:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.140+0000 7f540efab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041ae470 con 0x55cbd905afd0 2024-03-26T06:28:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.140+0000 7f540efab740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041ad150 con 0x55cbd9064550 2024-03-26T06:28:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f5409968640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9060370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f540a169640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd905fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f5409968640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9060370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58600/0 (socket says 172.21.15.37:58600) 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f5409968640 1 -- 172.21.15.37:0/3526834270 learned_addr learned my addr 172.21.15.37:0/3526834270 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f540a96a640 1 --2- 172.21.15.37:0/3526834270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd905e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f5409968640 1 -- 172.21.15.37:0/3526834270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 msgr2=0x55cbd905e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f5409968640 1 --2- 172.21.15.37:0/3526834270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd905e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f5409968640 1 -- 172.21.15.37:0/3526834270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 msgr2=0x55cbd905fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f5409968640 1 --2- 172.21.15.37:0/3526834270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd905fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f5409968640 1 -- 172.21.15.37:0/3526834270 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbd9077150 con 0x55cbd905bd40 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f540a169640 1 --2- 172.21.15.37:0/3526834270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd905fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f5409968640 1 --2- 172.21.15.37:0/3526834270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9060370 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f53f4005e30 tx=0x7f53f402ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28255326b457aad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f53effff640 1 -- 172.21.15.37:0/3526834270 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4053040 con 0x55cbd905bd40 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f53effff640 1 -- 172.21.15.37:0/3526834270 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53f4004a90 con 0x55cbd905bd40 2024-03-26T06:28:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.141+0000 7f53effff640 1 -- 172.21.15.37:0/3526834270 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4005380 con 0x55cbd905bd40 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.142+0000 7f540efab740 1 -- 172.21.15.37:0/3526834270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 msgr2=0x55cbd9060370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.142+0000 7f540efab740 1 --2- 172.21.15.37:0/3526834270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9060370 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f53f4005e30 tx=0x7f53f402ce40 comp rx=0 tx=0).stop 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.142+0000 7f540efab740 1 -- 172.21.15.37:0/3526834270 shutdown_connections 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.142+0000 7f540efab740 1 --2- 172.21.15.37:0/3526834270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd9064550 0x55cbd905e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.142+0000 7f540efab740 1 --2- 172.21.15.37:0/3526834270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd905bd40 0x55cbd9060370 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.142+0000 7f540efab740 1 --2- 172.21.15.37:0/3526834270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905afd0 0x55cbd905fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.142+0000 7f540efab740 1 -- 172.21.15.37:0/3526834270 >> 172.21.15.37:0/3526834270 conn(0x55cbd90225b0 msgr2=0x55cbd905c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.142+0000 7f540efab740 1 -- 172.21.15.37:0/3526834270 shutdown_connections 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.142+0000 7f540efab740 1 -- 172.21.15.37:0/3526834270 wait complete. 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.142+0000 7f540efab740 1 -- start start 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.142+0000 7f540efab740 1 Processor -- start 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540efab740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd905afd0 0x55cbd906b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540efab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905bd40 0x55cbd9069d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540efab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd9064550 0x55cbd906a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540efab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041ade10 con 0x55cbd9064550 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540efab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041ad7b0 con 0x55cbd905bd40 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540efab740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041aad90 con 0x55cbd905afd0 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540efab740 10 client.0 _ll_register_callbacks cb 0x7f54000095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540a96a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd9064550 0x55cbd906a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f5409968640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905bd40 0x55cbd9069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540a96a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd9064550 0x55cbd906a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58606/0 (socket says 172.21.15.37:58606) 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f5409968640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905bd40 0x55cbd9069d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:57006/0 (socket says 172.21.15.37:57006) 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540a96a640 1 -- 172.21.15.37:0/2381760930 learned_addr learned my addr 172.21.15.37:0/2381760930 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540a169640 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd905afd0 0x55cbd906b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540a96a640 1 -- 172.21.15.37:0/2381760930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd905afd0 msgr2=0x55cbd906b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:30.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540a96a640 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd905afd0 0x55cbd906b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540a96a640 1 -- 172.21.15.37:0/2381760930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905bd40 msgr2=0x55cbd9069d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540a96a640 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905bd40 0x55cbd9069d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540a96a640 1 -- 172.21.15.37:0/2381760930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbd906aa50 con 0x55cbd9064550 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540a169640 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd905afd0 0x55cbd906b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f5409968640 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905bd40 0x55cbd9069d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f540a96a640 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd9064550 0x55cbd906a250 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f53fc098400 tx=0x7f53fc09baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f9c814d598a96b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.143+0000 7f53edffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f53edffb640 1 -- 172.21.15.37:0/2381760930 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0ccfa0 con 0x55cbd9064550 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f53edffb640 1 -- 172.21.15.37:0/2381760930 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f53fc0a3d40 con 0x55cbd9064550 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f540efab740 1 -- 172.21.15.37:0/2381760930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53f4004a90 con 0x55cbd9064550 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f53edffb640 1 -- 172.21.15.37:0/2381760930 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53fc0a3f40 con 0x55cbd9064550 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f540efab740 10 client.15945 fetch_fsmap learned FSMap version 38 2024-03-26T06:28:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f540efab740 1 -- 172.21.15.37:0/2381760930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55cbd90cbf10 con 0x55cbd9064550 2024-03-26T06:28:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f53edffb640 1 -- 172.21.15.37:0/2381760930 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53f4004a90 con 0x55cbd9064550 2024-03-26T06:28:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f53edffb640 1 -- 172.21.15.37:0/2381760930 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f53fc097780 con 0x55cbd9064550 2024-03-26T06:28:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f540efab740 10 client.15945 fetch_fsmap finished waiting for FSMap version 38 2024-03-26T06:28:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f540efab740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:28:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f540efab740 20 client.15945 populate_metadata read hostname 'smithi037' 2024-03-26T06:28:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f540efab740 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbd90cc5e0 0x55cbd90cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f540efab740 4 client.15945 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:28:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f540efab740 1 -- 172.21.15.37:0/2381760930 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cbd8fc6e80 con 0x55cbd90cc5e0 2024-03-26T06:28:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.144+0000 7f540a169640 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbd90cc5e0 0x55cbd90cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.145+0000 7f540a169640 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbd90cc5e0 0x55cbd90cea00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989b11faf7090ba2 server_cookie=b7e5bab9b6f7f197 in_seq=0 out_seq=0 2024-03-26T06:28:30.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.145+0000 7f53edffb640 10 client.15945 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.191+0000 7f53edffb640 1 -- 172.21.15.37:0/2381760930 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cbd8fc6e80 con 0x55cbd90cc5e0 2024-03-26T06:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.191+0000 7f53edffb640 10 client.15945 handle_command_reply: tid=0 2024-03-26T06:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.191+0000 7f540efab740 1 client.15945 shutdown 2024-03-26T06:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.191+0000 7f540efab740 1 -- 172.21.15.37:0/2381760930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd9064550 msgr2=0x55cbd906a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.191+0000 7f540efab740 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd9064550 0x55cbd906a250 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f53fc098400 tx=0x7f53fc09baf0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.191+0000 7f540efab740 1 -- 172.21.15.37:0/2381760930 shutdown_connections 2024-03-26T06:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540efab740 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbd90cc5e0 0x55cbd90cea00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540efab740 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbd9064550 0x55cbd906a250 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540efab740 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbd905bd40 0x55cbd9069d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540efab740 1 --2- 172.21.15.37:0/2381760930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbd905afd0 0x55cbd906b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f53edffb640 0 client.15945 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540efab740 1 -- 172.21.15.37:0/2381760930 >> 172.21.15.37:0/2381760930 conn(0x55cbd90225b0 msgr2=0x55cbd905fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540efab740 1 -- 172.21.15.37:0/2381760930 shutdown_connections 2024-03-26T06:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540efab740 1 -- 172.21.15.37:0/2381760930 wait complete. 2024-03-26T06:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540efab740 20 client.15945 trim_cache size 0 max 16384 2024-03-26T06:28:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:28:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:28:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:28:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540c3cd640 1 -- 172.21.15.37:0/1800045465 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53f00a2a10 msgr2=0x7f53f00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540c3cd640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53f00a2a10 0x7f53f00a4ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f53f4005ec0 tx=0x7f53f402c5d0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540c3cd640 1 -- 172.21.15.37:0/1800045465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5404105190 msgr2=0x7f5404197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540c3cd640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5404105190 0x7f5404197960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00ea60 tx=0x7f53fc00ef20 comp rx=0 tx=0).stop 2024-03-26T06:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540c3cd640 1 -- 172.21.15.37:0/1800045465 shutdown_connections 2024-03-26T06:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540c3cd640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53f00a2a10 0x7f53f00a4ed0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540c3cd640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5404105190 0x7f5404197960 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540c3cd640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5404100e00 0x7f540419dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540c3cd640 1 --2- 172.21.15.37:0/1800045465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5404100440 0x7f540419d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.192+0000 7f540c3cd640 1 -- 172.21.15.37:0/1800045465 >> 172.21.15.37:0/1800045465 conn(0x7f5404077040 msgr2=0x7f5404104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.193+0000 7f540c3cd640 1 -- 172.21.15.37:0/1800045465 shutdown_connections 2024-03-26T06:28:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:30.193+0000 7f540c3cd640 1 -- 172.21.15.37:0/1800045465 wait complete. 2024-03-26T06:28:30.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 174489 4.28 MB/sec execute 125 sec latency 133.525 ms 2024-03-26T06:28:31.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:30 smithi066 ceph-mon[36504]: pgmap v168: 97 pgs: 97 active+clean; 66 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-03-26T06:28:31.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:30 smithi149 ceph-mon[36681]: pgmap v168: 97 pgs: 97 active+clean; 66 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-03-26T06:28:31.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:30 smithi037 ceph-mon[31974]: pgmap v168: 97 pgs: 97 active+clean; 66 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-03-26T06:28:31.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 174822 4.25 MB/sec execute 126 sec latency 935.988 ms 2024-03-26T06:28:32.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 177170 4.32 MB/sec execute 127 sec latency 28.638 ms 2024-03-26T06:28:33.000 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:32 smithi037 ceph-mon[31974]: pgmap v169: 97 pgs: 97 active+clean; 70 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 288 op/s 2024-03-26T06:28:33.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:32 smithi066 ceph-mon[36504]: pgmap v169: 97 pgs: 97 active+clean; 70 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 288 op/s 2024-03-26T06:28:33.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:32 smithi149 ceph-mon[36681]: pgmap v169: 97 pgs: 97 active+clean; 70 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 288 op/s 2024-03-26T06:28:33.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 178553 4.34 MB/sec execute 128 sec latency 11.894 ms 2024-03-26T06:28:34.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 180279 4.39 MB/sec execute 129 sec latency 13.899 ms 2024-03-26T06:28:35.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:34 smithi066 ceph-mon[36504]: pgmap v170: 97 pgs: 97 active+clean; 70 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-03-26T06:28:35.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:34 smithi149 ceph-mon[36681]: pgmap v170: 97 pgs: 97 active+clean; 70 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-03-26T06:28:35.202 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:34 smithi037 ceph-mon[31974]: pgmap v170: 97 pgs: 97 active+clean; 70 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-03-26T06:28:35.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 181702 4.41 MB/sec execute 130 sec latency 11.881 ms 2024-03-26T06:28:36.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:35 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:36.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:35 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:36.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:35 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:36.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 182055 4.39 MB/sec execute 131 sec latency 798.940 ms 2024-03-26T06:28:37.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:36 smithi066 ceph-mon[36504]: pgmap v171: 97 pgs: 97 active+clean; 67 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-03-26T06:28:37.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:36 smithi149 ceph-mon[36681]: pgmap v171: 97 pgs: 97 active+clean; 67 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-03-26T06:28:37.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:36 smithi037 ceph-mon[31974]: pgmap v171: 97 pgs: 97 active+clean; 67 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-03-26T06:28:37.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 183833 4.43 MB/sec execute 132 sec latency 17.043 ms 2024-03-26T06:28:38.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 185256 4.45 MB/sec execute 133 sec latency 13.006 ms 2024-03-26T06:28:39.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:38 smithi066 ceph-mon[36504]: pgmap v172: 97 pgs: 97 active+clean; 67 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-03-26T06:28:39.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:38 smithi149 ceph-mon[36681]: pgmap v172: 97 pgs: 97 active+clean; 67 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-03-26T06:28:39.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:38 smithi037 ceph-mon[31974]: pgmap v172: 97 pgs: 97 active+clean; 67 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-03-26T06:28:39.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 186815 4.47 MB/sec execute 134 sec latency 27.161 ms 2024-03-26T06:28:40.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 188440 4.53 MB/sec execute 135 sec latency 10.253 ms 2024-03-26T06:28:41.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:40 smithi066 ceph-mon[36504]: pgmap v173: 97 pgs: 97 active+clean; 67 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-03-26T06:28:41.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:40 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:41.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:40 smithi149 ceph-mon[36681]: pgmap v173: 97 pgs: 97 active+clean; 67 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-03-26T06:28:41.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:40 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:41.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:40 smithi037 ceph-mon[31974]: pgmap v173: 97 pgs: 97 active+clean; 67 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-03-26T06:28:41.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:40 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:28:41.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 189701 4.53 MB/sec execute 136 sec latency 17.165 ms 2024-03-26T06:28:42.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 191771 4.60 MB/sec execute 137 sec latency 25.152 ms 2024-03-26T06:28:43.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:42 smithi066 ceph-mon[36504]: pgmap v174: 97 pgs: 97 active+clean; 71 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-03-26T06:28:43.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:42 smithi149 ceph-mon[36681]: pgmap v174: 97 pgs: 97 active+clean; 71 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-03-26T06:28:43.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:42 smithi037 ceph-mon[31974]: pgmap v174: 97 pgs: 97 active+clean; 71 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-03-26T06:28:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.287+0000 7ff23b337640 1 -- 172.21.15.37:0/461814392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234100e00 msgr2=0x7ff23410d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.287+0000 7ff23b337640 1 --2- 172.21.15.37:0/461814392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234100e00 0x7ff23410d020 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009ca0 tx=0x7ff22c02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.287+0000 7ff23b337640 1 -- 172.21.15.37:0/461814392 shutdown_connections 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.287+0000 7ff23b337640 1 --2- 172.21.15.37:0/461814392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234100e00 0x7ff23410d020 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.287+0000 7ff23b337640 1 --2- 172.21.15.37:0/461814392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff234100440 0x7ff2341008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.287+0000 7ff23b337640 1 --2- 172.21.15.37:0/461814392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff234105190 0x7ff234105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.287+0000 7ff23b337640 1 -- 172.21.15.37:0/461814392 >> 172.21.15.37:0/461814392 conn(0x7ff234077040 msgr2=0x7ff234077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.287+0000 7ff23b337640 1 -- 172.21.15.37:0/461814392 shutdown_connections 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.287+0000 7ff23b337640 1 -- 172.21.15.37:0/461814392 wait complete. 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff23b337640 1 Processor -- start 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff23b337640 1 -- start start 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff23b337640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff234100440 0x7ff234101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff23b337640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234100e00 0x7ff2341023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff23b337640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff234105190 0x7ff2341028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff23b337640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff234110650 con 0x7ff234100e00 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff23b337640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341104d0 con 0x7ff234100440 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff23b337640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341107d0 con 0x7ff234105190 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff2388d2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234100e00 0x7ff2341023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff2388d2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234100e00 0x7ff2341023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37298/0 (socket says 172.21.15.37:37298) 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff2390d3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff234100440 0x7ff234101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff2388d2640 1 -- 172.21.15.37:0/3615152937 learned_addr learned my addr 172.21.15.37:0/3615152937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.288+0000 7ff2398d4640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff234105190 0x7ff2341028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff2388d2640 1 -- 172.21.15.37:0/3615152937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff234105190 msgr2=0x7ff2341028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff2388d2640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff234105190 0x7ff2341028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff2388d2640 1 -- 172.21.15.37:0/3615152937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff234100440 msgr2=0x7ff234101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff2388d2640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff234100440 0x7ff234101e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff2388d2640 1 -- 172.21.15.37:0/3615152937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2341a60b0 con 0x7ff234100e00 2024-03-26T06:28:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff2390d3640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff234100440 0x7ff234101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:28:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff2388d2640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234100e00 0x7ff2341023a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff22400e9f0 tx=0x7ff22400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff22a7fc640 1 -- 172.21.15.37:0/3615152937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22400cd80 con 0x7ff234100e00 2024-03-26T06:28:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff22a7fc640 1 -- 172.21.15.37:0/3615152937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff224010910 con 0x7ff234100e00 2024-03-26T06:28:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff23b337640 1 -- 172.21.15.37:0/3615152937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2341a63a0 con 0x7ff234100e00 2024-03-26T06:28:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff22a7fc640 1 -- 172.21.15.37:0/3615152937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224018990 con 0x7ff234100e00 2024-03-26T06:28:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.289+0000 7ff23b337640 1 -- 172.21.15.37:0/3615152937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2341a68f0 con 0x7ff234100e00 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.291+0000 7ff22a7fc640 1 -- 172.21.15.37:0/3615152937 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff224010ac0 con 0x7ff234100e00 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff23dfee740 1 Processor -- start 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff23dfee740 1 -- start start 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff22a7fc640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2200a2a10 0x7ff2200a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 0x5653d40fda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d40fdfd0 0x5653d40fe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d4107550 0x5653d4109940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff23dfee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d40625f0 con 0x5653d40fed40 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff23dfee740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d400b930 con 0x5653d4107550 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff23dfee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d405e690 con 0x5653d40fdfd0 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff2390d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 0x5653d40fda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff2398d4640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2200a2a10 0x7ff2200a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff2390d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 0x5653d40fda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37312/0 (socket says 172.21.15.37:37312) 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff2390d3640 1 -- 172.21.15.37:0/1408971419 learned_addr learned my addr 172.21.15.37:0/1408971419 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.292+0000 7ff2388d2640 1 --2- 172.21.15.37:0/1408971419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d40fdfd0 0x5653d40fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff2398d4640 1 --2- 172.21.15.37:0/1408971419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d4107550 0x5653d4109940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff22a7fc640 1 -- 172.21.15.37:0/3615152937 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff2240cf190 con 0x7ff234100e00 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff2390d3640 1 -- 172.21.15.37:0/1408971419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d40fdfd0 msgr2=0x5653d40fe450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff2390d3640 1 --2- 172.21.15.37:0/1408971419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d40fdfd0 0x5653d40fe450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff2390d3640 1 -- 172.21.15.37:0/1408971419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d4107550 msgr2=0x5653d4109940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff2390d3640 1 --2- 172.21.15.37:0/1408971419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d4107550 0x5653d4109940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff2390d3640 1 -- 172.21.15.37:0/1408971419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d3d77210 con 0x5653d40fed40 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff2398d4640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2200a2a10 0x7ff2200a4ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff22c004af0 tx=0x7ff22c0323c0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff2390d3640 1 --2- 172.21.15.37:0/1408971419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 0x5653d40fda90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff2300114c0 tx=0x7ff230011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621e6f506fc3ca03 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff21f7fe640 1 -- 172.21.15.37:0/1408971419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230011030 con 0x5653d40fed40 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff21f7fe640 1 -- 172.21.15.37:0/1408971419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2300111d0 con 0x5653d40fed40 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.293+0000 7ff21f7fe640 1 -- 172.21.15.37:0/1408971419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000f5c0 con 0x5653d40fed40 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.294+0000 7ff23dfee740 1 -- 172.21.15.37:0/1408971419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 msgr2=0x5653d40fda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.294+0000 7ff23dfee740 1 --2- 172.21.15.37:0/1408971419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 0x5653d40fda90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff2300114c0 tx=0x7ff230011980 comp rx=0 tx=0).stop 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.294+0000 7ff23dfee740 1 -- 172.21.15.37:0/1408971419 shutdown_connections 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.294+0000 7ff23dfee740 1 --2- 172.21.15.37:0/1408971419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d4107550 0x5653d4109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.294+0000 7ff23dfee740 1 --2- 172.21.15.37:0/1408971419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d40fdfd0 0x5653d40fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.294+0000 7ff23dfee740 1 --2- 172.21.15.37:0/1408971419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 0x5653d40fda90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.294+0000 7ff23dfee740 1 -- 172.21.15.37:0/1408971419 >> 172.21.15.37:0/1408971419 conn(0x5653d40c55b0 msgr2=0x5653d40f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.294+0000 7ff23dfee740 1 -- 172.21.15.37:0/1408971419 shutdown_connections 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.294+0000 7ff23dfee740 1 -- 172.21.15.37:0/1408971419 wait complete. 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.294+0000 7ff23dfee740 1 -- start start 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.294+0000 7ff23dfee740 1 Processor -- start 2024-03-26T06:28:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4104540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d40fed40 0x5653d4104a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d4107550 0x5653d4104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff23dfee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d410a280 con 0x5653d40fdfd0 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff23dfee740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d401ed90 con 0x5653d40fed40 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff23dfee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d410a400 con 0x5653d4107550 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff2390d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff2390d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4104540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37318/0 (socket says 172.21.15.37:37318) 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff2390d3640 1 -- 172.21.15.37:0/1086363705 learned_addr learned my addr 172.21.15.37:0/1086363705 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff23dfee740 10 client.0 _ll_register_callbacks cb 0x5653d3d76a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff2388d2640 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d40fed40 0x5653d4104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff2398d4640 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d4107550 0x5653d4104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff2390d3640 1 -- 172.21.15.37:0/1086363705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d4107550 msgr2=0x5653d4104fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff2390d3640 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d4107550 0x5653d4104fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff2390d3640 1 -- 172.21.15.37:0/1086363705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d40fed40 msgr2=0x5653d4104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff2390d3640 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d40fed40 0x5653d4104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff2390d3640 1 -- 172.21.15.37:0/1086363705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653d3e43620 con 0x5653d40fdfd0 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff2390d3640 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4104540 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff2300058c0 tx=0x7ff230003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe22efef573547 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.295+0000 7ff21cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff21cff9640 1 -- 172.21.15.37:0/1086363705 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23001d040 con 0x5653d40fdfd0 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff21cff9640 1 -- 172.21.15.37:0/1086363705 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff230004110 con 0x5653d40fdfd0 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff23dfee740 1 -- 172.21.15.37:0/1086363705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2300111d0 con 0x5653d40fdfd0 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff21cff9640 1 -- 172.21.15.37:0/1086363705 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff230030050 con 0x5653d40fdfd0 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff23dfee740 10 client.15957 fetch_fsmap learned FSMap version 40 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff23dfee740 1 -- 172.21.15.37:0/1086363705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5653d41164f0 con 0x5653d40fdfd0 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff21cff9640 1 -- 172.21.15.37:0/1086363705 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2300111d0 con 0x5653d40fdfd0 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff21cff9640 1 -- 172.21.15.37:0/1086363705 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff2300047c0 con 0x5653d40fdfd0 2024-03-26T06:28:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff23dfee740 10 client.15957 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff23dfee740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff23dfee740 20 client.15957 populate_metadata read hostname 'smithi037' 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff23dfee740 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5653d4117950 0x5653d4119d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff23dfee740 4 client.15957 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff23dfee740 1 -- 172.21.15.37:0/1086363705 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653d406fb30 con 0x5653d4117950 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.296+0000 7ff2388d2640 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5653d4117950 0x5653d4119d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.297+0000 7ff2388d2640 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5653d4117950 0x5653d4119d40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ef0838bc4ff2de server_cookie=737e1f44ec7717a8 in_seq=0 out_seq=0 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.297+0000 7ff21cff9640 10 client.15957 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.297+0000 7ff21cff9640 1 -- 172.21.15.37:0/1086363705 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653d406fb30 con 0x5653d4117950 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.297+0000 7ff21cff9640 10 client.15957 handle_command_reply: tid=0 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.297+0000 7ff23dfee740 1 client.15957 shutdown 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.297+0000 7ff23dfee740 1 -- 172.21.15.37:0/1086363705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 msgr2=0x5653d4104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.297+0000 7ff23dfee740 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4104540 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff2300058c0 tx=0x7ff230003f20 comp rx=0 tx=0).stop 2024-03-26T06:28:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.298+0000 7ff23dfee740 1 -- 172.21.15.37:0/1086363705 shutdown_connections 2024-03-26T06:28:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.298+0000 7ff23dfee740 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5653d4117950 0x5653d4119d40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.298+0000 7ff23dfee740 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d4107550 0x5653d4104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.298+0000 7ff21cff9640 0 client.15957 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.298+0000 7ff23dfee740 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d40fed40 0x5653d4104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.298+0000 7ff23dfee740 1 --2- 172.21.15.37:0/1086363705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4104540 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.298+0000 7ff23dfee740 1 -- 172.21.15.37:0/1086363705 >> 172.21.15.37:0/1086363705 conn(0x5653d40c55b0 msgr2=0x5653d40faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.298+0000 7ff23dfee740 1 -- 172.21.15.37:0/1086363705 shutdown_connections 2024-03-26T06:28:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.298+0000 7ff23dfee740 1 -- 172.21.15.37:0/1086363705 wait complete. 2024-03-26T06:28:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.298+0000 7ff23dfee740 20 client.15957 trim_cache size 0 max 16384 2024-03-26T06:28:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff23dfee740 1 Processor -- start 2024-03-26T06:28:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff23dfee740 1 -- start start 2024-03-26T06:28:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4106c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d40fed40 0x5653d4103e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d4107550 0x5653d40fa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff23dfee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341aed80 con 0x5653d40fdfd0 2024-03-26T06:28:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff23dfee740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341b07c0 con 0x5653d40fed40 2024-03-26T06:28:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff23dfee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341af3e0 con 0x5653d4107550 2024-03-26T06:28:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff2390d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4106c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff2390d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4106c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37322/0 (socket says 172.21.15.37:37322) 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff2390d3640 1 -- 172.21.15.37:0/2203010029 learned_addr learned my addr 172.21.15.37:0/2203010029 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff2388d2640 1 --2- 172.21.15.37:0/2203010029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d40fed40 0x5653d4103e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff2398d4640 1 --2- 172.21.15.37:0/2203010029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d4107550 0x5653d40fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff2388d2640 1 -- 172.21.15.37:0/2203010029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d4107550 msgr2=0x5653d40fa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff2388d2640 1 --2- 172.21.15.37:0/2203010029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d4107550 0x5653d40fa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff2388d2640 1 -- 172.21.15.37:0/2203010029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 msgr2=0x5653d4106c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff2388d2640 1 --2- 172.21.15.37:0/2203010029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4106c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff2388d2640 1 -- 172.21.15.37:0/2203010029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d411a0b0 con 0x5653d40fed40 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.312+0000 7ff2390d3640 1 --2- 172.21.15.37:0/2203010029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.313+0000 7ff2388d2640 1 --2- 172.21.15.37:0/2203010029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d40fed40 0x5653d4103e20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff2240cc910 tx=0x7ff224018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beb7438be88cfb49 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.313+0000 7ff21effd640 1 -- 172.21.15.37:0/2203010029 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224099970 con 0x5653d40fed40 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.313+0000 7ff21effd640 1 -- 172.21.15.37:0/2203010029 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff224099bd0 con 0x5653d40fed40 2024-03-26T06:28:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.313+0000 7ff21effd640 1 -- 172.21.15.37:0/2203010029 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224018b30 con 0x5653d40fed40 2024-03-26T06:28:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 -- 172.21.15.37:0/2203010029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d40fed40 msgr2=0x5653d4103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 --2- 172.21.15.37:0/2203010029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d40fed40 0x5653d4103e20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff2240cc910 tx=0x7ff224018420 comp rx=0 tx=0).stop 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 -- 172.21.15.37:0/2203010029 shutdown_connections 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 --2- 172.21.15.37:0/2203010029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d4107550 0x5653d40fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 --2- 172.21.15.37:0/2203010029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d40fed40 0x5653d4103e20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 --2- 172.21.15.37:0/2203010029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fdfd0 0x5653d4106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 -- 172.21.15.37:0/2203010029 >> 172.21.15.37:0/2203010029 conn(0x5653d40c55b0 msgr2=0x5653d40fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 -- 172.21.15.37:0/2203010029 shutdown_connections 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 -- 172.21.15.37:0/2203010029 wait complete. 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 -- start start 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 Processor -- start 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d40fdfd0 0x5653d410e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 0x5653d410eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d4107550 0x5653d410cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341b00a0 con 0x5653d40fed40 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341afa40 con 0x5653d4107550 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341ad020 con 0x5653d40fdfd0 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff23dfee740 10 client.0 _ll_register_callbacks cb 0x7ff230009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.314+0000 7ff2388d2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 0x5653d410eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff2390d3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d40fdfd0 0x5653d410e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff2398d4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d4107550 0x5653d410cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff2388d2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 0x5653d410eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37332/0 (socket says 172.21.15.37:37332) 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff2388d2640 1 -- 172.21.15.37:0/3209668847 learned_addr learned my addr 172.21.15.37:0/3209668847 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff2390d3640 1 -- 172.21.15.37:0/3209668847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d4107550 msgr2=0x5653d410cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff2390d3640 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d4107550 0x5653d410cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff2390d3640 1 -- 172.21.15.37:0/3209668847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 msgr2=0x5653d410eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff2390d3640 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 0x5653d410eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff2390d3640 1 -- 172.21.15.37:0/3209668847 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653d410d3f0 con 0x5653d40fdfd0 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff2398d4640 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d4107550 0x5653d410cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff2390d3640 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d40fdfd0 0x5653d410e650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff2300058c0 tx=0x7ff230009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=641c3d8a63a916d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff21cff9640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff21cff9640 1 -- 172.21.15.37:0/3209668847 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23001d040 con 0x5653d40fdfd0 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff23dfee740 1 -- 172.21.15.37:0/3209668847 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff224099bd0 con 0x5653d40fdfd0 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff21cff9640 1 -- 172.21.15.37:0/3209668847 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff23000f840 con 0x5653d40fdfd0 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff21cff9640 1 -- 172.21.15.37:0/3209668847 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff230011340 con 0x5653d40fdfd0 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff23dfee740 10 client.25390 fetch_fsmap learned FSMap version 40 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.315+0000 7ff23dfee740 1 -- 172.21.15.37:0/3209668847 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5653d416eab0 con 0x5653d40fdfd0 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.316+0000 7ff21cff9640 1 -- 172.21.15.37:0/3209668847 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff224099bd0 con 0x5653d40fdfd0 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.316+0000 7ff21cff9640 1 -- 172.21.15.37:0/3209668847 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff230005460 con 0x5653d40fdfd0 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.316+0000 7ff23dfee740 10 client.25390 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:28:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.316+0000 7ff23dfee740 10 client.25390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:28:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.316+0000 7ff23dfee740 20 client.25390 populate_metadata read hostname 'smithi037' 2024-03-26T06:28:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.316+0000 7ff23dfee740 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5653d416f140 0x5653d4171560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.316+0000 7ff23dfee740 4 client.25390 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:28:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.316+0000 7ff23dfee740 1 -- 172.21.15.37:0/3209668847 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5653d4069e80 con 0x5653d416f140 2024-03-26T06:28:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.316+0000 7ff2388d2640 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5653d416f140 0x5653d4171560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.316+0000 7ff21cff9640 10 client.25390 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.316+0000 7ff2388d2640 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5653d416f140 0x5653d4171560 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdd7b5d26268a9fb server_cookie=869f5b4475c6c4ef in_seq=0 out_seq=0 2024-03-26T06:28:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff21cff9640 1 -- 172.21.15.37:0/3209668847 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5653d4069e80 con 0x5653d416f140 2024-03-26T06:28:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff21cff9640 10 client.25390 handle_command_reply: tid=0 2024-03-26T06:28:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 1 client.25390 shutdown 2024-03-26T06:28:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 1 -- 172.21.15.37:0/3209668847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d40fdfd0 msgr2=0x5653d410e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d40fdfd0 0x5653d410e650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff2300058c0 tx=0x7ff230009fd0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 1 -- 172.21.15.37:0/3209668847 shutdown_connections 2024-03-26T06:28:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5653d416f140 0x5653d4171560 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5653d4107550 0x5653d410cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d40fed40 0x5653d410eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff21cff9640 0 client.25390 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 1 --2- 172.21.15.37:0/3209668847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653d40fdfd0 0x5653d410e650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 1 -- 172.21.15.37:0/3209668847 >> 172.21.15.37:0/3209668847 conn(0x5653d40c55b0 msgr2=0x5653d40f7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 1 -- 172.21.15.37:0/3209668847 shutdown_connections 2024-03-26T06:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 1 -- 172.21.15.37:0/3209668847 wait complete. 2024-03-26T06:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.354+0000 7ff23dfee740 20 client.25390 trim_cache size 0 max 16384 2024-03-26T06:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 -- 172.21.15.37:0/3615152937 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2200a2a10 msgr2=0x7ff2200a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2200a2a10 0x7ff2200a4ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff22c004af0 tx=0x7ff22c0323c0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 -- 172.21.15.37:0/3615152937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234100e00 msgr2=0x7ff2341023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234100e00 0x7ff2341023a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff22400e9f0 tx=0x7ff22400eeb0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 -- 172.21.15.37:0/3615152937 shutdown_connections 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff2200a2a10 0x7ff2200a4ed0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff234105190 0x7ff2341028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234100e00 0x7ff2341023a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 --2- 172.21.15.37:0/3615152937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff234100440 0x7ff234101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 -- 172.21.15.37:0/3615152937 >> 172.21.15.37:0/3615152937 conn(0x7ff234077040 msgr2=0x7ff234068cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 -- 172.21.15.37:0/3615152937 shutdown_connections 2024-03-26T06:28:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:43.355+0000 7ff23b337640 1 -- 172.21.15.37:0/3615152937 wait complete. 2024-03-26T06:28:43.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 193000 4.60 MB/sec execute 138 sec latency 31.382 ms 2024-03-26T06:28:44.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 195322 4.67 MB/sec execute 139 sec latency 16.900 ms 2024-03-26T06:28:45.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:44 smithi066 ceph-mon[36504]: pgmap v175: 97 pgs: 97 active+clean; 71 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-03-26T06:28:45.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:44 smithi149 ceph-mon[36681]: pgmap v175: 97 pgs: 97 active+clean; 71 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-03-26T06:28:45.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:44 smithi037 ceph-mon[31974]: pgmap v175: 97 pgs: 97 active+clean; 71 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-03-26T06:28:45.502 INFO:tasks.workunit.client.0.smithi037.stdout: 1 196548 4.67 MB/sec execute 140 sec latency 15.537 ms 2024-03-26T06:28:46.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 198301 4.70 MB/sec execute 141 sec latency 15.023 ms 2024-03-26T06:28:47.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:46 smithi066 ceph-mon[36504]: pgmap v176: 97 pgs: 97 active+clean; 73 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 334 op/s 2024-03-26T06:28:47.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:46 smithi149 ceph-mon[36681]: pgmap v176: 97 pgs: 97 active+clean; 73 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 334 op/s 2024-03-26T06:28:47.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:46 smithi037 ceph-mon[31974]: pgmap v176: 97 pgs: 97 active+clean; 73 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 334 op/s 2024-03-26T06:28:47.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 199429 4.71 MB/sec execute 142 sec latency 57.560 ms 2024-03-26T06:28:48.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 199429 4.68 MB/sec execute 143 sec latency 1057.654 ms 2024-03-26T06:28:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:48 smithi066 ceph-mon[36504]: pgmap v177: 97 pgs: 97 active+clean; 73 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 217 op/s 2024-03-26T06:28:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:48 smithi149 ceph-mon[36681]: pgmap v177: 97 pgs: 97 active+clean; 73 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 217 op/s 2024-03-26T06:28:49.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:48 smithi037 ceph-mon[31974]: pgmap v177: 97 pgs: 97 active+clean; 73 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 217 op/s 2024-03-26T06:28:49.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 199429 4.65 MB/sec execute 144 sec latency 2057.738 ms 2024-03-26T06:28:50.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 199429 4.61 MB/sec execute 145 sec latency 3057.787 ms 2024-03-26T06:28:51.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:50 smithi066 ceph-mon[36504]: pgmap v178: 97 pgs: 97 active+clean; 73 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 217 op/s 2024-03-26T06:28:51.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:50 smithi149 ceph-mon[36681]: pgmap v178: 97 pgs: 97 active+clean; 73 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 217 op/s 2024-03-26T06:28:51.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:50 smithi037 ceph-mon[31974]: pgmap v178: 97 pgs: 97 active+clean; 73 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 217 op/s 2024-03-26T06:28:51.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 199794 4.59 MB/sec execute 146 sec latency 3858.214 ms 2024-03-26T06:28:52.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:51 smithi066 ceph-mon[36504]: pgmap v179: 97 pgs: 97 active+clean; 77 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-03-26T06:28:52.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:51 smithi149 ceph-mon[36681]: pgmap v179: 97 pgs: 97 active+clean; 77 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-03-26T06:28:52.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:51 smithi037 ceph-mon[31974]: pgmap v179: 97 pgs: 97 active+clean; 77 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-03-26T06:28:52.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 201977 4.65 MB/sec execute 147 sec latency 29.608 ms 2024-03-26T06:28:53.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 202852 4.65 MB/sec execute 148 sec latency 182.833 ms 2024-03-26T06:28:54.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:53 smithi149 ceph-mon[36681]: pgmap v180: 97 pgs: 97 active+clean; 77 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-03-26T06:28:54.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:53 smithi037 ceph-mon[31974]: pgmap v180: 97 pgs: 97 active+clean; 77 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-03-26T06:28:54.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:53 smithi066 ceph-mon[36504]: pgmap v180: 97 pgs: 97 active+clean; 77 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-03-26T06:28:54.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 202852 4.62 MB/sec execute 149 sec latency 1182.940 ms 2024-03-26T06:28:55.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 202852 4.58 MB/sec execute 150 sec latency 2183.057 ms 2024-03-26T06:28:56.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:55 smithi149 ceph-mon[36681]: pgmap v181: 97 pgs: 97 active+clean; 77 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-03-26T06:28:56.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:55 smithi037 ceph-mon[31974]: pgmap v181: 97 pgs: 97 active+clean; 77 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-03-26T06:28:56.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:55 smithi066 ceph-mon[36504]: pgmap v181: 97 pgs: 97 active+clean; 77 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-03-26T06:28:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.455+0000 7faeb0268640 1 -- 172.21.15.37:0/4131312649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faea8105190 msgr2=0x7faea8105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.455+0000 7faeb0268640 1 --2- 172.21.15.37:0/4131312649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faea8105190 0x7faea8105570 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7faea4009900 tx=0x7faea402bc30 comp rx=0 tx=0).stop 2024-03-26T06:28:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.456+0000 7faeb0268640 1 -- 172.21.15.37:0/4131312649 shutdown_connections 2024-03-26T06:28:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.456+0000 7faeb0268640 1 --2- 172.21.15.37:0/4131312649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faea8100e00 0x7faea810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.456+0000 7faeb0268640 1 --2- 172.21.15.37:0/4131312649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea8100440 0x7faea81008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.456+0000 7faeb0268640 1 --2- 172.21.15.37:0/4131312649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faea8105190 0x7faea8105570 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.456+0000 7faeb0268640 1 -- 172.21.15.37:0/4131312649 >> 172.21.15.37:0/4131312649 conn(0x7faea8077040 msgr2=0x7faea8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.456+0000 7faeb0268640 1 -- 172.21.15.37:0/4131312649 shutdown_connections 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.456+0000 7faeb0268640 1 -- 172.21.15.37:0/4131312649 wait complete. 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.456+0000 7faeb0268640 1 Processor -- start 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.456+0000 7faeb0268640 1 -- start start 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faeb0268640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea8100440 0x7faea8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faeb0268640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faea8100e00 0x7faea8198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faeb0268640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faea8105190 0x7faea81a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faeb0268640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea8110650 con 0x7faea8100e00 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faeb0268640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81104d0 con 0x7faea8100440 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faeb0268640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81107d0 con 0x7faea8105190 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faead803640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faea8100e00 0x7faea8198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faead803640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faea8100e00 0x7faea8198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54698/0 (socket says 172.21.15.37:54698) 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faead803640 1 -- 172.21.15.37:0/314340212 learned_addr learned my addr 172.21.15.37:0/314340212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faeae004640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea8100440 0x7faea8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faeae805640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faea8105190 0x7faea81a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faead803640 1 -- 172.21.15.37:0/314340212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faea8105190 msgr2=0x7faea81a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faead803640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faea8105190 0x7faea81a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faead803640 1 -- 172.21.15.37:0/314340212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea8100440 msgr2=0x7faea8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faead803640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea8100440 0x7faea8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faead803640 1 -- 172.21.15.37:0/314340212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea81a3fd0 con 0x7faea8100e00 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faead803640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faea8100e00 0x7faea8198f50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9f0 tx=0x7fae9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7fae9f7fe640 1 -- 172.21.15.37:0/314340212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9800cd80 con 0x7faea8100e00 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7fae9f7fe640 1 -- 172.21.15.37:0/314340212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae98010910 con 0x7faea8100e00 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7faeb0268640 1 -- 172.21.15.37:0/314340212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea81a41a0 con 0x7faea8100e00 2024-03-26T06:28:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.457+0000 7fae9f7fe640 1 -- 172.21.15.37:0/314340212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98018990 con 0x7faea8100e00 2024-03-26T06:28:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.458+0000 7faeb0268640 1 -- 172.21.15.37:0/314340212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea81a4660 con 0x7faea8100e00 2024-03-26T06:28:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.460+0000 7fae9f7fe640 1 -- 172.21.15.37:0/314340212 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fae98018b30 con 0x7faea8100e00 2024-03-26T06:28:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeb2e48740 1 Processor -- start 2024-03-26T06:28:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeb2e48740 1 -- start start 2024-03-26T06:28:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7fae9f7fe640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fae940a2a10 0x7fae940a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dc98a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc98fd0 0x55a22dc99450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 0x55a22dca4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeb2e48740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22dbfd5f0 con 0x55a22dc98fd0 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeb2e48740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22dba6930 con 0x55a22dca2550 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeb2e48740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22dbf9690 con 0x55a22dc99d40 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7fae9f7fe640 1 -- 172.21.15.37:0/314340212 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fae980cf190 con 0x7faea8100e00 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeae004640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dc98a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeae805640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fae940a2a10 0x7fae940a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeae004640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dc98a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:37548/0 (socket says 172.21.15.37:37548) 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.461+0000 7faeae004640 1 -- 172.21.15.37:0/4210421493 learned_addr learned my addr 172.21.15.37:0/4210421493 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7faead803640 1 --2- 172.21.15.37:0/4210421493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc98fd0 0x55a22dc99450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7faeae805640 1 --2- 172.21.15.37:0/4210421493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 0x55a22dca4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7faeae004640 1 -- 172.21.15.37:0/4210421493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 msgr2=0x55a22dca4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7faeae004640 1 --2- 172.21.15.37:0/4210421493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 0x55a22dca4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7faeae004640 1 -- 172.21.15.37:0/4210421493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc98fd0 msgr2=0x55a22dc99450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7faeae004640 1 --2- 172.21.15.37:0/4210421493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc98fd0 0x55a22dc99450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7faeae004640 1 -- 172.21.15.37:0/4210421493 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a22d912210 con 0x55a22dc99d40 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7faeae805640 1 --2- 172.21.15.37:0/4210421493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 0x55a22dca4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7faeae004640 1 --2- 172.21.15.37:0/4210421493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dc98a90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faea4009900 tx=0x7faea4005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b28e19e654602e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7faeae805640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fae940a2a10 0x7fae940a4ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55a22dc94e50 tx=0x7faea0009a80 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7fae9cbfe640 1 -- 172.21.15.37:0/4210421493 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4043070 con 0x55a22dc99d40 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7fae9cbfe640 1 -- 172.21.15.37:0/4210421493 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faea402c8b0 con 0x55a22dc99d40 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.462+0000 7fae9cbfe640 1 -- 172.21.15.37:0/4210421493 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4004ec0 con 0x55a22dc99d40 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.463+0000 7faeb2e48740 1 -- 172.21.15.37:0/4210421493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 msgr2=0x55a22dc98a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.463+0000 7faeb2e48740 1 --2- 172.21.15.37:0/4210421493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dc98a90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faea4009900 tx=0x7faea4005be0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.463+0000 7faeb2e48740 1 -- 172.21.15.37:0/4210421493 shutdown_connections 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.463+0000 7faeb2e48740 1 --2- 172.21.15.37:0/4210421493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 0x55a22dca4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.463+0000 7faeb2e48740 1 --2- 172.21.15.37:0/4210421493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc98fd0 0x55a22dc99450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.463+0000 7faeb2e48740 1 --2- 172.21.15.37:0/4210421493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dc98a90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.463+0000 7faeb2e48740 1 -- 172.21.15.37:0/4210421493 >> 172.21.15.37:0/4210421493 conn(0x55a22dc605b0 msgr2=0x55a22dc93cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.463+0000 7faeb2e48740 1 -- 172.21.15.37:0/4210421493 shutdown_connections 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.463+0000 7faeb2e48740 1 -- 172.21.15.37:0/4210421493 wait complete. 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeb2e48740 1 -- start start 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeb2e48740 1 Processor -- start 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 0x55a22dc9f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dc9fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeb2e48740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22dca5220 con 0x55a22dca2550 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeb2e48740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22dbb9d90 con 0x55a22dc98fd0 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeb2e48740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22dca53a0 con 0x55a22dc99d40 2024-03-26T06:28:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeb2e48740 10 client.0 _ll_register_callbacks cb 0x55a22d911a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeae805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeae805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54718/0 (socket says 172.21.15.37:54718) 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeae805640 1 -- 172.21.15.37:0/3233939827 learned_addr learned my addr 172.21.15.37:0/3233939827 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faeae004640 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 0x55a22dc9f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.464+0000 7faead803640 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dc9fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7faeae805640 1 -- 172.21.15.37:0/3233939827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 msgr2=0x55a22dc9fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7faeae805640 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dc9fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7faeae805640 1 -- 172.21.15.37:0/3233939827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 msgr2=0x55a22dc9f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7faeae805640 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 0x55a22dc9f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7faeae805640 1 -- 172.21.15.37:0/3233939827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a22d9de620 con 0x55a22dca2550 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7faeae004640 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 0x55a22dc9f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7faeae805640 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca0010 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faea001eb00 tx=0x7faea0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6431858e5050c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7fae91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:28:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7fae91ffb640 1 -- 172.21.15.37:0/3233939827 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea001e570 con 0x55a22dca2550 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7fae91ffb640 1 -- 172.21.15.37:0/3233939827 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7faea0009040 con 0x55a22dca2550 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7faeb2e48740 1 -- 172.21.15.37:0/3233939827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faea402c8b0 con 0x55a22dca2550 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7fae91ffb640 1 -- 172.21.15.37:0/3233939827 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7faea0038050 con 0x55a22dca2550 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7faeb2e48740 10 client.15975 fetch_fsmap learned FSMap version 40 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7faeb2e48740 1 -- 172.21.15.37:0/3233939827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a22dcb1570 con 0x55a22dca2550 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.465+0000 7fae91ffb640 1 -- 172.21.15.37:0/3233939827 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faea402c8b0 con 0x55a22dca2550 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.466+0000 7fae91ffb640 1 -- 172.21.15.37:0/3233939827 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7faea00104c0 con 0x55a22dca2550 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.466+0000 7faeb2e48740 10 client.15975 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.466+0000 7faeb2e48740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.466+0000 7faeb2e48740 20 client.15975 populate_metadata read hostname 'smithi037' 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.466+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a22dcb2a80 0x55a22dcb4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.466+0000 7faeb2e48740 4 client.15975 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:28:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.466+0000 7faeb2e48740 1 -- 172.21.15.37:0/3233939827 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a22dc0ab30 con 0x55a22dcb2a80 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.466+0000 7faeae004640 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a22dcb2a80 0x55a22dcb4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.466+0000 7fae91ffb640 10 client.15975 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.466+0000 7faeae004640 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a22dcb2a80 0x55a22dcb4ee0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5d7aa88ec9f34e9 server_cookie=a7adc50ae4124f5e in_seq=0 out_seq=0 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7fae91ffb640 1 -- 172.21.15.37:0/3233939827 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a22dc0ab30 con 0x55a22dcb2a80 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7fae91ffb640 10 client.15975 handle_command_reply: tid=0 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 1 client.15975 shutdown 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 1 -- 172.21.15.37:0/3233939827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 msgr2=0x55a22dca0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca0010 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faea001eb00 tx=0x7faea0009500 comp rx=0 tx=0).stop 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 1 -- 172.21.15.37:0/3233939827 shutdown_connections 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a22dcb2a80 0x55a22dcb4ee0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca0010 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dc9fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7fae91ffb640 0 client.15975 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3233939827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 0x55a22dc9f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 1 -- 172.21.15.37:0/3233939827 >> 172.21.15.37:0/3233939827 conn(0x55a22dc605b0 msgr2=0x55a22dc95a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 1 -- 172.21.15.37:0/3233939827 shutdown_connections 2024-03-26T06:28:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 1 -- 172.21.15.37:0/3233939827 wait complete. 2024-03-26T06:28:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.467+0000 7faeb2e48740 20 client.15975 trim_cache size 0 max 16384 2024-03-26T06:28:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeb2e48740 1 Processor -- start 2024-03-26T06:28:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeb2e48740 1 -- start start 2024-03-26T06:28:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc98fd0 0x55a22dca1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc99d40 0x55a22dc9de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 0x55a22dc9e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeb2e48740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81aa950 con 0x55a22dc99d40 2024-03-26T06:28:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeb2e48740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81ac390 con 0x55a22dca2550 2024-03-26T06:28:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeb2e48740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81aafb0 con 0x55a22dc98fd0 2024-03-26T06:28:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faead803640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc99d40 0x55a22dc9de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeae805640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 0x55a22dc9e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faead803640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc99d40 0x55a22dc9de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54726/0 (socket says 172.21.15.37:54726) 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faead803640 1 -- 172.21.15.37:0/3592458931 learned_addr learned my addr 172.21.15.37:0/3592458931 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeae004640 1 --2- 172.21.15.37:0/3592458931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc98fd0 0x55a22dca1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faead803640 1 -- 172.21.15.37:0/3592458931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc98fd0 msgr2=0x55a22dca1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faead803640 1 --2- 172.21.15.37:0/3592458931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc98fd0 0x55a22dca1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faead803640 1 -- 172.21.15.37:0/3592458931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 msgr2=0x55a22dc9e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faead803640 1 --2- 172.21.15.37:0/3592458931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 0x55a22dc9e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faead803640 1 -- 172.21.15.37:0/3592458931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a22dc94e80 con 0x55a22dc99d40 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeae004640 1 --2- 172.21.15.37:0/3592458931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc98fd0 0x55a22dca1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.486+0000 7faeae805640 1 --2- 172.21.15.37:0/3592458931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 0x55a22dc9e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.487+0000 7faead803640 1 --2- 172.21.15.37:0/3592458931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc99d40 0x55a22dc9de60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fae9809b860 tx=0x7fae9809b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a87db8530d2861 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.487+0000 7fae93fff640 1 -- 172.21.15.37:0/3592458931 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae980a3fc0 con 0x55a22dc99d40 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.487+0000 7fae93fff640 1 -- 172.21.15.37:0/3592458931 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae98010ac0 con 0x55a22dc99d40 2024-03-26T06:28:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.487+0000 7fae93fff640 1 -- 172.21.15.37:0/3592458931 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9809ad20 con 0x55a22dc99d40 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 -- 172.21.15.37:0/3592458931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc99d40 msgr2=0x55a22dc9de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3592458931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc99d40 0x55a22dc9de60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fae9809b860 tx=0x7fae9809b2f0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 -- 172.21.15.37:0/3592458931 shutdown_connections 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3592458931 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dca2550 0x55a22dc9e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3592458931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dc99d40 0x55a22dc9de60 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3592458931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc98fd0 0x55a22dca1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 -- 172.21.15.37:0/3592458931 >> 172.21.15.37:0/3592458931 conn(0x55a22dc605b0 msgr2=0x55a22dc9a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 -- 172.21.15.37:0/3592458931 shutdown_connections 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 -- 172.21.15.37:0/3592458931 wait complete. 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 -- start start 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 Processor -- start 2024-03-26T06:28:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 0x55a22dca8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dca8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81abc70 con 0x55a22dca2550 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81ab610 con 0x55a22dc98fd0 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea81a8c10 con 0x55a22dc99d40 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.488+0000 7faeb2e48740 10 client.0 _ll_register_callbacks cb 0x7faea00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeae805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeae805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54738/0 (socket says 172.21.15.37:54738) 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeae805640 1 -- 172.21.15.37:0/3214885486 learned_addr learned my addr 172.21.15.37:0/3214885486 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeae004640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 0x55a22dca8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faead803640 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dca8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeae805640 1 -- 172.21.15.37:0/3214885486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 msgr2=0x55a22dca8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeae805640 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dca8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeae805640 1 -- 172.21.15.37:0/3214885486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 msgr2=0x55a22dca8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeae805640 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 0x55a22dca8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeae805640 1 -- 172.21.15.37:0/3214885486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a22dc982a0 con 0x55a22dca2550 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeae004640 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 0x55a22dca8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeae805640 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca8c90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7faea0010960 tx=0x7faea0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6df371f15e8266 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7fae91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7fae91ffb640 1 -- 172.21.15.37:0/3214885486 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea001eb30 con 0x55a22dca2550 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7fae91ffb640 1 -- 172.21.15.37:0/3214885486 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7faea0010ad0 con 0x55a22dca2550 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeb2e48740 1 -- 172.21.15.37:0/3214885486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae98010ac0 con 0x55a22dca2550 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7fae91ffb640 1 -- 172.21.15.37:0/3214885486 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7faea004a050 con 0x55a22dca2550 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeb2e48740 10 client.15981 fetch_fsmap learned FSMap version 40 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.489+0000 7faeb2e48740 1 -- 172.21.15.37:0/3214885486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a22dd0a290 con 0x55a22dca2550 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.490+0000 7fae91ffb640 1 -- 172.21.15.37:0/3214885486 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae98010ac0 con 0x55a22dca2550 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.490+0000 7fae91ffb640 1 -- 172.21.15.37:0/3214885486 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7faea0005670 con 0x55a22dca2550 2024-03-26T06:28:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.490+0000 7faeb2e48740 10 client.15981 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:28:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.490+0000 7faeb2e48740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:28:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.490+0000 7faeb2e48740 20 client.15981 populate_metadata read hostname 'smithi037' 2024-03-26T06:28:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.490+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a22dd0a9e0 0x55a22dd0ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:28:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.490+0000 7faeb2e48740 4 client.15981 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:28:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.490+0000 7faeb2e48740 1 -- 172.21.15.37:0/3214885486 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a22dc04e80 con 0x55a22dd0a9e0 2024-03-26T06:28:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.490+0000 7faeae004640 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a22dd0a9e0 0x55a22dd0ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:28:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.490+0000 7fae91ffb640 10 client.15981 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.490+0000 7faeae004640 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a22dd0a9e0 0x55a22dd0ce00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5eff28040b839 server_cookie=eb503c3194027e51 in_seq=0 out_seq=0 2024-03-26T06:28:56.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 203129 4.56 MB/sec execute 151 sec latency 2986.215 ms 2024-03-26T06:28:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7fae91ffb640 1 -- 172.21.15.37:0/3214885486 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a22dc04e80 con 0x55a22dd0a9e0 2024-03-26T06:28:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7fae91ffb640 10 client.15981 handle_command_reply: tid=0 2024-03-26T06:28:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7faeb2e48740 1 client.15981 shutdown 2024-03-26T06:28:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7faeb2e48740 1 -- 172.21.15.37:0/3214885486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 msgr2=0x55a22dca8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca8c90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7faea0010960 tx=0x7faea0005b50 comp rx=0 tx=0).stop 2024-03-26T06:28:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7faeb2e48740 1 -- 172.21.15.37:0/3214885486 shutdown_connections 2024-03-26T06:28:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a22dd0a9e0 0x55a22dd0ce00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a22dca2550 0x55a22dca8c90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a22dc99d40 0x55a22dca8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7fae91ffb640 0 client.15981 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:28:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7faeb2e48740 1 --2- 172.21.15.37:0/3214885486 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a22dc98fd0 0x55a22dca8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.526+0000 7faeb2e48740 1 -- 172.21.15.37:0/3214885486 >> 172.21.15.37:0/3214885486 conn(0x55a22dc605b0 msgr2=0x55a22dc9dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb2e48740 1 -- 172.21.15.37:0/3214885486 shutdown_connections 2024-03-26T06:28:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb2e48740 1 -- 172.21.15.37:0/3214885486 wait complete. 2024-03-26T06:28:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb2e48740 20 client.15981 trim_cache size 0 max 16384 2024-03-26T06:28:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:28:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:28:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:28:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:28:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 -- 172.21.15.37:0/314340212 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fae940a2a10 msgr2=0x7fae940a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fae940a2a10 0x7fae940a4ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55a22dc94e50 tx=0x7faea0009a80 comp rx=0 tx=0).stop 2024-03-26T06:28:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 -- 172.21.15.37:0/314340212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faea8100e00 msgr2=0x7faea8198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:28:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faea8100e00 0x7faea8198f50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9f0 tx=0x7fae9800eeb0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 -- 172.21.15.37:0/314340212 shutdown_connections 2024-03-26T06:28:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fae940a2a10 0x7fae940a4ed0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faea8105190 0x7faea81a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faea8100e00 0x7faea8198f50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 --2- 172.21.15.37:0/314340212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea8100440 0x7faea8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:28:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 -- 172.21.15.37:0/314340212 >> 172.21.15.37:0/314340212 conn(0x7faea8077040 msgr2=0x7faea80fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:28:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 -- 172.21.15.37:0/314340212 shutdown_connections 2024-03-26T06:28:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:28:56.527+0000 7faeb0268640 1 -- 172.21.15.37:0/314340212 wait complete. 2024-03-26T06:28:57.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 205354 4.61 MB/sec execute 152 sec latency 24.016 ms 2024-03-26T06:28:58.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:57 smithi149 ceph-mon[36681]: pgmap v182: 97 pgs: 97 active+clean; 77 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-03-26T06:28:58.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:57 smithi037 ceph-mon[31974]: pgmap v182: 97 pgs: 97 active+clean; 77 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-03-26T06:28:58.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:57 smithi066 ceph-mon[36504]: pgmap v182: 97 pgs: 97 active+clean; 77 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-03-26T06:28:58.503 INFO:tasks.workunit.client.0.smithi037.stdout: 1 206396 4.62 MB/sec execute 153 sec latency 91.763 ms 2024-03-26T06:28:59.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 206396 4.59 MB/sec execute 154 sec latency 1091.855 ms 2024-03-26T06:29:00.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:28:59 smithi037 ceph-mon[31974]: pgmap v183: 97 pgs: 97 active+clean; 77 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-03-26T06:29:00.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:28:59 smithi066 ceph-mon[36504]: pgmap v183: 97 pgs: 97 active+clean; 77 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-03-26T06:29:00.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:28:59 smithi149 ceph-mon[36681]: pgmap v183: 97 pgs: 97 active+clean; 77 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-03-26T06:29:00.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 206396 4.56 MB/sec execute 155 sec latency 2091.916 ms 2024-03-26T06:29:01.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 206737 4.54 MB/sec execute 156 sec latency 2281.064 ms 2024-03-26T06:29:02.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:01 smithi037 ceph-mon[31974]: pgmap v184: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 209 op/s 2024-03-26T06:29:02.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:01 smithi066 ceph-mon[36504]: pgmap v184: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 209 op/s 2024-03-26T06:29:02.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:01 smithi149 ceph-mon[36681]: pgmap v184: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 209 op/s 2024-03-26T06:29:02.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 208197 4.55 MB/sec execute 157 sec latency 25.518 ms 2024-03-26T06:29:03.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 209752 4.59 MB/sec execute 158 sec latency 32.841 ms 2024-03-26T06:29:04.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:03 smithi037 ceph-mon[31974]: pgmap v185: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-03-26T06:29:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:03 smithi066 ceph-mon[36504]: pgmap v185: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-03-26T06:29:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:03 smithi149 ceph-mon[36681]: pgmap v185: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-03-26T06:29:04.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 211721 4.61 MB/sec execute 159 sec latency 21.020 ms 2024-03-26T06:29:05.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 213363 4.66 MB/sec execute 160 sec latency 13.964 ms 2024-03-26T06:29:06.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:05 smithi037 ceph-mon[31974]: pgmap v186: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-03-26T06:29:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:05 smithi066 ceph-mon[36504]: pgmap v186: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-03-26T06:29:06.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:05 smithi149 ceph-mon[36681]: pgmap v186: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-03-26T06:29:06.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 213609 4.63 MB/sec execute 161 sec latency 650.520 ms 2024-03-26T06:29:07.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 213609 4.60 MB/sec execute 162 sec latency 1151.687 ms 2024-03-26T06:29:08.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:07 smithi037 ceph-mon[31974]: pgmap v187: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-03-26T06:29:08.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:07 smithi066 ceph-mon[36504]: pgmap v187: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-03-26T06:29:08.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:07 smithi149 ceph-mon[36681]: pgmap v187: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-03-26T06:29:08.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 213609 4.57 MB/sec execute 163 sec latency 2151.807 ms 2024-03-26T06:29:09.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 213609 4.54 MB/sec execute 164 sec latency 3151.841 ms 2024-03-26T06:29:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.616+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1637516856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c7c100440 msgr2=0x7f0c7c1008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.616+0000 7f0c80a6c640 1 --2- 172.21.15.37:0/1637516856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c7c100440 0x7f0c7c1008c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009900 tx=0x7f0c6c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:29:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.616+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1637516856 shutdown_connections 2024-03-26T06:29:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.616+0000 7f0c80a6c640 1 --2- 172.21.15.37:0/1637516856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c7c100e00 0x7f0c7c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.616+0000 7f0c80a6c640 1 --2- 172.21.15.37:0/1637516856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c7c100440 0x7f0c7c1008c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.616+0000 7f0c80a6c640 1 --2- 172.21.15.37:0/1637516856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c7c105190 0x7f0c7c105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.616+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1637516856 >> 172.21.15.37:0/1637516856 conn(0x7f0c7c077040 msgr2=0x7f0c7c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.616+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1637516856 shutdown_connections 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.616+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1637516856 wait complete. 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c80a6c640 1 Processor -- start 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c80a6c640 1 -- start start 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c80a6c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c7c100440 0x7f0c7c19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c80a6c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c7c100e00 0x7f0c7c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c80a6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c7c105190 0x7f0c7c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c80a6c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c110650 con 0x7f0c7c105190 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c80a6c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1104d0 con 0x7f0c7c100e00 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c80a6c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1107d0 con 0x7f0c7c100440 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c7c105190 0x7f0c7c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c7c105190 0x7f0c7c197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45262/0 (socket says 172.21.15.37:45262) 2024-03-26T06:29:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c79d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c7c100e00 0x7f0c7c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c7ad9d640 1 -- 172.21.15.37:0/1765387280 learned_addr learned my addr 172.21.15.37:0/1765387280 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c7c100440 0x7f0c7c19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c7ad9d640 1 -- 172.21.15.37:0/1765387280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c7c100440 msgr2=0x7f0c7c19d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c7ad9d640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c7c100440 0x7f0c7c19d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c7ad9d640 1 -- 172.21.15.37:0/1765387280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c7c100e00 msgr2=0x7f0c7c19dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c7ad9d640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c7c100e00 0x7f0c7c19dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.617+0000 7f0c7ad9d640 1 -- 172.21.15.37:0/1765387280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c1981d0 con 0x7f0c7c105190 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.618+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c7c100e00 0x7f0c7c19dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.618+0000 7f0c7ad9d640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c7c105190 0x7f0c7c197a50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400ea60 tx=0x7f0c7400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.618+0000 7f0c6b7fe640 1 -- 172.21.15.37:0/1765387280 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7400ce40 con 0x7f0c7c105190 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.618+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1765387280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7c1984c0 con 0x7f0c7c105190 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.618+0000 7f0c6b7fe640 1 -- 172.21.15.37:0/1765387280 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c74005020 con 0x7f0c7c105190 2024-03-26T06:29:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.618+0000 7f0c6b7fe640 1 -- 172.21.15.37:0/1765387280 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74010690 con 0x7f0c7c105190 2024-03-26T06:29:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.618+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1765387280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c1a4710 con 0x7f0c7c105190 2024-03-26T06:29:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.620+0000 7f0c6b7fe640 1 -- 172.21.15.37:0/1765387280 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0c74005520 con 0x7f0c7c105190 2024-03-26T06:29:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.620+0000 7f0c8364f740 1 Processor -- start 2024-03-26T06:29:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.620+0000 7f0c8364f740 1 -- start start 2024-03-26T06:29:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e1d40 0x55c48b8e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8ec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c8364f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48b8455f0 con 0x55c48b8ea550 2024-03-26T06:29:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c8364f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48b7ee930 con 0x55c48b8e0fd0 2024-03-26T06:29:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c8364f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48b841690 con 0x55c48b8e1d40 2024-03-26T06:29:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c6b7fe640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0c640a2a10 0x7f0c640a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8ec940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45266/0 (socket says 172.21.15.37:45266) 2024-03-26T06:29:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c79d9b640 1 -- 172.21.15.37:0/2106831979 learned_addr learned my addr 172.21.15.37:0/2106831979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/2106831979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e1d40 0x55c48b8e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/2106831979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c7ad9d640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0c640a2a10 0x7f0c640a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c6b7fe640 1 -- 172.21.15.37:0/1765387280 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0c740cf400 con 0x7f0c7c105190 2024-03-26T06:29:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c79d9b640 1 -- 172.21.15.37:0/2106831979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e1d40 msgr2=0x55c48b8e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/2106831979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e1d40 0x55c48b8e0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c79d9b640 1 -- 172.21.15.37:0/2106831979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 msgr2=0x55c48b8e1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/2106831979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c79d9b640 1 -- 172.21.15.37:0/2106831979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c48b55a210 con 0x55c48b8ea550 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/2106831979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c7ad9d640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0c640a2a10 0x7f0c640a4ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c7409b730 tx=0x7f0c7406a840 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.621+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/2106831979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8ec940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c02c170 tx=0x7f0c6c005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987674382fb8e9c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.622+0000 7f0c68ff9640 1 -- 172.21.15.37:0/2106831979 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c004240 con 0x55c48b8ea550 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.622+0000 7f0c68ff9640 1 -- 172.21.15.37:0/2106831979 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c6c0043e0 con 0x55c48b8ea550 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.622+0000 7f0c68ff9640 1 -- 172.21.15.37:0/2106831979 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c005560 con 0x55c48b8ea550 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.622+0000 7f0c8364f740 1 -- 172.21.15.37:0/2106831979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 msgr2=0x55c48b8ec940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.622+0000 7f0c8364f740 1 --2- 172.21.15.37:0/2106831979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8ec940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c02c170 tx=0x7f0c6c005bb0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.622+0000 7f0c8364f740 1 -- 172.21.15.37:0/2106831979 shutdown_connections 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.622+0000 7f0c8364f740 1 --2- 172.21.15.37:0/2106831979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8ec940 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.622+0000 7f0c8364f740 1 --2- 172.21.15.37:0/2106831979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.622+0000 7f0c8364f740 1 --2- 172.21.15.37:0/2106831979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e1d40 0x55c48b8e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.622+0000 7f0c8364f740 1 -- 172.21.15.37:0/2106831979 >> 172.21.15.37:0/2106831979 conn(0x55c48b8a85b0 msgr2=0x55c48b8dbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.622+0000 7f0c8364f740 1 -- 172.21.15.37:0/2106831979 shutdown_connections 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c8364f740 1 -- 172.21.15.37:0/2106831979 wait complete. 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c8364f740 1 -- start start 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c8364f740 1 Processor -- start 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e1d40 0x55c48b8e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8e8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c8364f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48b8ed220 con 0x55c48b8ea550 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c8364f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48b801d90 con 0x55c48b8e1d40 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c8364f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48b8ed3a0 con 0x55c48b8e0fd0 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c8364f740 10 client.0 _ll_register_callbacks cb 0x55c48b559a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8e8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45274/0 (socket says 172.21.15.37:45274) 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c79d9b640 1 -- 172.21.15.37:0/1604258890 learned_addr learned my addr 172.21.15.37:0/1604258890 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e1d40 0x55c48b8e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.623+0000 7f0c79d9b640 1 -- 172.21.15.37:0/1604258890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e0fd0 msgr2=0x55c48b8e7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c79d9b640 1 -- 172.21.15.37:0/1604258890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e1d40 msgr2=0x55c48b8e7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e1d40 0x55c48b8e7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c79d9b640 1 -- 172.21.15.37:0/1604258890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c48b626620 con 0x55c48b8ea550 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e1d40 0x55c48b8e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8e8010 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c036040 tx=0x7f0c6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64ffa6d804723f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c61ffb640 1 -- 172.21.15.37:0/1604258890 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c0055a0 con 0x55c48b8ea550 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c61ffb640 1 -- 172.21.15.37:0/1604258890 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0c6c02c8b0 con 0x55c48b8ea550 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c8364f740 1 -- 172.21.15.37:0/1604258890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c6c0043e0 con 0x55c48b8ea550 2024-03-26T06:29:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c61ffb640 1 -- 172.21.15.37:0/1604258890 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c6c05c050 con 0x55c48b8ea550 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c8364f740 10 client.15993 fetch_fsmap learned FSMap version 40 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c8364f740 1 -- 172.21.15.37:0/1604258890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c48b8f9550 con 0x55c48b8ea550 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c61ffb640 1 -- 172.21.15.37:0/1604258890 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c6c0043e0 con 0x55c48b8ea550 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c61ffb640 1 -- 172.21.15.37:0/1604258890 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0c6c034440 con 0x55c48b8ea550 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c8364f740 10 client.15993 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c8364f740 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.624+0000 7f0c8364f740 20 client.15993 populate_metadata read hostname 'smithi037' 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.625+0000 7f0c8364f740 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c48b8fa8a0 0x55c48b8fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.625+0000 7f0c8364f740 4 client.15993 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.625+0000 7f0c8364f740 1 -- 172.21.15.37:0/1604258890 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c48b852b30 con 0x55c48b8fa8a0 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.625+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c48b8fa8a0 0x55c48b8fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.625+0000 7f0c61ffb640 10 client.15993 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.625+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c48b8fa8a0 0x55c48b8fcc90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76308041fba89a8e server_cookie=7c96d63eb33df341 in_seq=0 out_seq=0 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c61ffb640 1 -- 172.21.15.37:0/1604258890 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c48b852b30 con 0x55c48b8fa8a0 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c61ffb640 10 client.15993 handle_command_reply: tid=0 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 1 client.15993 shutdown 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 1 -- 172.21.15.37:0/1604258890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 msgr2=0x55c48b8e8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8e8010 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c036040 tx=0x7f0c6c0057f0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 1 -- 172.21.15.37:0/1604258890 shutdown_connections 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c48b8fa8a0 0x55c48b8fcc90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8e8010 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e1d40 0x55c48b8e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c61ffb640 0 client.15993 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 1 --2- 172.21.15.37:0/1604258890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 1 -- 172.21.15.37:0/1604258890 >> 172.21.15.37:0/1604258890 conn(0x55c48b8a85b0 msgr2=0x55c48b8dda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 1 -- 172.21.15.37:0/1604258890 shutdown_connections 2024-03-26T06:29:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 1 -- 172.21.15.37:0/1604258890 wait complete. 2024-03-26T06:29:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.626+0000 7f0c8364f740 20 client.15993 trim_cache size 0 max 16384 2024-03-26T06:29:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.640+0000 7f0c8364f740 1 Processor -- start 2024-03-26T06:29:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.640+0000 7f0c8364f740 1 -- start start 2024-03-26T06:29:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.640+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.640+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8e1d40 0x55c48b8dacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.640+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8ea550 0x55c48b8e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.640+0000 7f0c8364f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1acc10 con 0x55c48b8e1d40 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.640+0000 7f0c8364f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1ae650 con 0x55c48b8e0fd0 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.640+0000 7f0c8364f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1ad270 con 0x55c48b8ea550 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.640+0000 7f0c7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8e1d40 0x55c48b8dacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.640+0000 7f0c7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8e1d40 0x55c48b8dacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45282/0 (socket says 172.21.15.37:45282) 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.640+0000 7f0c7a59c640 1 -- 172.21.15.37:0/1422167760 learned_addr learned my addr 172.21.15.37:0/1422167760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/1422167760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8ea550 0x55c48b8e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/1422167760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c7a59c640 1 -- 172.21.15.37:0/1422167760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8ea550 msgr2=0x55c48b8e47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/1422167760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8ea550 0x55c48b8e47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c7a59c640 1 -- 172.21.15.37:0/1422167760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 msgr2=0x55c48b8e9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/1422167760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c7a59c640 1 -- 172.21.15.37:0/1422167760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c48b8e02a0 con 0x55c48b8e1d40 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/1422167760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/1422167760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8e1d40 0x55c48b8dacc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55c48b8e6df0 tx=0x7f0c70005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9a7ad4918655d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c63fff640 1 -- 172.21.15.37:0/1422167760 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70004220 con 0x55c48b8e1d40 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c63fff640 1 -- 172.21.15.37:0/1422167760 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c70024070 con 0x55c48b8e1d40 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.641+0000 7f0c63fff640 1 -- 172.21.15.37:0/1422167760 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70003a20 con 0x55c48b8e1d40 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.642+0000 7f0c8364f740 1 -- 172.21.15.37:0/1422167760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8e1d40 msgr2=0x55c48b8dacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.642+0000 7f0c8364f740 1 --2- 172.21.15.37:0/1422167760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8e1d40 0x55c48b8dacc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55c48b8e6df0 tx=0x7f0c70005f90 comp rx=0 tx=0).stop 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.642+0000 7f0c8364f740 1 -- 172.21.15.37:0/1422167760 shutdown_connections 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.642+0000 7f0c8364f740 1 --2- 172.21.15.37:0/1422167760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8ea550 0x55c48b8e47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.642+0000 7f0c8364f740 1 --2- 172.21.15.37:0/1422167760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8e1d40 0x55c48b8dacc0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.642+0000 7f0c8364f740 1 --2- 172.21.15.37:0/1422167760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.642+0000 7f0c8364f740 1 -- 172.21.15.37:0/1422167760 >> 172.21.15.37:0/1422167760 conn(0x55c48b8a85b0 msgr2=0x55c48b8da5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.642+0000 7f0c8364f740 1 -- 172.21.15.37:0/1422167760 shutdown_connections 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.642+0000 7f0c8364f740 1 -- 172.21.15.37:0/1422167760 wait complete. 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.642+0000 7f0c8364f740 1 -- start start 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.642+0000 7f0c8364f740 1 Processor -- start 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8f0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e1d40 0x55c48b8f0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c8364f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c8364f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1adf30 con 0x55c48b8ea550 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c8364f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1ad8d0 con 0x55c48b8e0fd0 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c8364f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1aaeb0 con 0x55c48b8e1d40 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c8364f740 10 client.0 _ll_register_callbacks cb 0x7f0c740093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8f0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45294/0 (socket says 172.21.15.37:45294) 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c79d9b640 1 -- 172.21.15.37:0/522638948 learned_addr learned my addr 172.21.15.37:0/522638948 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e1d40 0x55c48b8f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c79d9b640 1 -- 172.21.15.37:0/522638948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e1d40 msgr2=0x55c48b8f0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e1d40 0x55c48b8f0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c79d9b640 1 -- 172.21.15.37:0/522638948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 msgr2=0x55c48b8f0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8f0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c79d9b640 1 -- 172.21.15.37:0/522638948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c48b950cd0 con 0x55c48b8ea550 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c79d9b640 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8f0e00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c034810 tx=0x7f0c6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717c21b0d3210cd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.643+0000 7f0c61ffb640 1 -- 172.21.15.37:0/522638948 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c004d60 con 0x55c48b8ea550 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c61ffb640 1 -- 172.21.15.37:0/522638948 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0c6c02c5d0 con 0x55c48b8ea550 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c8364f740 1 -- 172.21.15.37:0/522638948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c70024070 con 0x55c48b8ea550 2024-03-26T06:29:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c61ffb640 1 -- 172.21.15.37:0/522638948 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c6c034440 con 0x55c48b8ea550 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c8364f740 10 client.15999 fetch_fsmap learned FSMap version 40 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c8364f740 1 -- 172.21.15.37:0/522638948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c48b951fe0 con 0x55c48b8ea550 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c61ffb640 1 -- 172.21.15.37:0/522638948 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c70024070 con 0x55c48b8ea550 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c61ffb640 1 -- 172.21.15.37:0/522638948 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0c6c02c940 con 0x55c48b8ea550 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c8364f740 10 client.15999 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c8364f740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c8364f740 20 client.15999 populate_metadata read hostname 'smithi037' 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c8364f740 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c48b952600 0x55c48b954a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c8364f740 4 client.15999 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c8364f740 1 -- 172.21.15.37:0/522638948 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c48b84ce80 con 0x55c48b952600 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.644+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c48b952600 0x55c48b954a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.645+0000 7f0c7a59c640 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c48b952600 0x55c48b954a20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23bc5d92b855abcd server_cookie=42aba5f01b89125c in_seq=0 out_seq=0 2024-03-26T06:29:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.645+0000 7f0c61ffb640 10 client.15999 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.674+0000 7f0c61ffb640 1 -- 172.21.15.37:0/522638948 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c48b84ce80 con 0x55c48b952600 2024-03-26T06:29:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.674+0000 7f0c61ffb640 10 client.15999 handle_command_reply: tid=0 2024-03-26T06:29:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 1 client.15999 shutdown 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 1 -- 172.21.15.37:0/522638948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 msgr2=0x55c48b8f0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8f0e00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c034810 tx=0x7f0c6c0057f0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 1 -- 172.21.15.37:0/522638948 shutdown_connections 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c48b952600 0x55c48b954a20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c61ffb640 0 client.15999 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c48b8ea550 0x55c48b8f0e00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c48b8e1d40 0x55c48b8f0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 1 --2- 172.21.15.37:0/522638948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c48b8e0fd0 0x55c48b8f0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 1 -- 172.21.15.37:0/522638948 >> 172.21.15.37:0/522638948 conn(0x55c48b8a85b0 msgr2=0x55c48b8dae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 1 -- 172.21.15.37:0/522638948 shutdown_connections 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 1 -- 172.21.15.37:0/522638948 wait complete. 2024-03-26T06:29:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.675+0000 7f0c8364f740 20 client.15999 trim_cache size 0 max 16384 2024-03-26T06:29:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:29:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:29:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:29:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:29:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1765387280 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0c640a2a10 msgr2=0x7f0c640a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0c640a2a10 0x7f0c640a4ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c7409b730 tx=0x7f0c7406a840 comp rx=0 tx=0).stop 2024-03-26T06:29:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1765387280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c7c105190 msgr2=0x7f0c7c197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c7c105190 0x7f0c7c197a50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400ea60 tx=0x7f0c7400ef20 comp rx=0 tx=0).stop 2024-03-26T06:29:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1765387280 shutdown_connections 2024-03-26T06:29:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0c640a2a10 0x7f0c640a4ed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c7c105190 0x7f0c7c197a50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c7c100e00 0x7f0c7c19dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 --2- 172.21.15.37:0/1765387280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c7c100440 0x7f0c7c19d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1765387280 >> 172.21.15.37:0/1765387280 conn(0x7f0c7c077040 msgr2=0x7f0c7c068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1765387280 shutdown_connections 2024-03-26T06:29:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:09.676+0000 7f0c80a6c640 1 -- 172.21.15.37:0/1765387280 wait complete. 2024-03-26T06:29:10.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:09 smithi037 ceph-mon[31974]: pgmap v188: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-03-26T06:29:10.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:09 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:29:10.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:09 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:29:10.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:09 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:29:10.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:09 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:29:10.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:09 smithi066 ceph-mon[36504]: pgmap v188: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-03-26T06:29:10.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:09 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:29:10.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:09 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:29:10.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:09 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:29:10.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:09 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:29:10.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:09 smithi149 ceph-mon[36681]: pgmap v188: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-03-26T06:29:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:09 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:29:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:09 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:29:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:09 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:29:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:09 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:29:10.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 213609 4.52 MB/sec execute 165 sec latency 4151.920 ms 2024-03-26T06:29:11.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 213983 4.50 MB/sec execute 166 sec latency 4952.641 ms 2024-03-26T06:29:12.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:11 smithi037 ceph-mon[31974]: pgmap v189: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 252 op/s 2024-03-26T06:29:12.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:11 smithi066 ceph-mon[36504]: pgmap v189: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 252 op/s 2024-03-26T06:29:12.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:11 smithi149 ceph-mon[36681]: pgmap v189: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 252 op/s 2024-03-26T06:29:12.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 216387 4.57 MB/sec execute 167 sec latency 17.238 ms 2024-03-26T06:29:13.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 217673 4.56 MB/sec execute 168 sec latency 21.907 ms 2024-03-26T06:29:14.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:13 smithi037 ceph-mon[31974]: pgmap v190: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-03-26T06:29:14.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:13 smithi066 ceph-mon[36504]: pgmap v190: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-03-26T06:29:14.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:13 smithi149 ceph-mon[36681]: pgmap v190: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-03-26T06:29:14.504 INFO:tasks.workunit.client.0.smithi037.stdout: 1 220044 4.63 MB/sec execute 169 sec latency 15.709 ms 2024-03-26T06:29:15.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 220756 4.61 MB/sec execute 170 sec latency 293.782 ms 2024-03-26T06:29:16.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:15 smithi037 ceph-mon[31974]: pgmap v191: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-03-26T06:29:16.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:15 smithi066 ceph-mon[36504]: pgmap v191: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-03-26T06:29:16.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:15 smithi149 ceph-mon[36681]: pgmap v191: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-03-26T06:29:16.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 221135 4.59 MB/sec execute 171 sec latency 1094.463 ms 2024-03-26T06:29:17.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223573 4.66 MB/sec execute 172 sec latency 13.757 ms 2024-03-26T06:29:18.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:17 smithi037 ceph-mon[31974]: pgmap v192: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:29:18.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:17 smithi066 ceph-mon[36504]: pgmap v192: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:29:18.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:17 smithi149 ceph-mon[36681]: pgmap v192: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:29:18.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 224180 4.64 MB/sec execute 173 sec latency 301.805 ms 2024-03-26T06:29:19.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 224751 4.62 MB/sec execute 174 sec latency 969.736 ms 2024-03-26T06:29:20.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:19 smithi037 ceph-mon[31974]: pgmap v193: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:29:20.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:19 smithi066 ceph-mon[36504]: pgmap v193: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:29:20.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:19 smithi149 ceph-mon[36681]: pgmap v193: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:29:20.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 227195 4.69 MB/sec execute 175 sec latency 18.155 ms 2024-03-26T06:29:21.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 227846 4.67 MB/sec execute 176 sec latency 189.890 ms 2024-03-26T06:29:22.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:21 smithi037 ceph-mon[31974]: pgmap v194: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 233 op/s 2024-03-26T06:29:22.350 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:21 smithi066 ceph-mon[36504]: pgmap v194: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 233 op/s 2024-03-26T06:29:22.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:21 smithi149 ceph-mon[36681]: pgmap v194: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 233 op/s 2024-03-26T06:29:22.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 227846 4.64 MB/sec execute 177 sec latency 1165.726 ms 2024-03-26T06:29:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.773+0000 7f7e79574640 1 -- 172.21.15.37:0/3683882973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e74105190 msgr2=0x7f7e74105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.773+0000 7f7e79574640 1 --2- 172.21.15.37:0/3683882973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e74105190 0x7f7e74105570 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009900 tx=0x7f7e6402bc30 comp rx=0 tx=0).stop 2024-03-26T06:29:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.773+0000 7f7e79574640 1 -- 172.21.15.37:0/3683882973 shutdown_connections 2024-03-26T06:29:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.773+0000 7f7e79574640 1 --2- 172.21.15.37:0/3683882973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e74100e00 0x7f7e7410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.773+0000 7f7e79574640 1 --2- 172.21.15.37:0/3683882973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e74100440 0x7f7e741008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.773+0000 7f7e79574640 1 --2- 172.21.15.37:0/3683882973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e74105190 0x7f7e74105570 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.773+0000 7f7e79574640 1 -- 172.21.15.37:0/3683882973 >> 172.21.15.37:0/3683882973 conn(0x7f7e74077040 msgr2=0x7f7e74077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.773+0000 7f7e79574640 1 -- 172.21.15.37:0/3683882973 shutdown_connections 2024-03-26T06:29:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.773+0000 7f7e79574640 1 -- 172.21.15.37:0/3683882973 wait complete. 2024-03-26T06:29:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.773+0000 7f7e79574640 1 Processor -- start 2024-03-26T06:29:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e79574640 1 -- start start 2024-03-26T06:29:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e79574640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e74100440 0x7f7e7419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e79574640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e74100e00 0x7f7e7419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e79574640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e74105190 0x7f7e74197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e79574640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e74110630 con 0x7f7e74100e00 2024-03-26T06:29:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e79574640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741104b0 con 0x7f7e74105190 2024-03-26T06:29:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e79574640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741107b0 con 0x7f7e74100440 2024-03-26T06:29:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e727fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e74100e00 0x7f7e7419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e727fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e74100e00 0x7f7e7419dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35972/0 (socket says 172.21.15.37:35972) 2024-03-26T06:29:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e727fc640 1 -- 172.21.15.37:0/2547793217 learned_addr learned my addr 172.21.15.37:0/2547793217 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e737fe640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e74105190 0x7f7e74197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e74100440 0x7f7e7419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e727fc640 1 -- 172.21.15.37:0/2547793217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e74100440 msgr2=0x7f7e7419d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e727fc640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e74100440 0x7f7e7419d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e727fc640 1 -- 172.21.15.37:0/2547793217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e74105190 msgr2=0x7f7e74197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e727fc640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e74105190 0x7f7e74197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e727fc640 1 -- 172.21.15.37:0/2547793217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e74198180 con 0x7f7e74100e00 2024-03-26T06:29:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.774+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e74100440 0x7f7e7419d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:29:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.775+0000 7f7e727fc640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e74100e00 0x7f7e7419dd90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800e9f0 tx=0x7f7e6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.775+0000 7f7e5bfff640 1 -- 172.21.15.37:0/2547793217 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6800cd80 con 0x7f7e74100e00 2024-03-26T06:29:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.775+0000 7f7e5bfff640 1 -- 172.21.15.37:0/2547793217 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e68010910 con 0x7f7e74100e00 2024-03-26T06:29:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.775+0000 7f7e79574640 1 -- 172.21.15.37:0/2547793217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e74198470 con 0x7f7e74100e00 2024-03-26T06:29:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.775+0000 7f7e5bfff640 1 -- 172.21.15.37:0/2547793217 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68018990 con 0x7f7e74100e00 2024-03-26T06:29:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.775+0000 7f7e79574640 1 -- 172.21.15.37:0/2547793217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e741a4710 con 0x7f7e74100e00 2024-03-26T06:29:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.777+0000 7f7e5bfff640 1 -- 172.21.15.37:0/2547793217 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7e68010ac0 con 0x7f7e74100e00 2024-03-26T06:29:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.777+0000 7f7e7c22b740 1 Processor -- start 2024-03-26T06:29:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e7c22b740 1 -- start start 2024-03-26T06:29:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac21d40 0x55eebac20a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac21450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac2a550 0x55eebac2c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebab855f0 con 0x55eebac20fd0 2024-03-26T06:29:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebab2e930 con 0x55eebac2a550 2024-03-26T06:29:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebab81690 con 0x55eebac21d40 2024-03-26T06:29:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e5bfff640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7e5c0a2a10 0x7f7e5c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e72ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac21450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e72ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac21450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35984/0 (socket says 172.21.15.37:35984) 2024-03-26T06:29:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e72ffd640 1 -- 172.21.15.37:0/1220632660 learned_addr learned my addr 172.21.15.37:0/1220632660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e737fe640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7e5c0a2a10 0x7f7e5c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e5bfff640 1 -- 172.21.15.37:0/2547793217 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7e680cf280 con 0x7f7e74100e00 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e737fe640 1 --2- 172.21.15.37:0/1220632660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac21d40 0x55eebac20a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e727fc640 1 --2- 172.21.15.37:0/1220632660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac2a550 0x55eebac2c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e72ffd640 1 -- 172.21.15.37:0/1220632660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac21d40 msgr2=0x55eebac20a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/1220632660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac21d40 0x55eebac20a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e72ffd640 1 -- 172.21.15.37:0/1220632660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac2a550 msgr2=0x55eebac2c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/1220632660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac2a550 0x55eebac2c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.778+0000 7f7e72ffd640 1 -- 172.21.15.37:0/1220632660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeba89a210 con 0x55eebac20fd0 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/1220632660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac21450 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7e6402efd0 tx=0x7f7e64005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1363ab899dfdb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e737fe640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7e5c0a2a10 0x7f7e5c0a4ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c005fd0 tx=0x7f7e6c009ac0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e597fa640 1 -- 172.21.15.37:0/1220632660 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64042070 con 0x55eebac20fd0 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e597fa640 1 -- 172.21.15.37:0/1220632660 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e64004240 con 0x55eebac20fd0 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e597fa640 1 -- 172.21.15.37:0/1220632660 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e640046c0 con 0x55eebac20fd0 2024-03-26T06:29:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e7c22b740 1 -- 172.21.15.37:0/1220632660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 msgr2=0x55eebac21450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/1220632660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac21450 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7e6402efd0 tx=0x7f7e64005e60 comp rx=0 tx=0).stop 2024-03-26T06:29:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e7c22b740 1 -- 172.21.15.37:0/1220632660 shutdown_connections 2024-03-26T06:29:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/1220632660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac2a550 0x55eebac2c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/1220632660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac21450 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/1220632660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac21d40 0x55eebac20a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.779+0000 7f7e7c22b740 1 -- 172.21.15.37:0/1220632660 >> 172.21.15.37:0/1220632660 conn(0x55eebabe85b0 msgr2=0x55eebac1bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e7c22b740 1 -- 172.21.15.37:0/1220632660 shutdown_connections 2024-03-26T06:29:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e7c22b740 1 -- 172.21.15.37:0/1220632660 wait complete. 2024-03-26T06:29:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e7c22b740 1 -- start start 2024-03-26T06:29:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e7c22b740 1 Processor -- start 2024-03-26T06:29:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac20fd0 0x55eebac27620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac27b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac2a550 0x55eebac280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebac2d220 con 0x55eebac21d40 2024-03-26T06:29:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebab41d90 con 0x55eebac2a550 2024-03-26T06:29:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebac2d3a0 con 0x55eebac20fd0 2024-03-26T06:29:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e7c22b740 10 client.0 _ll_register_callbacks cb 0x55eeba899a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:29:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e727fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac27b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e727fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac27b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35992/0 (socket says 172.21.15.37:35992) 2024-03-26T06:29:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e727fc640 1 -- 172.21.15.37:0/3941910811 learned_addr learned my addr 172.21.15.37:0/3941910811 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac20fd0 0x55eebac27620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.780+0000 7f7e737fe640 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac2a550 0x55eebac280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e727fc640 1 -- 172.21.15.37:0/3941910811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac20fd0 msgr2=0x55eebac27620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e727fc640 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac20fd0 0x55eebac27620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e727fc640 1 -- 172.21.15.37:0/3941910811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac2a550 msgr2=0x55eebac280a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e727fc640 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac2a550 0x55eebac280a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e727fc640 1 -- 172.21.15.37:0/3941910811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeba966620 con 0x55eebac21d40 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e727fc640 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac27b60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c6a0 tx=0x7f7e6809b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb0af3eaac0a5bd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e56ff5640 1 -- 172.21.15.37:0/3941910811 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e680862b0 con 0x55eebac21d40 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e56ff5640 1 -- 172.21.15.37:0/3941910811 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7e680ccc20 con 0x55eebac21d40 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e7c22b740 1 -- 172.21.15.37:0/3941910811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e64004240 con 0x55eebac21d40 2024-03-26T06:29:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e56ff5640 1 -- 172.21.15.37:0/3941910811 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e680cce20 con 0x55eebac21d40 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e7c22b740 10 client.16011 fetch_fsmap learned FSMap version 40 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e7c22b740 1 -- 172.21.15.37:0/3941910811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55eebac39590 con 0x55eebac21d40 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e56ff5640 1 -- 172.21.15.37:0/3941910811 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e64004240 con 0x55eebac21d40 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.781+0000 7f7e56ff5640 1 -- 172.21.15.37:0/3941910811 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7e68018b30 con 0x55eebac21d40 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.782+0000 7f7e7c22b740 10 client.16011 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.782+0000 7f7e7c22b740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.782+0000 7f7e7c22b740 20 client.16011 populate_metadata read hostname 'smithi037' 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.782+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55eebac3aaa0 0x55eebac3cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.782+0000 7f7e7c22b740 4 client.16011 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.782+0000 7f7e7c22b740 1 -- 172.21.15.37:0/3941910811 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eebab92b30 con 0x55eebac3aaa0 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.782+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55eebac3aaa0 0x55eebac3cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.782+0000 7f7e56ff5640 10 client.16011 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.782+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55eebac3aaa0 0x55eebac3cf00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f80335195b9b8 server_cookie=295caa055fcaa3d1 in_seq=0 out_seq=0 2024-03-26T06:29:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e56ff5640 1 -- 172.21.15.37:0/3941910811 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eebab92b30 con 0x55eebac3aaa0 2024-03-26T06:29:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e56ff5640 10 client.16011 handle_command_reply: tid=0 2024-03-26T06:29:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 1 client.16011 shutdown 2024-03-26T06:29:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 1 -- 172.21.15.37:0/3941910811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 msgr2=0x55eebac27b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac27b60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c6a0 tx=0x7f7e6809b320 comp rx=0 tx=0).stop 2024-03-26T06:29:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 1 -- 172.21.15.37:0/3941910811 shutdown_connections 2024-03-26T06:29:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55eebac3aaa0 0x55eebac3cf00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac2a550 0x55eebac280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac27b60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/3941910811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac20fd0 0x55eebac27620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 1 -- 172.21.15.37:0/3941910811 >> 172.21.15.37:0/3941910811 conn(0x55eebabe85b0 msgr2=0x55eebac1da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e56ff5640 0 client.16011 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 1 -- 172.21.15.37:0/3941910811 shutdown_connections 2024-03-26T06:29:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 1 -- 172.21.15.37:0/3941910811 wait complete. 2024-03-26T06:29:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.783+0000 7f7e7c22b740 20 client.16011 trim_cache size 0 max 16384 2024-03-26T06:29:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.796+0000 7f7e7c22b740 1 Processor -- start 2024-03-26T06:29:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.796+0000 7f7e7c22b740 1 -- start start 2024-03-26T06:29:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebabe85b0 0x55eebac29d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac20fd0 0x55eebac31610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac31b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741acb30 con 0x55eebac21d40 2024-03-26T06:29:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741ae570 con 0x55eebabe85b0 2024-03-26T06:29:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741ad190 con 0x55eebac20fd0 2024-03-26T06:29:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac31b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac31b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35994/0 (socket says 172.21.15.37:35994) 2024-03-26T06:29:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e737fe640 1 -- 172.21.15.37:0/2882944972 learned_addr learned my addr 172.21.15.37:0/2882944972 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e727fc640 1 --2- 172.21.15.37:0/2882944972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac20fd0 0x55eebac31610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/2882944972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebabe85b0 0x55eebac29d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e737fe640 1 -- 172.21.15.37:0/2882944972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac20fd0 msgr2=0x55eebac31610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e737fe640 1 --2- 172.21.15.37:0/2882944972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac20fd0 0x55eebac31610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e737fe640 1 -- 172.21.15.37:0/2882944972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebabe85b0 msgr2=0x55eebac29d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e737fe640 1 --2- 172.21.15.37:0/2882944972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebabe85b0 0x55eebac29d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e737fe640 1 -- 172.21.15.37:0/2882944972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eebac23240 con 0x55eebac21d40 2024-03-26T06:29:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/2882944972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebabe85b0 0x55eebac29d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:29:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.797+0000 7f7e737fe640 1 --2- 172.21.15.37:0/2882944972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac31b50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c010bd0 tx=0x7f7e6c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aab325fd04619f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e58ff9640 1 -- 172.21.15.37:0/2882944972 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c020e20 con 0x55eebac21d40 2024-03-26T06:29:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e58ff9640 1 -- 172.21.15.37:0/2882944972 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e6c01e920 con 0x55eebac21d40 2024-03-26T06:29:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e58ff9640 1 -- 172.21.15.37:0/2882944972 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c02a880 con 0x55eebac21d40 2024-03-26T06:29:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e7c22b740 1 -- 172.21.15.37:0/2882944972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 msgr2=0x55eebac31b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/2882944972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac31b50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c010bd0 tx=0x7f7e6c0095f0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e7c22b740 1 -- 172.21.15.37:0/2882944972 shutdown_connections 2024-03-26T06:29:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/2882944972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac21d40 0x55eebac31b50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/2882944972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebac20fd0 0x55eebac31610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/2882944972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebabe85b0 0x55eebac29d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e7c22b740 1 -- 172.21.15.37:0/2882944972 >> 172.21.15.37:0/2882944972 conn(0x55eebac2cb20 msgr2=0x55eebac418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e7c22b740 1 -- 172.21.15.37:0/2882944972 shutdown_connections 2024-03-26T06:29:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e7c22b740 1 -- 172.21.15.37:0/2882944972 wait complete. 2024-03-26T06:29:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.798+0000 7f7e7c22b740 1 -- start start 2024-03-26T06:29:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e7c22b740 1 Processor -- start 2024-03-26T06:29:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebabe85b0 0x55eebac31b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e7c22b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac21d40 0x55eebac247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741ade50 con 0x55eebac20fd0 2024-03-26T06:29:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741ad7f0 con 0x55eebac21d40 2024-03-26T06:29:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e7c22b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741aadd0 con 0x55eebabe85b0 2024-03-26T06:29:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e7c22b740 10 client.0 _ll_register_callbacks cb 0x7f7e68005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:29:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e727fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e727fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac240b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36002/0 (socket says 172.21.15.37:36002) 2024-03-26T06:29:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e727fc640 1 -- 172.21.15.37:0/272628426 learned_addr learned my addr 172.21.15.37:0/272628426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e737fe640 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac21d40 0x55eebac247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e727fc640 1 -- 172.21.15.37:0/272628426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebabe85b0 msgr2=0x55eebac31b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e727fc640 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebabe85b0 0x55eebac31b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e727fc640 1 -- 172.21.15.37:0/272628426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac21d40 msgr2=0x55eebac247d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e727fc640 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac21d40 0x55eebac247d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e727fc640 1 -- 172.21.15.37:0/272628426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eebac91020 con 0x55eebac20fd0 2024-03-26T06:29:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e727fc640 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac240b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7e6809b840 tx=0x7f7e68018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b6baf1a07dff51 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:29:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e56ff5640 1 -- 172.21.15.37:0/272628426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6809b3f0 con 0x55eebac20fd0 2024-03-26T06:29:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e56ff5640 1 -- 172.21.15.37:0/272628426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7e6809cbe0 con 0x55eebac20fd0 2024-03-26T06:29:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.799+0000 7f7e7c22b740 1 -- 172.21.15.37:0/272628426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e6c01e920 con 0x55eebac20fd0 2024-03-26T06:29:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e56ff5640 1 -- 172.21.15.37:0/272628426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e68018d40 con 0x55eebac20fd0 2024-03-26T06:29:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e7c22b740 10 client.16017 fetch_fsmap learned FSMap version 40 2024-03-26T06:29:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e7c22b740 1 -- 172.21.15.37:0/272628426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55eebac92150 con 0x55eebac20fd0 2024-03-26T06:29:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e56ff5640 1 -- 172.21.15.37:0/272628426 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e6c01e920 con 0x55eebac20fd0 2024-03-26T06:29:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e56ff5640 1 -- 172.21.15.37:0/272628426 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7e6809ad00 con 0x55eebac20fd0 2024-03-26T06:29:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e7c22b740 10 client.16017 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:29:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e7c22b740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:29:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e7c22b740 20 client.16017 populate_metadata read hostname 'smithi037' 2024-03-26T06:29:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55eebac2a550 0x55eebac94840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e7c22b740 4 client.16017 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:29:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e7c22b740 1 -- 172.21.15.37:0/272628426 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eebab8ce80 con 0x55eebac2a550 2024-03-26T06:29:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.800+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55eebac2a550 0x55eebac94840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.801+0000 7f7e56ff5640 10 client.16017 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.801+0000 7f7e72ffd640 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55eebac2a550 0x55eebac94840 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb82c52f9eb86400 server_cookie=99e0c7e181449511 in_seq=0 out_seq=0 2024-03-26T06:29:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e56ff5640 1 -- 172.21.15.37:0/272628426 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eebab8ce80 con 0x55eebac2a550 2024-03-26T06:29:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e56ff5640 10 client.16017 handle_command_reply: tid=0 2024-03-26T06:29:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 1 client.16017 shutdown 2024-03-26T06:29:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 1 -- 172.21.15.37:0/272628426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 msgr2=0x55eebac240b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac240b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7e6809b840 tx=0x7f7e68018420 comp rx=0 tx=0).stop 2024-03-26T06:29:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 1 -- 172.21.15.37:0/272628426 shutdown_connections 2024-03-26T06:29:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55eebac2a550 0x55eebac94840 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eebac21d40 0x55eebac247d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eebac20fd0 0x55eebac240b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e56ff5640 0 client.16017 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 1 --2- 172.21.15.37:0/272628426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eebabe85b0 0x55eebac31b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 1 -- 172.21.15.37:0/272628426 >> 172.21.15.37:0/272628426 conn(0x55eebac2cb20 msgr2=0x55eebac1a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 1 -- 172.21.15.37:0/272628426 shutdown_connections 2024-03-26T06:29:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 1 -- 172.21.15.37:0/272628426 wait complete. 2024-03-26T06:29:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.834+0000 7f7e7c22b740 20 client.16017 trim_cache size 0 max 16384 2024-03-26T06:29:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:29:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:29:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:29:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:29:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 -- 172.21.15.37:0/2547793217 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7e5c0a2a10 msgr2=0x7f7e5c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7e5c0a2a10 0x7f7e5c0a4ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c005fd0 tx=0x7f7e6c009ac0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 -- 172.21.15.37:0/2547793217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e74100e00 msgr2=0x7f7e7419dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e74100e00 0x7f7e7419dd90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800e9f0 tx=0x7f7e6800eeb0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 -- 172.21.15.37:0/2547793217 shutdown_connections 2024-03-26T06:29:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7e5c0a2a10 0x7f7e5c0a4ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e74105190 0x7f7e74197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e74100e00 0x7f7e7419dd90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 --2- 172.21.15.37:0/2547793217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e74100440 0x7f7e7419d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 -- 172.21.15.37:0/2547793217 >> 172.21.15.37:0/2547793217 conn(0x7f7e74077040 msgr2=0x7f7e74068c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 -- 172.21.15.37:0/2547793217 shutdown_connections 2024-03-26T06:29:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:22.835+0000 7f7e79574640 1 -- 172.21.15.37:0/2547793217 wait complete. 2024-03-26T06:29:23.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 227846 4.62 MB/sec execute 178 sec latency 2165.828 ms 2024-03-26T06:29:24.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:23 smithi037 ceph-mon[31974]: pgmap v195: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-03-26T06:29:24.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:23 smithi066 ceph-mon[36504]: pgmap v195: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-03-26T06:29:24.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:23 smithi149 ceph-mon[36681]: pgmap v195: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-03-26T06:29:24.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 227846 4.59 MB/sec execute 179 sec latency 3165.958 ms 2024-03-26T06:29:25.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 227846 4.57 MB/sec execute 180 sec latency 4166.061 ms 2024-03-26T06:29:26.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:25 smithi037 ceph-mon[31974]: pgmap v196: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-03-26T06:29:26.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:25 smithi066 ceph-mon[36504]: pgmap v196: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-03-26T06:29:26.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:25 smithi149 ceph-mon[36681]: pgmap v196: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-03-26T06:29:26.505 INFO:tasks.workunit.client.0.smithi037.stdout: 1 228198 4.55 MB/sec execute 181 sec latency 4966.805 ms 2024-03-26T06:29:27.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 230608 4.61 MB/sec execute 182 sec latency 14.384 ms 2024-03-26T06:29:28.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:27 smithi037 ceph-mon[31974]: pgmap v197: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-03-26T06:29:28.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:27 smithi066 ceph-mon[36504]: pgmap v197: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-03-26T06:29:28.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:27 smithi149 ceph-mon[36681]: pgmap v197: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-03-26T06:29:28.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 231327 4.59 MB/sec execute 183 sec latency 343.325 ms 2024-03-26T06:29:29.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 231327 4.57 MB/sec execute 184 sec latency 1343.416 ms 2024-03-26T06:29:30.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:29 smithi149 ceph-mon[36681]: pgmap v198: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-03-26T06:29:30.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:29 smithi066 ceph-mon[36504]: pgmap v198: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-03-26T06:29:30.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:29 smithi037 ceph-mon[31974]: pgmap v198: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-03-26T06:29:30.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 231327 4.55 MB/sec execute 185 sec latency 2343.520 ms 2024-03-26T06:29:31.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 231395 4.52 MB/sec execute 186 sec latency 3142.378 ms 2024-03-26T06:29:32.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:31 smithi066 ceph-mon[36504]: pgmap v199: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-03-26T06:29:32.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:31 smithi149 ceph-mon[36681]: pgmap v199: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-03-26T06:29:32.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:31 smithi037 ceph-mon[31974]: pgmap v199: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-03-26T06:29:32.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 231395 4.50 MB/sec execute 187 sec latency 1153.667 ms 2024-03-26T06:29:33.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 231395 4.47 MB/sec execute 188 sec latency 2153.751 ms 2024-03-26T06:29:34.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:33 smithi037 ceph-mon[31974]: pgmap v200: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-03-26T06:29:34.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:33 smithi066 ceph-mon[36504]: pgmap v200: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-03-26T06:29:34.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:33 smithi149 ceph-mon[36681]: pgmap v200: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-03-26T06:29:34.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 231395 4.45 MB/sec execute 189 sec latency 3153.874 ms 2024-03-26T06:29:35.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 231395 4.43 MB/sec execute 190 sec latency 4153.932 ms 2024-03-26T06:29:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.935+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/717789921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf381073e0 msgr2=0x7fbf381077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.935+0000 7fbf3d9a9640 1 --2- 172.21.15.37:0/717789921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf381073e0 0x7fbf381077c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009900 tx=0x7fbf2802bcd0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.935+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/717789921 shutdown_connections 2024-03-26T06:29:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.935+0000 7fbf3d9a9640 1 --2- 172.21.15.37:0/717789921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf38069310 0x7fbf3810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.935+0000 7fbf3d9a9640 1 --2- 172.21.15.37:0/717789921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf38068970 0x7fbf38068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.935+0000 7fbf3d9a9640 1 --2- 172.21.15.37:0/717789921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf381073e0 0x7fbf381077c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.935+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/717789921 >> 172.21.15.37:0/717789921 conn(0x7fbf38077040 msgr2=0x7fbf38077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.935+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/717789921 shutdown_connections 2024-03-26T06:29:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.935+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/717789921 wait complete. 2024-03-26T06:29:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.936+0000 7fbf3d9a9640 1 Processor -- start 2024-03-26T06:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.936+0000 7fbf3d9a9640 1 -- start start 2024-03-26T06:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.936+0000 7fbf3d9a9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf38068970 0x7fbf3819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.936+0000 7fbf3d9a9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf38069310 0x7fbf3819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf3d9a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf381073e0 0x7fbf38199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf3d9a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf38111660 con 0x7fbf381073e0 2024-03-26T06:29:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf3d9a9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf381114e0 con 0x7fbf38069310 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf3d9a9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf381117e0 con 0x7fbf38068970 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf36ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf38068970 0x7fbf3819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf367fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf38069310 0x7fbf3819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf36ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf38068970 0x7fbf3819f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:52584/0 (socket says 172.21.15.37:52584) 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf36ffd640 1 -- 172.21.15.37:0/411184312 learned_addr learned my addr 172.21.15.37:0/411184312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf377fe640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf381073e0 0x7fbf38199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf377fe640 1 -- 172.21.15.37:0/411184312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf38068970 msgr2=0x7fbf3819f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf377fe640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf38068970 0x7fbf3819f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf377fe640 1 -- 172.21.15.37:0/411184312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf38069310 msgr2=0x7fbf3819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf377fe640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf38069310 0x7fbf3819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf377fe640 1 -- 172.21.15.37:0/411184312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3819a260 con 0x7fbf381073e0 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf367fc640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf38069310 0x7fbf3819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:29:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.937+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf38068970 0x7fbf3819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.938+0000 7fbf377fe640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf381073e0 0x7fbf38199ae0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbf30005fd0 tx=0x7fbf3000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.938+0000 7fbf3c9a7640 1 -- 172.21.15.37:0/411184312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000cd80 con 0x7fbf381073e0 2024-03-26T06:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.938+0000 7fbf3c9a7640 1 -- 172.21.15.37:0/411184312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf30005020 con 0x7fbf381073e0 2024-03-26T06:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.938+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/411184312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3819a550 con 0x7fbf381073e0 2024-03-26T06:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.938+0000 7fbf3c9a7640 1 -- 172.21.15.37:0/411184312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30010720 con 0x7fbf381073e0 2024-03-26T06:29:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.938+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/411184312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf381a67f0 con 0x7fbf381073e0 2024-03-26T06:29:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.940+0000 7fbf3c9a7640 1 -- 172.21.15.37:0/411184312 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fbf300051d0 con 0x7fbf381073e0 2024-03-26T06:29:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf40587740 1 Processor -- start 2024-03-26T06:29:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf40587740 1 -- start start 2024-03-26T06:29:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf40587740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b4ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf3c9a7640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbf200a2a10 0x7fbf200a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf40587740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b4f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf40587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b58550 0x5583e8b5a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf40587740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e8ab35f0 con 0x5583e8b4efd0 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf40587740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e8a5c930 con 0x5583e8b4fd40 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf40587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e8aaf690 con 0x5583e8b58550 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbf200a2a10 0x7fbf200a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf367fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b4f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf3c9a7640 1 -- 172.21.15.37:0/411184312 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fbf300cf450 con 0x7fbf381073e0 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf367fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b4f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36906/0 (socket says 172.21.15.37:36906) 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf367fc640 1 -- 172.21.15.37:0/117980377 learned_addr learned my addr 172.21.15.37:0/117980377 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf377fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b58550 0x5583e8b5a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf367fc640 1 --2- 172.21.15.37:0/117980377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b4ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf377fe640 1 -- 172.21.15.37:0/117980377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 msgr2=0x5583e8b4ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf377fe640 1 --2- 172.21.15.37:0/117980377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b4ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf377fe640 1 -- 172.21.15.37:0/117980377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 msgr2=0x5583e8b4f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf377fe640 1 --2- 172.21.15.37:0/117980377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b4f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf377fe640 1 -- 172.21.15.37:0/117980377 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583e87c8210 con 0x5583e8b58550 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.941+0000 7fbf367fc640 1 --2- 172.21.15.37:0/117980377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b4f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.942+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbf200a2a10 0x7fbf200a4ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbf28000c00 tx=0x7fbf2802cf90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.942+0000 7fbf377fe640 1 --2- 172.21.15.37:0/117980377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b58550 0x5583e8b5a940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf300ccb70 tx=0x7fbf3009c2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17eef50ab277d062 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.942+0000 7fbf1dffb640 1 -- 172.21.15.37:0/117980377 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30099d80 con 0x5583e8b58550 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.942+0000 7fbf1dffb640 1 -- 172.21.15.37:0/117980377 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf30005540 con 0x5583e8b58550 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.942+0000 7fbf1dffb640 1 -- 172.21.15.37:0/117980377 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300a3d90 con 0x5583e8b58550 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.943+0000 7fbf40587740 1 -- 172.21.15.37:0/117980377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b58550 msgr2=0x5583e8b5a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.943+0000 7fbf40587740 1 --2- 172.21.15.37:0/117980377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b58550 0x5583e8b5a940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf300ccb70 tx=0x7fbf3009c2c0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.943+0000 7fbf40587740 1 -- 172.21.15.37:0/117980377 shutdown_connections 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.943+0000 7fbf40587740 1 --2- 172.21.15.37:0/117980377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b58550 0x5583e8b5a940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.943+0000 7fbf40587740 1 --2- 172.21.15.37:0/117980377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b4f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.943+0000 7fbf40587740 1 --2- 172.21.15.37:0/117980377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b4ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.943+0000 7fbf40587740 1 -- 172.21.15.37:0/117980377 >> 172.21.15.37:0/117980377 conn(0x5583e8b165b0 msgr2=0x5583e8b49cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.943+0000 7fbf40587740 1 -- 172.21.15.37:0/117980377 shutdown_connections 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.943+0000 7fbf40587740 1 -- 172.21.15.37:0/117980377 wait complete. 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.943+0000 7fbf40587740 1 -- start start 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.943+0000 7fbf40587740 1 Processor -- start 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf40587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b4efd0 0x5583e8b555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf40587740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b55b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf40587740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b58550 0x5583e8b56050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf40587740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e8b5b280 con 0x5583e8b58550 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf40587740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e8a6fd90 con 0x5583e8b4fd40 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf40587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e8b5b400 con 0x5583e8b4efd0 2024-03-26T06:29:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf40587740 10 client.0 _ll_register_callbacks cb 0x5583e87c7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b58550 0x5583e8b56050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b58550 0x5583e8b56050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36914/0 (socket says 172.21.15.37:36914) 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf377fe640 1 -- 172.21.15.37:0/2094502204 learned_addr learned my addr 172.21.15.37:0/2094502204 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b4efd0 0x5583e8b555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf367fc640 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b55b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf377fe640 1 -- 172.21.15.37:0/2094502204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b4efd0 msgr2=0x5583e8b555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf377fe640 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b4efd0 0x5583e8b555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf377fe640 1 -- 172.21.15.37:0/2094502204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 msgr2=0x5583e8b55b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf377fe640 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b55b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf377fe640 1 -- 172.21.15.37:0/2094502204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583e8894620 con 0x5583e8b58550 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf377fe640 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b58550 0x5583e8b56050 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009fd0 tx=0x7fbf300056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463d68eb3a71e393 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.944+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2094502204 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300b3020 con 0x5583e8b58550 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2094502204 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fbf300ba2e0 con 0x5583e8b58550 2024-03-26T06:29:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf40587740 1 -- 172.21.15.37:0/2094502204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf30005540 con 0x5583e8b58550 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2094502204 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf300afe60 con 0x5583e8b58550 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf40587740 10 client.16029 fetch_fsmap learned FSMap version 40 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf40587740 1 -- 172.21.15.37:0/2094502204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5583e8b675b0 con 0x5583e8b58550 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2094502204 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf30005540 con 0x5583e8b58550 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2094502204 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbf3009aed0 con 0x5583e8b58550 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf40587740 10 client.16029 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf40587740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf40587740 20 client.16029 populate_metadata read hostname 'smithi037' 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf40587740 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5583e8b68900 0x5583e8b6acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf40587740 4 client.16029 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf40587740 1 -- 172.21.15.37:0/2094502204 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583e8ac0b30 con 0x5583e8b68900 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.945+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5583e8b68900 0x5583e8b6acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.946+0000 7fbf16ffd640 10 client.16029 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.946+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5583e8b68900 0x5583e8b6acf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd84874db98f2833 server_cookie=2ecfb230d039487b in_seq=0 out_seq=0 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2094502204 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583e8ac0b30 con 0x5583e8b68900 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf16ffd640 10 client.16029 handle_command_reply: tid=0 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 1 client.16029 shutdown 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 1 -- 172.21.15.37:0/2094502204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b58550 msgr2=0x5583e8b56050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b58550 0x5583e8b56050 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009fd0 tx=0x7fbf300056f0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 1 -- 172.21.15.37:0/2094502204 shutdown_connections 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5583e8b68900 0x5583e8b6acf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b58550 0x5583e8b56050 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b55b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf16ffd640 0 client.16029 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 1 --2- 172.21.15.37:0/2094502204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b4efd0 0x5583e8b555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 1 -- 172.21.15.37:0/2094502204 >> 172.21.15.37:0/2094502204 conn(0x5583e8b165b0 msgr2=0x5583e8b4ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 1 -- 172.21.15.37:0/2094502204 shutdown_connections 2024-03-26T06:29:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 1 -- 172.21.15.37:0/2094502204 wait complete. 2024-03-26T06:29:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.947+0000 7fbf40587740 20 client.16029 trim_cache size 0 max 16384 2024-03-26T06:29:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf40587740 1 Processor -- start 2024-03-26T06:29:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf40587740 1 -- start start 2024-03-26T06:29:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf40587740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b5f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf40587740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b5fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf40587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b58550 0x5583e8b53ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf40587740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf381aed60 con 0x5583e8b4efd0 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf40587740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf381b07a0 con 0x5583e8b4fd40 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf40587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf381af3c0 con 0x5583e8b58550 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf36ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b5f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf36ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b5f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36918/0 (socket says 172.21.15.37:36918) 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf36ffd640 1 -- 172.21.15.37:0/2963460152 learned_addr learned my addr 172.21.15.37:0/2963460152 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf377fe640 1 --2- 172.21.15.37:0/2963460152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b58550 0x5583e8b53ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf367fc640 1 --2- 172.21.15.37:0/2963460152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b5fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf36ffd640 1 -- 172.21.15.37:0/2963460152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b58550 msgr2=0x5583e8b53ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/2963460152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b58550 0x5583e8b53ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf36ffd640 1 -- 172.21.15.37:0/2963460152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 msgr2=0x5583e8b5fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/2963460152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b5fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf36ffd640 1 -- 172.21.15.37:0/2963460152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583e8b4e2a0 con 0x5583e8b4efd0 2024-03-26T06:29:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf367fc640 1 --2- 172.21.15.37:0/2963460152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b5fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.961+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/2963460152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b5f670 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbf28004d20 tx=0x7fbf2802e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b4f888e97f44ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf1d7fa640 1 -- 172.21.15.37:0/2963460152 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28058070 con 0x5583e8b4efd0 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf1d7fa640 1 -- 172.21.15.37:0/2963460152 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf28054de0 con 0x5583e8b4efd0 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf1d7fa640 1 -- 172.21.15.37:0/2963460152 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28005120 con 0x5583e8b4efd0 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf40587740 1 -- 172.21.15.37:0/2963460152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 msgr2=0x5583e8b5f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf40587740 1 --2- 172.21.15.37:0/2963460152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b5f670 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbf28004d20 tx=0x7fbf2802e050 comp rx=0 tx=0).stop 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf40587740 1 -- 172.21.15.37:0/2963460152 shutdown_connections 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf40587740 1 --2- 172.21.15.37:0/2963460152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b58550 0x5583e8b53ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf40587740 1 --2- 172.21.15.37:0/2963460152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b4fd40 0x5583e8b5fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf40587740 1 --2- 172.21.15.37:0/2963460152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4efd0 0x5583e8b5f670 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf40587740 1 -- 172.21.15.37:0/2963460152 >> 172.21.15.37:0/2963460152 conn(0x5583e8b165b0 msgr2=0x5583e8b497f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf40587740 1 -- 172.21.15.37:0/2963460152 shutdown_connections 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.962+0000 7fbf40587740 1 -- 172.21.15.37:0/2963460152 wait complete. 2024-03-26T06:29:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf40587740 1 -- start start 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf40587740 1 Processor -- start 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf40587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b4efd0 0x5583e8b5f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf40587740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4fd40 0x5583e8b51f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf40587740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b58550 0x5583e8b52480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf40587740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf381b0080 con 0x5583e8b4fd40 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf40587740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf381afa20 con 0x5583e8b58550 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf40587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf381ad000 con 0x5583e8b4efd0 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf40587740 10 client.0 _ll_register_callbacks cb 0x7fbf30097d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf367fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4fd40 0x5583e8b51f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf367fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4fd40 0x5583e8b51f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36932/0 (socket says 172.21.15.37:36932) 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf367fc640 1 -- 172.21.15.37:0/2022244245 learned_addr learned my addr 172.21.15.37:0/2022244245 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.963+0000 7fbf377fe640 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b58550 0x5583e8b52480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b4efd0 0x5583e8b5f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf367fc640 1 -- 172.21.15.37:0/2022244245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b4efd0 msgr2=0x5583e8b5f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf367fc640 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b4efd0 0x5583e8b5f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf367fc640 1 -- 172.21.15.37:0/2022244245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b58550 msgr2=0x5583e8b52480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf367fc640 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b58550 0x5583e8b52480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf367fc640 1 -- 172.21.15.37:0/2022244245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583e8b52c10 con 0x5583e8b4fd40 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf367fc640 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4fd40 0x5583e8b51f40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c004540 tx=0x7fbf2c011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a1d56292fc6e1b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2022244245 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c003d40 con 0x5583e8b4fd40 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2022244245 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fbf2c0047a0 con 0x5583e8b4fd40 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf40587740 1 -- 172.21.15.37:0/2022244245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf28054de0 con 0x5583e8b4fd40 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2022244245 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf2c0049a0 con 0x5583e8b4fd40 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf40587740 10 client.16035 fetch_fsmap learned FSMap version 40 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.964+0000 7fbf40587740 1 -- 172.21.15.37:0/2022244245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5583e8bbfa10 con 0x5583e8b4fd40 2024-03-26T06:29:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.965+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2022244245 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf28054de0 con 0x5583e8b4fd40 2024-03-26T06:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.965+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2022244245 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbf2c01d070 con 0x5583e8b4fd40 2024-03-26T06:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.965+0000 7fbf40587740 10 client.16035 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.965+0000 7fbf40587740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.965+0000 7fbf40587740 20 client.16035 populate_metadata read hostname 'smithi037' 2024-03-26T06:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.965+0000 7fbf40587740 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5583e8bc0280 0x5583e8bc26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.965+0000 7fbf40587740 4 client.16035 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.965+0000 7fbf40587740 1 -- 172.21.15.37:0/2022244245 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5583e8abae80 con 0x5583e8bc0280 2024-03-26T06:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.965+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5583e8bc0280 0x5583e8bc26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.965+0000 7fbf16ffd640 10 client.16035 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.965+0000 7fbf36ffd640 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5583e8bc0280 0x5583e8bc26a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c86e136743c2aeda server_cookie=7a5cc0a9d56874e6 in_seq=0 out_seq=0 2024-03-26T06:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf16ffd640 1 -- 172.21.15.37:0/2022244245 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5583e8abae80 con 0x5583e8bc0280 2024-03-26T06:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf16ffd640 10 client.16035 handle_command_reply: tid=0 2024-03-26T06:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 1 client.16035 shutdown 2024-03-26T06:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 1 -- 172.21.15.37:0/2022244245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4fd40 msgr2=0x5583e8b51f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4fd40 0x5583e8b51f40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c004540 tx=0x7fbf2c011cd0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 1 -- 172.21.15.37:0/2022244245 shutdown_connections 2024-03-26T06:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5583e8bc0280 0x5583e8bc26a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583e8b58550 0x5583e8b52480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583e8b4fd40 0x5583e8b51f40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 1 --2- 172.21.15.37:0/2022244245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583e8b4efd0 0x5583e8b5f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf16ffd640 0 client.16035 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 1 -- 172.21.15.37:0/2022244245 >> 172.21.15.37:0/2022244245 conn(0x5583e8b165b0 msgr2=0x5583e8b53aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 1 -- 172.21.15.37:0/2022244245 shutdown_connections 2024-03-26T06:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 1 -- 172.21.15.37:0/2022244245 wait complete. 2024-03-26T06:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf40587740 20 client.16035 trim_cache size 0 max 16384 2024-03-26T06:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:29:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:29:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:29:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/411184312 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbf200a2a10 msgr2=0x7fbf200a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbf200a2a10 0x7fbf200a4ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbf28000c00 tx=0x7fbf2802cf90 comp rx=0 tx=0).stop 2024-03-26T06:29:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/411184312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf381073e0 msgr2=0x7fbf38199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf381073e0 0x7fbf38199ae0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbf30005fd0 tx=0x7fbf3000ee60 comp rx=0 tx=0).stop 2024-03-26T06:29:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/411184312 shutdown_connections 2024-03-26T06:29:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbf200a2a10 0x7fbf200a4ed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf381073e0 0x7fbf38199ae0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf38069310 0x7fbf3819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 --2- 172.21.15.37:0/411184312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf38068970 0x7fbf3819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/411184312 >> 172.21.15.37:0/411184312 conn(0x7fbf38077040 msgr2=0x7fbf3810c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/411184312 shutdown_connections 2024-03-26T06:29:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:35.992+0000 7fbf3d9a9640 1 -- 172.21.15.37:0/411184312 wait complete. 2024-03-26T06:29:36.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:35 smithi066 ceph-mon[36504]: pgmap v201: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-26T06:29:36.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:35 smithi149 ceph-mon[36681]: pgmap v201: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-26T06:29:36.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:35 smithi037 ceph-mon[31974]: pgmap v201: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-26T06:29:36.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 231759 4.41 MB/sec execute 191 sec latency 4953.007 ms 2024-03-26T06:29:37.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 234227 4.47 MB/sec execute 192 sec latency 17.539 ms 2024-03-26T06:29:38.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:37 smithi066 ceph-mon[36504]: pgmap v202: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 799 KiB/s wr, 73 op/s 2024-03-26T06:29:38.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:37 smithi149 ceph-mon[36681]: pgmap v202: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 799 KiB/s wr, 73 op/s 2024-03-26T06:29:38.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:37 smithi037 ceph-mon[31974]: pgmap v202: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 799 KiB/s wr, 73 op/s 2024-03-26T06:29:38.506 INFO:tasks.workunit.client.0.smithi037.stdout: 1 234814 4.46 MB/sec execute 193 sec latency 412.801 ms 2024-03-26T06:29:39.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 234814 4.43 MB/sec execute 194 sec latency 1412.840 ms 2024-03-26T06:29:40.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:39 smithi066 ceph-mon[36504]: pgmap v203: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 799 KiB/s wr, 73 op/s 2024-03-26T06:29:40.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:39 smithi149 ceph-mon[36681]: pgmap v203: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 799 KiB/s wr, 73 op/s 2024-03-26T06:29:40.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:39 smithi037 ceph-mon[31974]: pgmap v203: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 799 KiB/s wr, 73 op/s 2024-03-26T06:29:40.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 234814 4.41 MB/sec execute 195 sec latency 2412.934 ms 2024-03-26T06:29:41.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 234938 4.39 MB/sec execute 196 sec latency 3218.071 ms 2024-03-26T06:29:42.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:41 smithi066 ceph-mon[36504]: pgmap v204: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-03-26T06:29:42.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:41 smithi149 ceph-mon[36681]: pgmap v204: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-03-26T06:29:42.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:41 smithi037 ceph-mon[31974]: pgmap v204: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-03-26T06:29:42.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 234938 4.37 MB/sec execute 197 sec latency 1107.385 ms 2024-03-26T06:29:43.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 234938 4.34 MB/sec execute 198 sec latency 2107.451 ms 2024-03-26T06:29:44.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:43 smithi066 ceph-mon[36504]: pgmap v205: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 801 KiB/s wr, 79 op/s 2024-03-26T06:29:44.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:43 smithi149 ceph-mon[36681]: pgmap v205: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 801 KiB/s wr, 79 op/s 2024-03-26T06:29:44.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:43 smithi037 ceph-mon[31974]: pgmap v205: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 801 KiB/s wr, 79 op/s 2024-03-26T06:29:44.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 234938 4.32 MB/sec execute 199 sec latency 3107.578 ms 2024-03-26T06:29:45.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 234938 4.30 MB/sec execute 200 sec latency 4107.712 ms 2024-03-26T06:29:46.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:45 smithi066 ceph-mon[36504]: pgmap v206: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-03-26T06:29:46.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:45 smithi149 ceph-mon[36681]: pgmap v206: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-03-26T06:29:46.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:45 smithi037 ceph-mon[31974]: pgmap v206: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-03-26T06:29:46.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 235290 4.29 MB/sec execute 201 sec latency 4907.480 ms 2024-03-26T06:29:47.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 237700 4.34 MB/sec execute 202 sec latency 15.222 ms 2024-03-26T06:29:48.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:47 smithi066 ceph-mon[36504]: pgmap v207: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 822 KiB/s wr, 83 op/s 2024-03-26T06:29:48.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:47 smithi149 ceph-mon[36681]: pgmap v207: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 822 KiB/s wr, 83 op/s 2024-03-26T06:29:48.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:47 smithi037 ceph-mon[31974]: pgmap v207: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 822 KiB/s wr, 83 op/s 2024-03-26T06:29:48.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238363 4.33 MB/sec execute 203 sec latency 269.128 ms 2024-03-26T06:29:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.086+0000 7f71ffc6c640 1 -- 172.21.15.37:0/2446333602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71f8100440 msgr2=0x7f71f81008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.086+0000 7f71ffc6c640 1 --2- 172.21.15.37:0/2446333602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71f8100440 0x7f71f81008c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009900 tx=0x7f71e802bc30 comp rx=0 tx=0).stop 2024-03-26T06:29:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.086+0000 7f71ffc6c640 1 -- 172.21.15.37:0/2446333602 shutdown_connections 2024-03-26T06:29:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.086+0000 7f71ffc6c640 1 --2- 172.21.15.37:0/2446333602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71f8100e00 0x7f71f810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.086+0000 7f71ffc6c640 1 --2- 172.21.15.37:0/2446333602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71f8100440 0x7f71f81008c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.086+0000 7f71ffc6c640 1 --2- 172.21.15.37:0/2446333602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71f8105190 0x7f71f8105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.086+0000 7f71ffc6c640 1 -- 172.21.15.37:0/2446333602 >> 172.21.15.37:0/2446333602 conn(0x7f71f8077040 msgr2=0x7f71f8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.086+0000 7f71ffc6c640 1 -- 172.21.15.37:0/2446333602 shutdown_connections 2024-03-26T06:29:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.086+0000 7f71ffc6c640 1 -- 172.21.15.37:0/2446333602 wait complete. 2024-03-26T06:29:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.086+0000 7f71ffc6c640 1 Processor -- start 2024-03-26T06:29:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.086+0000 7f71ffc6c640 1 -- start start 2024-03-26T06:29:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71ffc6c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71f8100440 0x7f71f8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71ffc6c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71f8100e00 0x7f71f819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71ffc6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71f8105190 0x7f71f819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71ffc6c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f8110680 con 0x7f71f8105190 2024-03-26T06:29:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71ffc6c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f8110500 con 0x7f71f8100440 2024-03-26T06:29:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71ffc6c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f8110800 con 0x7f71f8100e00 2024-03-26T06:29:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fe209640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71f8105190 0x7f71f819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fe209640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71f8105190 0x7f71f819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41124/0 (socket says 172.21.15.37:41124) 2024-03-26T06:29:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fe209640 1 -- 172.21.15.37:0/529373901 learned_addr learned my addr 172.21.15.37:0/529373901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fd207640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71f8100e00 0x7f71f819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fda08640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71f8100440 0x7f71f8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fe209640 1 -- 172.21.15.37:0/529373901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71f8100e00 msgr2=0x7f71f819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fe209640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71f8100e00 0x7f71f819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fe209640 1 -- 172.21.15.37:0/529373901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71f8100440 msgr2=0x7f71f8199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fe209640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71f8100440 0x7f71f8199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fe209640 1 -- 172.21.15.37:0/529373901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f819ed20 con 0x7f71f8105190 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fd207640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71f8100e00 0x7f71f819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fda08640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71f8100440 0x7f71f8199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71fe209640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71f8105190 0x7f71f819e5f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f71f000da40 tx=0x7f71f000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71eeffd640 1 -- 172.21.15.37:0/529373901 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0017070 con 0x7f71f8105190 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71eeffd640 1 -- 172.21.15.37:0/529373901 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71f0003e50 con 0x7f71f8105190 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71ffc6c640 1 -- 172.21.15.37:0/529373901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f819f010 con 0x7f71f8105190 2024-03-26T06:29:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.087+0000 7f71eeffd640 1 -- 172.21.15.37:0/529373901 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0010460 con 0x7f71f8105190 2024-03-26T06:29:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.088+0000 7f71ffc6c640 1 -- 172.21.15.37:0/529373901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f81a68a0 con 0x7f71f8105190 2024-03-26T06:29:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.090+0000 7f71eeffd640 1 -- 172.21.15.37:0/529373901 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f71f0003970 con 0x7f71f8105190 2024-03-26T06:29:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.090+0000 7f71eeffd640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f71e40a2a10 0x7f71e40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.090+0000 7f720284f740 1 Processor -- start 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.090+0000 7f71fda08640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f71e40a2a10 0x7f71e40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.090+0000 7f71eeffd640 1 -- 172.21.15.37:0/529373901 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f71f00cebd0 con 0x7f71f8105190 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f720284f740 1 -- start start 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f720284f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f4691d40 0x5573f4690a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f720284f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4690fd0 0x5573f4691450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f720284f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f469a550 0x5573f469c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f720284f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f45f55f0 con 0x5573f469a550 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f720284f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f459e930 con 0x5573f4690fd0 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f720284f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f45f1690 con 0x5573f4691d40 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fda08640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f71e40a2a10 0x7f71e40a4ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009ab0 tx=0x7f71f40094b0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fe209640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f469a550 0x5573f469c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fe209640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f469a550 0x5573f469c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41140/0 (socket says 172.21.15.37:41140) 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fe209640 1 -- 172.21.15.37:0/803098244 learned_addr learned my addr 172.21.15.37:0/803098244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fd207640 1 --2- 172.21.15.37:0/803098244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4690fd0 0x5573f4691450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fda08640 1 --2- 172.21.15.37:0/803098244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f4691d40 0x5573f4690a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fe209640 1 -- 172.21.15.37:0/803098244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f4691d40 msgr2=0x5573f4690a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fe209640 1 --2- 172.21.15.37:0/803098244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f4691d40 0x5573f4690a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fe209640 1 -- 172.21.15.37:0/803098244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4690fd0 msgr2=0x5573f4691450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fe209640 1 --2- 172.21.15.37:0/803098244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4690fd0 0x5573f4691450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fe209640 1 -- 172.21.15.37:0/803098244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573f430a210 con 0x5573f469a550 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fda08640 1 --2- 172.21.15.37:0/803098244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f4691d40 0x5573f4690a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.091+0000 7f71fe209640 1 --2- 172.21.15.37:0/803098244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f469a550 0x5573f469c940 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f71f0097c30 tx=0x7f71f009b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76022e423a4f2804 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.092+0000 7f71e3fff640 1 -- 172.21.15.37:0/803098244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0020040 con 0x5573f469a550 2024-03-26T06:29:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.092+0000 7f71e3fff640 1 -- 172.21.15.37:0/803098244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71f00a0650 con 0x5573f469a550 2024-03-26T06:29:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.092+0000 7f71e3fff640 1 -- 172.21.15.37:0/803098244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f00cc5d0 con 0x5573f469a550 2024-03-26T06:29:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.092+0000 7f720284f740 1 -- 172.21.15.37:0/803098244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f469a550 msgr2=0x5573f469c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.092+0000 7f720284f740 1 --2- 172.21.15.37:0/803098244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f469a550 0x5573f469c940 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f71f0097c30 tx=0x7f71f009b960 comp rx=0 tx=0).stop 2024-03-26T06:29:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.092+0000 7f720284f740 1 -- 172.21.15.37:0/803098244 shutdown_connections 2024-03-26T06:29:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.092+0000 7f720284f740 1 --2- 172.21.15.37:0/803098244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f469a550 0x5573f469c940 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.092+0000 7f720284f740 1 --2- 172.21.15.37:0/803098244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4690fd0 0x5573f4691450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.092+0000 7f720284f740 1 --2- 172.21.15.37:0/803098244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f4691d40 0x5573f4690a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.092+0000 7f720284f740 1 -- 172.21.15.37:0/803098244 >> 172.21.15.37:0/803098244 conn(0x5573f46585b0 msgr2=0x5573f468bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.093+0000 7f720284f740 1 -- 172.21.15.37:0/803098244 shutdown_connections 2024-03-26T06:29:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.093+0000 7f720284f740 1 -- 172.21.15.37:0/803098244 wait complete. 2024-03-26T06:29:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.093+0000 7f720284f740 1 -- start start 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.093+0000 7f720284f740 1 Processor -- start 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.093+0000 7f720284f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4690fd0 0x5573f4697550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.093+0000 7f720284f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4691d40 0x5573f4697a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.093+0000 7f720284f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f469a550 0x5573f4697fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.093+0000 7f720284f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f469d220 con 0x5573f4690fd0 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.093+0000 7f720284f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f45b1d90 con 0x5573f4691d40 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.093+0000 7f720284f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f469d3a0 con 0x5573f469a550 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.093+0000 7f720284f740 10 client.0 _ll_register_callbacks cb 0x5573f4309a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fda08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4690fd0 0x5573f4697550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fda08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4690fd0 0x5573f4697550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41156/0 (socket says 172.21.15.37:41156) 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fda08640 1 -- 172.21.15.37:0/1793010653 learned_addr learned my addr 172.21.15.37:0/1793010653 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fe209640 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f469a550 0x5573f4697fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fd207640 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4691d40 0x5573f4697a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fda08640 1 -- 172.21.15.37:0/1793010653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f469a550 msgr2=0x5573f4697fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fda08640 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f469a550 0x5573f4697fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fda08640 1 -- 172.21.15.37:0/1793010653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4691d40 msgr2=0x5573f4697a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fda08640 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4691d40 0x5573f4697a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fda08640 1 -- 172.21.15.37:0/1793010653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573f43d6620 con 0x5573f4690fd0 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fe209640 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f469a550 0x5573f4697fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fd207640 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4691d40 0x5573f4697a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71fda08640 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4690fd0 0x5573f4697550 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f71f4011890 tx=0x7f71f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be85c28d853b7605 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71e17fa640 1 -- 172.21.15.37:0/1793010653 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4023070 con 0x5573f4690fd0 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71e17fa640 1 -- 172.21.15.37:0/1793010653 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f71f4010d60 con 0x5573f4690fd0 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f720284f740 1 -- 172.21.15.37:0/1793010653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71f00a0650 con 0x5573f4690fd0 2024-03-26T06:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f71e17fa640 1 -- 172.21.15.37:0/1793010653 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71f4011e10 con 0x5573f4690fd0 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f720284f740 10 client.16047 fetch_fsmap learned FSMap version 40 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.094+0000 7f720284f740 1 -- 172.21.15.37:0/1793010653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5573f46a9520 con 0x5573f4690fd0 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.095+0000 7f71e17fa640 1 -- 172.21.15.37:0/1793010653 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71f00a0650 con 0x5573f4690fd0 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.095+0000 7f71e17fa640 1 -- 172.21.15.37:0/1793010653 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f71f400f040 con 0x5573f4690fd0 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.095+0000 7f720284f740 10 client.16047 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.095+0000 7f720284f740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.095+0000 7f720284f740 20 client.16047 populate_metadata read hostname 'smithi037' 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.095+0000 7f720284f740 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5573f46aa990 0x5573f46ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.095+0000 7f720284f740 4 client.16047 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.095+0000 7f720284f740 1 -- 172.21.15.37:0/1793010653 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573f4602b30 con 0x5573f46aa990 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.095+0000 7f71fd207640 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5573f46aa990 0x5573f46ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f71fd207640 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5573f46aa990 0x5573f46ace10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eddbcb983abb680e server_cookie=91d7de8d9f4b1e8b in_seq=0 out_seq=0 2024-03-26T06:29:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f71e17fa640 10 client.16047 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f71e17fa640 1 -- 172.21.15.37:0/1793010653 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5573f4602b30 con 0x5573f46aa990 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f71e17fa640 10 client.16047 handle_command_reply: tid=0 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f720284f740 1 client.16047 shutdown 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f720284f740 1 -- 172.21.15.37:0/1793010653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4690fd0 msgr2=0x5573f4697550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f720284f740 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4690fd0 0x5573f4697550 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f71f4011890 tx=0x7f71f40046c0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f720284f740 1 -- 172.21.15.37:0/1793010653 shutdown_connections 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f720284f740 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5573f46aa990 0x5573f46ace10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f720284f740 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f469a550 0x5573f4697fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f720284f740 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4691d40 0x5573f4697a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f71e17fa640 0 client.16047 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f720284f740 1 --2- 172.21.15.37:0/1793010653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4690fd0 0x5573f4697550 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f720284f740 1 -- 172.21.15.37:0/1793010653 >> 172.21.15.37:0/1793010653 conn(0x5573f46585b0 msgr2=0x5573f468da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f720284f740 1 -- 172.21.15.37:0/1793010653 shutdown_connections 2024-03-26T06:29:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.096+0000 7f720284f740 1 -- 172.21.15.37:0/1793010653 wait complete. 2024-03-26T06:29:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.097+0000 7f720284f740 20 client.16047 trim_cache size 0 max 16384 2024-03-26T06:29:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.110+0000 7f720284f740 1 Processor -- start 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.110+0000 7f720284f740 1 -- start start 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f720284f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4690fd0 0x5573f4699c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f720284f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f4696530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f720284f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f469a550 0x5573f46947b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f720284f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81aedc0 con 0x5573f4691d40 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f720284f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81b0800 con 0x5573f4690fd0 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f720284f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81af420 con 0x5573f469a550 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71fd207640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f4696530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71fd207640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f4696530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41170/0 (socket says 172.21.15.37:41170) 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71fd207640 1 -- 172.21.15.37:0/1820064068 learned_addr learned my addr 172.21.15.37:0/1820064068 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71fda08640 1 --2- 172.21.15.37:0/1820064068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4690fd0 0x5573f4699c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71fe209640 1 --2- 172.21.15.37:0/1820064068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f469a550 0x5573f46947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71fd207640 1 -- 172.21.15.37:0/1820064068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f469a550 msgr2=0x5573f46947b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71fd207640 1 --2- 172.21.15.37:0/1820064068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f469a550 0x5573f46947b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71fd207640 1 -- 172.21.15.37:0/1820064068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4690fd0 msgr2=0x5573f4699c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71fd207640 1 --2- 172.21.15.37:0/1820064068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4690fd0 0x5573f4699c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71fd207640 1 -- 172.21.15.37:0/1820064068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573f469ccb0 con 0x5573f4691d40 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71fd207640 1 --2- 172.21.15.37:0/1820064068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f4696530 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f71f810d440 tx=0x7f71e8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68995a5cb59327c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71e37fe640 1 -- 172.21.15.37:0/1820064068 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e80041e0 con 0x5573f4691d40 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71e37fe640 1 -- 172.21.15.37:0/1820064068 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e8049070 con 0x5573f4691d40 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.111+0000 7f71e37fe640 1 -- 172.21.15.37:0/1820064068 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e80052a0 con 0x5573f4691d40 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.112+0000 7f720284f740 1 -- 172.21.15.37:0/1820064068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 msgr2=0x5573f4696530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.112+0000 7f720284f740 1 --2- 172.21.15.37:0/1820064068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f4696530 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f71f810d440 tx=0x7f71e8009260 comp rx=0 tx=0).stop 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.112+0000 7f720284f740 1 -- 172.21.15.37:0/1820064068 shutdown_connections 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.112+0000 7f720284f740 1 --2- 172.21.15.37:0/1820064068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f469a550 0x5573f46947b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.112+0000 7f720284f740 1 --2- 172.21.15.37:0/1820064068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f4696530 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.112+0000 7f720284f740 1 --2- 172.21.15.37:0/1820064068 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f4690fd0 0x5573f4699c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.112+0000 7f720284f740 1 -- 172.21.15.37:0/1820064068 >> 172.21.15.37:0/1820064068 conn(0x5573f46585b0 msgr2=0x5573f4695e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.112+0000 7f720284f740 1 -- 172.21.15.37:0/1820064068 shutdown_connections 2024-03-26T06:29:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.112+0000 7f720284f740 1 -- 172.21.15.37:0/1820064068 wait complete. 2024-03-26T06:29:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.112+0000 7f720284f740 1 -- start start 2024-03-26T06:29:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.112+0000 7f720284f740 1 Processor -- start 2024-03-26T06:29:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f720284f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f4690fd0 0x5573f46a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f720284f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f469fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f720284f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f469a550 0x5573f46a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f720284f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81b00e0 con 0x5573f4691d40 2024-03-26T06:29:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f720284f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81afa80 con 0x5573f469a550 2024-03-26T06:29:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f720284f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81ad060 con 0x5573f4690fd0 2024-03-26T06:29:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f720284f740 10 client.0 _ll_register_callbacks cb 0x7f71f0009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:29:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f71fd207640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f469fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f71fd207640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f469fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41178/0 (socket says 172.21.15.37:41178) 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f71fd207640 1 -- 172.21.15.37:0/536713929 learned_addr learned my addr 172.21.15.37:0/536713929 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f71fda08640 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f4690fd0 0x5573f46a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f71fe209640 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f469a550 0x5573f46a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f71fd207640 1 -- 172.21.15.37:0/536713929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f4690fd0 msgr2=0x5573f46a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f71fd207640 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f4690fd0 0x5573f46a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f71fd207640 1 -- 172.21.15.37:0/536713929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f469a550 msgr2=0x5573f46a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f71fd207640 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f469a550 0x5573f46a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.113+0000 7f71fd207640 1 -- 172.21.15.37:0/536713929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573f4700f40 con 0x5573f4691d40 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f71fd207640 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f469fec0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f71e8046040 tx=0x7f71e802c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7452347cb0a0e2b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f71e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:29:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f71e17fa640 1 -- 172.21.15.37:0/536713929 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8051020 con 0x5573f4691d40 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f71e17fa640 1 -- 172.21.15.37:0/536713929 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f71e80043f0 con 0x5573f4691d40 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f720284f740 1 -- 172.21.15.37:0/536713929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71e8049070 con 0x5573f4691d40 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f71e17fa640 1 -- 172.21.15.37:0/536713929 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e805c050 con 0x5573f4691d40 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f720284f740 10 client.16053 fetch_fsmap learned FSMap version 40 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f720284f740 1 -- 172.21.15.37:0/536713929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5573f4702100 con 0x5573f4691d40 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f71e17fa640 1 -- 172.21.15.37:0/536713929 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e8049070 con 0x5573f4691d40 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f71e17fa640 1 -- 172.21.15.37:0/536713929 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f71e8004c20 con 0x5573f4691d40 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f720284f740 10 client.16053 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f720284f740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f720284f740 20 client.16053 populate_metadata read hostname 'smithi037' 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f720284f740 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5573f4702920 0x5573f4704d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f720284f740 4 client.16053 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.114+0000 7f720284f740 1 -- 172.21.15.37:0/536713929 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5573f45fce80 con 0x5573f4702920 2024-03-26T06:29:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.115+0000 7f71fda08640 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5573f4702920 0x5573f4704d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:29:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.115+0000 7f71fda08640 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5573f4702920 0x5573f4704d40 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3505f1fd5ec58e8 server_cookie=915f43634507c5ec in_seq=0 out_seq=0 2024-03-26T06:29:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.115+0000 7f71e17fa640 10 client.16053 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.150+0000 7f71e17fa640 1 -- 172.21.15.37:0/536713929 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5573f45fce80 con 0x5573f4702920 2024-03-26T06:29:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.150+0000 7f71e17fa640 10 client.16053 handle_command_reply: tid=0 2024-03-26T06:29:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.150+0000 7f720284f740 1 client.16053 shutdown 2024-03-26T06:29:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f720284f740 1 -- 172.21.15.37:0/536713929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 msgr2=0x5573f469fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f720284f740 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f469fec0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f71e8046040 tx=0x7f71e802c9a0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f720284f740 1 -- 172.21.15.37:0/536713929 shutdown_connections 2024-03-26T06:29:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f720284f740 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5573f4702920 0x5573f4704d40 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f720284f740 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573f469a550 0x5573f46a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f720284f740 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f4691d40 0x5573f469fec0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f720284f740 1 --2- 172.21.15.37:0/536713929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573f4690fd0 0x5573f46a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f71e17fa640 0 client.16053 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:29:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f720284f740 1 -- 172.21.15.37:0/536713929 >> 172.21.15.37:0/536713929 conn(0x5573f46585b0 msgr2=0x5573f468a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f720284f740 1 -- 172.21.15.37:0/536713929 shutdown_connections 2024-03-26T06:29:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f720284f740 1 -- 172.21.15.37:0/536713929 wait complete. 2024-03-26T06:29:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f720284f740 20 client.16053 trim_cache size 0 max 16384 2024-03-26T06:29:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:29:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:29:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:29:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f71ffc6c640 1 -- 172.21.15.37:0/529373901 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f71e40a2a10 msgr2=0x7f71e40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f71ffc6c640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f71e40a2a10 0x7f71e40a4ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009ab0 tx=0x7f71f40094b0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f71ffc6c640 1 -- 172.21.15.37:0/529373901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71f8105190 msgr2=0x7f71f819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f71ffc6c640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71f8105190 0x7f71f819e5f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f71f000da40 tx=0x7f71f000df00 comp rx=0 tx=0).stop 2024-03-26T06:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f71ffc6c640 1 -- 172.21.15.37:0/529373901 shutdown_connections 2024-03-26T06:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f71ffc6c640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f71e40a2a10 0x7f71e40a4ed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f71ffc6c640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71f8105190 0x7f71f819e5f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f71ffc6c640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71f8100e00 0x7f71f819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f71ffc6c640 1 --2- 172.21.15.37:0/529373901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71f8100440 0x7f71f8199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.151+0000 7f71ffc6c640 1 -- 172.21.15.37:0/529373901 >> 172.21.15.37:0/529373901 conn(0x7f71f8077040 msgr2=0x7f71f8068970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.152+0000 7f71ffc6c640 1 -- 172.21.15.37:0/529373901 shutdown_connections 2024-03-26T06:29:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:29:49.152+0000 7f71ffc6c640 1 -- 172.21.15.37:0/529373901 wait complete. 2024-03-26T06:29:49.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238363 4.31 MB/sec execute 204 sec latency 1269.211 ms 2024-03-26T06:29:50.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:49 smithi066 ceph-mon[36504]: pgmap v208: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 822 KiB/s wr, 83 op/s 2024-03-26T06:29:50.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:49 smithi149 ceph-mon[36681]: pgmap v208: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 822 KiB/s wr, 83 op/s 2024-03-26T06:29:50.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:49 smithi037 ceph-mon[31974]: pgmap v208: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 822 KiB/s wr, 83 op/s 2024-03-26T06:29:50.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238363 4.29 MB/sec execute 205 sec latency 2269.282 ms 2024-03-26T06:29:51.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238419 4.27 MB/sec execute 206 sec latency 3070.118 ms 2024-03-26T06:29:52.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:51 smithi066 ceph-mon[36504]: pgmap v209: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:29:52.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:51 smithi149 ceph-mon[36681]: pgmap v209: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:29:52.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:51 smithi037 ceph-mon[31974]: pgmap v209: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:29:52.507 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238419 4.25 MB/sec execute 207 sec latency 1154.413 ms 2024-03-26T06:29:53.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238419 4.23 MB/sec execute 208 sec latency 2154.536 ms 2024-03-26T06:29:54.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:53 smithi066 ceph-mon[36504]: pgmap v210: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-03-26T06:29:54.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:53 smithi149 ceph-mon[36681]: pgmap v210: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-03-26T06:29:54.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:53 smithi037 ceph-mon[31974]: pgmap v210: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-03-26T06:29:54.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238419 4.21 MB/sec execute 209 sec latency 3154.585 ms 2024-03-26T06:29:55.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238419 4.19 MB/sec execute 210 sec latency 4154.690 ms 2024-03-26T06:29:56.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:55 smithi066 ceph-mon[36504]: pgmap v211: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-03-26T06:29:56.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:55 smithi149 ceph-mon[36681]: pgmap v211: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-03-26T06:29:56.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:55 smithi037 ceph-mon[31974]: pgmap v211: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-03-26T06:29:56.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238487 4.17 MB/sec execute 211 sec latency 4955.903 ms 2024-03-26T06:29:57.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238487 4.15 MB/sec execute 212 sec latency 1149.380 ms 2024-03-26T06:29:58.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:57 smithi066 ceph-mon[36504]: pgmap v212: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.0 MiB/s wr, 91 op/s 2024-03-26T06:29:58.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:57 smithi149 ceph-mon[36681]: pgmap v212: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.0 MiB/s wr, 91 op/s 2024-03-26T06:29:58.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:57 smithi037 ceph-mon[31974]: pgmap v212: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.0 MiB/s wr, 91 op/s 2024-03-26T06:29:58.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238487 4.13 MB/sec execute 213 sec latency 2149.510 ms 2024-03-26T06:29:59.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238487 4.11 MB/sec execute 214 sec latency 3149.585 ms 2024-03-26T06:30:00.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:29:59 smithi066 ceph-mon[36504]: pgmap v213: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.0 MiB/s wr, 91 op/s 2024-03-26T06:30:00.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:29:59 smithi149 ceph-mon[36681]: pgmap v213: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.0 MiB/s wr, 91 op/s 2024-03-26T06:30:00.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:29:59 smithi037 ceph-mon[31974]: pgmap v213: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.0 MiB/s wr, 91 op/s 2024-03-26T06:30:00.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238487 4.09 MB/sec execute 215 sec latency 4149.675 ms 2024-03-26T06:30:01.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:00 smithi066 ceph-mon[36504]: overall HEALTH_OK 2024-03-26T06:30:01.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:00 smithi149 ceph-mon[36681]: overall HEALTH_OK 2024-03-26T06:30:01.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:00 smithi037 ceph-mon[31974]: overall HEALTH_OK 2024-03-26T06:30:01.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240073 4.10 MB/sec execute 216 sec latency 4429.016 ms 2024-03-26T06:30:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.240+0000 7f79b2d89640 1 -- 172.21.15.37:0/2390252683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79ac101f00 msgr2=0x7f79ac105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.240+0000 7f79b2d89640 1 --2- 172.21.15.37:0/2390252683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79ac101f00 0x7f79ac105f40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f799c009ca0 tx=0x7f799c02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.240+0000 7f79b2d89640 1 -- 172.21.15.37:0/2390252683 shutdown_connections 2024-03-26T06:30:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.240+0000 7f79b2d89640 1 --2- 172.21.15.37:0/2390252683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79ac106620 0x7f79ac10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.240+0000 7f79b2d89640 1 --2- 172.21.15.37:0/2390252683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79ac101f00 0x7f79ac105f40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.240+0000 7f79b2d89640 1 --2- 172.21.15.37:0/2390252683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79ac101550 0x7f79ac101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.240+0000 7f79b2d89640 1 -- 172.21.15.37:0/2390252683 >> 172.21.15.37:0/2390252683 conn(0x7f79ac0fcdb0 msgr2=0x7f79ac0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.240+0000 7f79b2d89640 1 -- 172.21.15.37:0/2390252683 shutdown_connections 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.240+0000 7f79b2d89640 1 -- 172.21.15.37:0/2390252683 wait complete. 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b2d89640 1 Processor -- start 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b2d89640 1 -- start start 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b2d89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79ac101550 0x7f79ac19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b2d89640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79ac106620 0x7f79ac19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b2d89640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79ac197990 0x7f79ac197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b2d89640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1116b0 con 0x7f79ac101550 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b2d89640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac111530 con 0x7f79ac106620 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b2d89640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac111830 con 0x7f79ac197990 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b0b25640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79ac101550 0x7f79ac19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b0b25640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79ac101550 0x7f79ac19d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53318/0 (socket says 172.21.15.37:53318) 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b0b25640 1 -- 172.21.15.37:0/331887117 learned_addr learned my addr 172.21.15.37:0/331887117 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79abfff640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79ac106620 0x7f79ac19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b1326640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79ac197990 0x7f79ac197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b0b25640 1 -- 172.21.15.37:0/331887117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79ac197990 msgr2=0x7f79ac197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b0b25640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79ac197990 0x7f79ac197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b0b25640 1 -- 172.21.15.37:0/331887117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79ac106620 msgr2=0x7f79ac19dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b0b25640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79ac106620 0x7f79ac19dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79b0b25640 1 -- 172.21.15.37:0/331887117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac1984b0 con 0x7f79ac101550 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.241+0000 7f79abfff640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79ac106620 0x7f79ac19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.242+0000 7f79b0b25640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79ac101550 0x7f79ac19d7e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f79a400e9f0 tx=0x7f79a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.242+0000 7f79a9ffb640 1 -- 172.21.15.37:0/331887117 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400cd80 con 0x7f79ac101550 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.242+0000 7f79b2d89640 1 -- 172.21.15.37:0/331887117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac1987a0 con 0x7f79ac101550 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.242+0000 7f79a9ffb640 1 -- 172.21.15.37:0/331887117 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79a4005500 con 0x7f79ac101550 2024-03-26T06:30:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.242+0000 7f79a9ffb640 1 -- 172.21.15.37:0/331887117 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4010690 con 0x7f79ac101550 2024-03-26T06:30:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.242+0000 7f79b2d89640 1 -- 172.21.15.37:0/331887117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac1a49f0 con 0x7f79ac101550 2024-03-26T06:30:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.244+0000 7f79a9ffb640 1 -- 172.21.15.37:0/331887117 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f79a40046c0 con 0x7f79ac101550 2024-03-26T06:30:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.244+0000 7f79b596b740 1 Processor -- start 2024-03-26T06:30:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.244+0000 7f79b596b740 1 -- start start 2024-03-26T06:30:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.244+0000 7f79a9ffb640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f79980a2a10 0x7f79980a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b596b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6fad40 0x55c4de6f9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b596b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6f9fd0 0x55c4de6fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b596b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 0x55c4de705940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b596b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4de65e5f0 con 0x55c4de703550 2024-03-26T06:30:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b596b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4de607930 con 0x55c4de6fad40 2024-03-26T06:30:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b596b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4de65a690 con 0x55c4de6f9fd0 2024-03-26T06:30:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b1326640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f79980a2a10 0x7f79980a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79a9ffb640 1 -- 172.21.15.37:0/331887117 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f79a40cf1d0 con 0x7f79ac101550 2024-03-26T06:30:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b1326640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 0x55c4de705940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b1326640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 0x55c4de705940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53322/0 (socket says 172.21.15.37:53322) 2024-03-26T06:30:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b1326640 1 -- 172.21.15.37:0/2818387200 learned_addr learned my addr 172.21.15.37:0/2818387200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b0b25640 1 --2- 172.21.15.37:0/2818387200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6fad40 0x55c4de6f9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79abfff640 1 --2- 172.21.15.37:0/2818387200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6f9fd0 0x55c4de6fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b1326640 1 -- 172.21.15.37:0/2818387200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6f9fd0 msgr2=0x55c4de6fa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b1326640 1 --2- 172.21.15.37:0/2818387200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6f9fd0 0x55c4de6fa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b1326640 1 -- 172.21.15.37:0/2818387200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6fad40 msgr2=0x55c4de6f9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b1326640 1 --2- 172.21.15.37:0/2818387200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6fad40 0x55c4de6f9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b1326640 1 -- 172.21.15.37:0/2818387200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4de373210 con 0x55c4de703550 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79abfff640 1 --2- 172.21.15.37:0/2818387200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6f9fd0 0x55c4de6fa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b0b25640 1 --2- 172.21.15.37:0/2818387200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6fad40 0x55c4de6f9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b1326640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f79980a2a10 0x7f79980a4ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7994009fd0 tx=0x7f7994009b80 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79b1326640 1 --2- 172.21.15.37:0/2818387200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 0x55c4de705940 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7994005a70 tx=0x7f799401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557d5c895c987a89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79a2ffd640 1 -- 172.21.15.37:0/2818387200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994024070 con 0x55c4de703550 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.245+0000 7f79a2ffd640 1 -- 172.21.15.37:0/2818387200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f799400fe50 con 0x55c4de703550 2024-03-26T06:30:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.246+0000 7f79a2ffd640 1 -- 172.21.15.37:0/2818387200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799401f9d0 con 0x55c4de703550 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.246+0000 7f79b596b740 1 -- 172.21.15.37:0/2818387200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 msgr2=0x55c4de705940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.246+0000 7f79b596b740 1 --2- 172.21.15.37:0/2818387200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 0x55c4de705940 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7994005a70 tx=0x7f799401d000 comp rx=0 tx=0).stop 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.246+0000 7f79b596b740 1 -- 172.21.15.37:0/2818387200 shutdown_connections 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.246+0000 7f79b596b740 1 --2- 172.21.15.37:0/2818387200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 0x55c4de705940 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.246+0000 7f79b596b740 1 --2- 172.21.15.37:0/2818387200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6f9fd0 0x55c4de6fa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.246+0000 7f79b596b740 1 --2- 172.21.15.37:0/2818387200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6fad40 0x55c4de6f9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.246+0000 7f79b596b740 1 -- 172.21.15.37:0/2818387200 >> 172.21.15.37:0/2818387200 conn(0x55c4de6c15b0 msgr2=0x55c4de6f4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.246+0000 7f79b596b740 1 -- 172.21.15.37:0/2818387200 shutdown_connections 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.246+0000 7f79b596b740 1 -- 172.21.15.37:0/2818387200 wait complete. 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b596b740 1 -- start start 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b596b740 1 Processor -- start 2024-03-26T06:30:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b596b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 0x55c4de700650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b596b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6fad40 0x55c4de700b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b596b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 0x55c4de7010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b596b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4de706270 con 0x55c4de703550 2024-03-26T06:30:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b596b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4de61ad90 con 0x55c4de6f9fd0 2024-03-26T06:30:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b596b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4de7063f0 con 0x55c4de6fad40 2024-03-26T06:30:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b596b740 10 client.0 _ll_register_callbacks cb 0x55c4de372a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:30:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b0b25640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 0x55c4de700650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b0b25640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 0x55c4de700650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:49774/0 (socket says 172.21.15.37:49774) 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79abfff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6fad40 0x55c4de700b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b0b25640 1 -- 172.21.15.37:0/508250102 learned_addr learned my addr 172.21.15.37:0/508250102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b1326640 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 0x55c4de7010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b0b25640 1 -- 172.21.15.37:0/508250102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6fad40 msgr2=0x55c4de700b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b0b25640 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6fad40 0x55c4de700b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b0b25640 1 -- 172.21.15.37:0/508250102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 msgr2=0x55c4de7010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.247+0000 7f79b0b25640 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 0x55c4de7010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f79b0b25640 1 -- 172.21.15.37:0/508250102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4de43f620 con 0x55c4de6f9fd0 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f79abfff640 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6fad40 0x55c4de700b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f79b0b25640 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 0x55c4de700650 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f79a40cca00 tx=0x7f79a409b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=171abad35b5494fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f798bfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:30:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f798bfff640 1 -- 172.21.15.37:0/508250102 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a409bf60 con 0x55c4de6f9fd0 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f798bfff640 1 -- 172.21.15.37:0/508250102 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f79a40860b0 con 0x55c4de6f9fd0 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f79b596b740 1 -- 172.21.15.37:0/508250102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f799400fe50 con 0x55c4de6f9fd0 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f798bfff640 1 -- 172.21.15.37:0/508250102 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79a401b160 con 0x55c4de6f9fd0 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f79b596b740 10 client.25619 fetch_fsmap learned FSMap version 40 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f79b596b740 1 -- 172.21.15.37:0/508250102 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c4de712620 con 0x55c4de6f9fd0 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f798bfff640 1 -- 172.21.15.37:0/508250102 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f799400fe50 con 0x55c4de6f9fd0 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f798bfff640 1 -- 172.21.15.37:0/508250102 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f79a401c030 con 0x55c4de6f9fd0 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f79b596b740 10 client.25619 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f79b596b740 10 client.25619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.248+0000 7f79b596b740 20 client.25619 populate_metadata read hostname 'smithi037' 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.249+0000 7f79b596b740 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c4de713a90 0x55c4de715ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.249+0000 7f79b596b740 4 client.25619 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.249+0000 7f79b596b740 1 -- 172.21.15.37:0/508250102 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4de66bb30 con 0x55c4de713a90 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.249+0000 7f79abfff640 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c4de713a90 0x55c4de715ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.249+0000 7f79abfff640 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c4de713a90 0x55c4de715ea0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeda6432cc69d04a server_cookie=3782635c4489ca7a in_seq=0 out_seq=0 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.249+0000 7f798bfff640 10 client.25619 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f798bfff640 1 -- 172.21.15.37:0/508250102 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c4de66bb30 con 0x55c4de713a90 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f798bfff640 10 client.25619 handle_command_reply: tid=0 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 1 client.25619 shutdown 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 1 -- 172.21.15.37:0/508250102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 msgr2=0x55c4de700650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 0x55c4de700650 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f79a40cca00 tx=0x7f79a409b460 comp rx=0 tx=0).stop 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 1 -- 172.21.15.37:0/508250102 shutdown_connections 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c4de713a90 0x55c4de715ea0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de703550 0x55c4de7010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6fad40 0x55c4de700b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 1 --2- 172.21.15.37:0/508250102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 0x55c4de700650 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f798bfff640 0 client.25619 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 1 -- 172.21.15.37:0/508250102 >> 172.21.15.37:0/508250102 conn(0x55c4de6c15b0 msgr2=0x55c4de6f6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 1 -- 172.21.15.37:0/508250102 shutdown_connections 2024-03-26T06:30:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 1 -- 172.21.15.37:0/508250102 wait complete. 2024-03-26T06:30:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.250+0000 7f79b596b740 20 client.25619 trim_cache size 0 max 16384 2024-03-26T06:30:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.263+0000 7f79b596b740 1 Processor -- start 2024-03-26T06:30:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.263+0000 7f79b596b740 1 -- start start 2024-03-26T06:30:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.263+0000 7f79b596b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6f9fd0 0x55c4de6f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.263+0000 7f79b596b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6fad40 0x55c4de6f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.263+0000 7f79b596b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de703550 0x55c4de6fef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b596b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1acc60 con 0x55c4de6f9fd0 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b596b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1ae6a0 con 0x55c4de703550 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b596b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1ad2c0 con 0x55c4de6fad40 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b0b25640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6f9fd0 0x55c4de6f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b0b25640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6f9fd0 0x55c4de6f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53342/0 (socket says 172.21.15.37:53342) 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b0b25640 1 -- 172.21.15.37:0/1215814742 learned_addr learned my addr 172.21.15.37:0/1215814742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79abfff640 1 --2- 172.21.15.37:0/1215814742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6fad40 0x55c4de6f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b1326640 1 --2- 172.21.15.37:0/1215814742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de703550 0x55c4de6fef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b0b25640 1 -- 172.21.15.37:0/1215814742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6fad40 msgr2=0x55c4de6f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b0b25640 1 --2- 172.21.15.37:0/1215814742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6fad40 0x55c4de6f4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b0b25640 1 -- 172.21.15.37:0/1215814742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de703550 msgr2=0x55c4de6fef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b0b25640 1 --2- 172.21.15.37:0/1215814742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de703550 0x55c4de6fef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b0b25640 1 -- 172.21.15.37:0/1215814742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4de6f9590 con 0x55c4de6f9fd0 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79b0b25640 1 --2- 172.21.15.37:0/1215814742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6f9fd0 0x55c4de6f47f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f79a40b6040 tx=0x7f79a4099d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5df677c64a612b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79a27fc640 1 -- 172.21.15.37:0/1215814742 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a409bf60 con 0x55c4de6f9fd0 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79a27fc640 1 -- 172.21.15.37:0/1215814742 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79a409b860 con 0x55c4de6f9fd0 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.264+0000 7f79a27fc640 1 -- 172.21.15.37:0/1215814742 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a40a3dd0 con 0x55c4de6f9fd0 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.265+0000 7f79b596b740 1 -- 172.21.15.37:0/1215814742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6f9fd0 msgr2=0x55c4de6f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.265+0000 7f79b596b740 1 --2- 172.21.15.37:0/1215814742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6f9fd0 0x55c4de6f47f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f79a40b6040 tx=0x7f79a4099d70 comp rx=0 tx=0).stop 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.265+0000 7f79b596b740 1 -- 172.21.15.37:0/1215814742 shutdown_connections 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.265+0000 7f79b596b740 1 --2- 172.21.15.37:0/1215814742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de703550 0x55c4de6fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.265+0000 7f79b596b740 1 --2- 172.21.15.37:0/1215814742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de6fad40 0x55c4de6f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.265+0000 7f79b596b740 1 --2- 172.21.15.37:0/1215814742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6f9fd0 0x55c4de6f47f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.265+0000 7f79b596b740 1 -- 172.21.15.37:0/1215814742 >> 172.21.15.37:0/1215814742 conn(0x55c4de6c15b0 msgr2=0x55c4de6fb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.265+0000 7f79b596b740 1 -- 172.21.15.37:0/1215814742 shutdown_connections 2024-03-26T06:30:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.265+0000 7f79b596b740 1 -- 172.21.15.37:0/1215814742 wait complete. 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.265+0000 7f79b596b740 1 -- start start 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.265+0000 7f79b596b740 1 Processor -- start 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79b596b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 0x55c4de70a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79b596b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6fad40 0x55c4de70a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79b596b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de703550 0x55c4de708a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79b596b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1adf80 con 0x55c4de6fad40 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79b596b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1ad920 con 0x55c4de6f9fd0 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79b596b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1aaf00 con 0x55c4de703550 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79b596b740 10 client.0 _ll_register_callbacks cb 0x7f7994011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6fad40 0x55c4de70a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6fad40 0x55c4de70a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53348/0 (socket says 172.21.15.37:53348) 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79abfff640 1 -- 172.21.15.37:0/2651701641 learned_addr learned my addr 172.21.15.37:0/2651701641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79b0b25640 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 0x55c4de70a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79b1326640 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de703550 0x55c4de708a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79abfff640 1 -- 172.21.15.37:0/2651701641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de703550 msgr2=0x55c4de708a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79abfff640 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de703550 0x55c4de708a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79abfff640 1 -- 172.21.15.37:0/2651701641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 msgr2=0x55c4de70a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79abfff640 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 0x55c4de70a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79abfff640 1 -- 172.21.15.37:0/2651701641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4de6f92a0 con 0x55c4de6fad40 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f79abfff640 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6fad40 0x55c4de70a8c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f799c004af0 tx=0x7f799c032d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8a41e39faf273f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.266+0000 7f798bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f798bfff640 1 -- 172.21.15.37:0/2651701641 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c039040 con 0x55c4de6fad40 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f798bfff640 1 -- 172.21.15.37:0/2651701641 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f799c032790 con 0x55c4de6fad40 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f79b596b740 1 -- 172.21.15.37:0/2651701641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79a409b860 con 0x55c4de6fad40 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f798bfff640 1 -- 172.21.15.37:0/2651701641 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f799c056050 con 0x55c4de6fad40 2024-03-26T06:30:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f79b596b740 10 client.16071 fetch_fsmap learned FSMap version 40 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f79b596b740 1 -- 172.21.15.37:0/2651701641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c4de6f5e80 con 0x55c4de6fad40 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f798bfff640 1 -- 172.21.15.37:0/2651701641 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79a409b860 con 0x55c4de6fad40 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f798bfff640 1 -- 172.21.15.37:0/2651701641 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f799c034c30 con 0x55c4de6fad40 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f79b596b740 10 client.16071 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f79b596b740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f79b596b740 20 client.16071 populate_metadata read hostname 'smithi037' 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f79b596b740 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c4de76b010 0x55c4de76d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f79b596b740 4 client.16071 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f79b596b740 1 -- 172.21.15.37:0/2651701641 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c4de665e80 con 0x55c4de76b010 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.267+0000 7f79b0b25640 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c4de76b010 0x55c4de76d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.268+0000 7f79b0b25640 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c4de76b010 0x55c4de76d430 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca8e6559eb09ba47 server_cookie=1db6a0428accfc80 in_seq=0 out_seq=0 2024-03-26T06:30:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.268+0000 7f798bfff640 10 client.16071 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f798bfff640 1 -- 172.21.15.37:0/2651701641 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c4de665e80 con 0x55c4de76b010 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f798bfff640 10 client.16071 handle_command_reply: tid=0 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 1 client.16071 shutdown 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 1 -- 172.21.15.37:0/2651701641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6fad40 msgr2=0x55c4de70a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6fad40 0x55c4de70a8c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f799c004af0 tx=0x7f799c032d80 comp rx=0 tx=0).stop 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 1 -- 172.21.15.37:0/2651701641 shutdown_connections 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c4de76b010 0x55c4de76d430 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4de703550 0x55c4de708a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f798bfff640 0 client.16071 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4de6fad40 0x55c4de70a8c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 1 --2- 172.21.15.37:0/2651701641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4de6f9fd0 0x55c4de70a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 1 -- 172.21.15.37:0/2651701641 >> 172.21.15.37:0/2651701641 conn(0x55c4de6c15b0 msgr2=0x55c4de6fb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 1 -- 172.21.15.37:0/2651701641 shutdown_connections 2024-03-26T06:30:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 1 -- 172.21.15.37:0/2651701641 wait complete. 2024-03-26T06:30:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.297+0000 7f79b596b740 20 client.16071 trim_cache size 0 max 16384 2024-03-26T06:30:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:30:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:30:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:30:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:30:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 -- 172.21.15.37:0/331887117 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f79980a2a10 msgr2=0x7f79980a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f79980a2a10 0x7f79980a4ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7994009fd0 tx=0x7f7994009b80 comp rx=0 tx=0).stop 2024-03-26T06:30:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 -- 172.21.15.37:0/331887117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79ac101550 msgr2=0x7f79ac19d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79ac101550 0x7f79ac19d7e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f79a400e9f0 tx=0x7f79a400edf0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 -- 172.21.15.37:0/331887117 shutdown_connections 2024-03-26T06:30:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f79980a2a10 0x7f79980a4ed0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79ac197990 0x7f79ac197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79ac106620 0x7f79ac19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 --2- 172.21.15.37:0/331887117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79ac101550 0x7f79ac19d7e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 -- 172.21.15.37:0/331887117 >> 172.21.15.37:0/331887117 conn(0x7f79ac0fcdb0 msgr2=0x7f79ac0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 -- 172.21.15.37:0/331887117 shutdown_connections 2024-03-26T06:30:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:02.298+0000 7f79b2d89640 1 -- 172.21.15.37:0/331887117 wait complete. 2024-03-26T06:30:02.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:01 smithi066 ceph-mon[36504]: pgmap v214: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T06:30:02.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:01 smithi149 ceph-mon[36681]: pgmap v214: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T06:30:02.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:01 smithi037 ceph-mon[31974]: pgmap v214: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T06:30:02.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241675 4.14 MB/sec execute 217 sec latency 17.825 ms 2024-03-26T06:30:03.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 242943 4.14 MB/sec execute 218 sec latency 16.580 ms 2024-03-26T06:30:04.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:04 smithi037 ceph-mon[31974]: pgmap v215: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 652 KiB/s wr, 43 op/s 2024-03-26T06:30:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:04 smithi066 ceph-mon[36504]: pgmap v215: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 652 KiB/s wr, 43 op/s 2024-03-26T06:30:04.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:04 smithi149 ceph-mon[36681]: pgmap v215: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 652 KiB/s wr, 43 op/s 2024-03-26T06:30:04.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 245000 4.19 MB/sec execute 219 sec latency 15.721 ms 2024-03-26T06:30:05.509 INFO:tasks.workunit.client.0.smithi037.stdout: 1 245577 4.17 MB/sec execute 220 sec latency 251.025 ms 2024-03-26T06:30:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:06 smithi066 ceph-mon[36504]: pgmap v216: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-03-26T06:30:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:06 smithi066 ceph-mon[36504]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T06:30:06.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:06 smithi149 ceph-mon[36681]: pgmap v216: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-03-26T06:30:06.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:06 smithi149 ceph-mon[36681]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T06:30:06.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:06 smithi037 ceph-mon[31974]: pgmap v216: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-03-26T06:30:06.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:06 smithi037 ceph-mon[31974]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T06:30:06.508 INFO:tasks.workunit.client.0.smithi037.stdout: 1 245966 4.16 MB/sec execute 221 sec latency 1049.371 ms 2024-03-26T06:30:07.509 INFO:tasks.workunit.client.0.smithi037.stdout: 1 248409 4.21 MB/sec execute 222 sec latency 15.700 ms 2024-03-26T06:30:08.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:08 smithi066 ceph-mon[36504]: pgmap v218: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-03-26T06:30:08.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:08 smithi149 ceph-mon[36681]: pgmap v218: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-03-26T06:30:08.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:08 smithi037 ceph-mon[31974]: pgmap v218: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-03-26T06:30:08.509 INFO:tasks.workunit.client.0.smithi037.stdout: 1 249052 4.20 MB/sec execute 223 sec latency 277.975 ms 2024-03-26T06:30:09.509 INFO:tasks.workunit.client.0.smithi037.stdout: 1 249052 4.18 MB/sec execute 224 sec latency 1278.085 ms 2024-03-26T06:30:10.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:10 smithi066 ceph-mon[36504]: pgmap v219: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-03-26T06:30:10.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:10 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:30:10.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:10 smithi149 ceph-mon[36681]: pgmap v219: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-03-26T06:30:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:10 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:30:10.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:10 smithi037 ceph-mon[31974]: pgmap v219: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-03-26T06:30:10.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:10 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:30:10.509 INFO:tasks.workunit.client.0.smithi037.stdout: 1 249052 4.16 MB/sec execute 225 sec latency 2278.146 ms 2024-03-26T06:30:11.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:11 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:30:11.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:11 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:30:11.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:11 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:30:11.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:11 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:30:11.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:11 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:30:11.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:11 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:30:11.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:11 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:30:11.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:11 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:30:11.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:11 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:30:11.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 249397 4.15 MB/sec execute 226 sec latency 3077.610 ms 2024-03-26T06:30:12.509 INFO:tasks.workunit.client.0.smithi037.stdout: 1 251818 4.20 MB/sec execute 227 sec latency 16.672 ms 2024-03-26T06:30:12.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:12 smithi066 ceph-mon[36504]: pgmap v220: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-03-26T06:30:12.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:12 smithi149 ceph-mon[36681]: pgmap v220: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-03-26T06:30:12.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:12 smithi037 ceph-mon[31974]: pgmap v220: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-03-26T06:30:13.509 INFO:tasks.workunit.client.0.smithi037.stdout: 1 252670 4.19 MB/sec execute 228 sec latency 213.517 ms 2024-03-26T06:30:14.509 INFO:tasks.workunit.client.0.smithi037.stdout: 1 252670 4.17 MB/sec execute 229 sec latency 1213.639 ms 2024-03-26T06:30:14.848 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:14 smithi066 ceph-mon[36504]: pgmap v221: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-03-26T06:30:14.910 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:14 smithi149 ceph-mon[36681]: pgmap v221: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-03-26T06:30:14.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:14 smithi037 ceph-mon[31974]: pgmap v221: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-03-26T06:30:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.396+0000 7fac5953b640 1 -- 172.21.15.37:0/500296894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac54101ec0 msgr2=0x7fac54105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.396+0000 7fac5953b640 1 --2- 172.21.15.37:0/500296894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac54101ec0 0x7fac54105f00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fac48009900 tx=0x7fac4802bc30 comp rx=0 tx=0).stop 2024-03-26T06:30:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.396+0000 7fac5953b640 1 -- 172.21.15.37:0/500296894 shutdown_connections 2024-03-26T06:30:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.396+0000 7fac5953b640 1 --2- 172.21.15.37:0/500296894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac541065e0 0x7fac5410e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.396+0000 7fac5953b640 1 --2- 172.21.15.37:0/500296894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac54101ec0 0x7fac54105f00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.396+0000 7fac5953b640 1 --2- 172.21.15.37:0/500296894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac54101510 0x7fac541018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.396+0000 7fac5953b640 1 -- 172.21.15.37:0/500296894 >> 172.21.15.37:0/500296894 conn(0x7fac540fcdb0 msgr2=0x7fac540ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.396+0000 7fac5953b640 1 -- 172.21.15.37:0/500296894 shutdown_connections 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.396+0000 7fac5953b640 1 -- 172.21.15.37:0/500296894 wait complete. 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.396+0000 7fac5953b640 1 Processor -- start 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac5953b640 1 -- start start 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac5953b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac54101510 0x7fac540735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac5953b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541065e0 0x7fac54073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac5953b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac54074070 0x7fac541a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac5953b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac54111670 con 0x7fac541065e0 2024-03-26T06:30:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac5953b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac541114f0 con 0x7fac54101510 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac5953b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac541117f0 con 0x7fac54074070 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541065e0 0x7fac54073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541065e0 0x7fac54073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34228/0 (socket says 172.21.15.37:34228) 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac527fc640 1 -- 172.21.15.37:0/489607306 learned_addr learned my addr 172.21.15.37:0/489607306 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac52ffd640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac54101510 0x7fac540735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac537fe640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac54074070 0x7fac541a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac527fc640 1 -- 172.21.15.37:0/489607306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac54074070 msgr2=0x7fac541a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac527fc640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac54074070 0x7fac541a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac527fc640 1 -- 172.21.15.37:0/489607306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac54101510 msgr2=0x7fac540735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac527fc640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac54101510 0x7fac540735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.397+0000 7fac527fc640 1 -- 172.21.15.37:0/489607306 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac541a6420 con 0x7fac541065e0 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.398+0000 7fac527fc640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541065e0 0x7fac54073b30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fac48036040 tx=0x7fac4802ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.398+0000 7fac3bfff640 1 -- 172.21.15.37:0/489607306 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48041070 con 0x7fac541065e0 2024-03-26T06:30:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.398+0000 7fac3bfff640 1 -- 172.21.15.37:0/489607306 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac480054e0 con 0x7fac541065e0 2024-03-26T06:30:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.398+0000 7fac5953b640 1 -- 172.21.15.37:0/489607306 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac541a66b0 con 0x7fac541065e0 2024-03-26T06:30:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.398+0000 7fac3bfff640 1 -- 172.21.15.37:0/489607306 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48034440 con 0x7fac541065e0 2024-03-26T06:30:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.398+0000 7fac5953b640 1 -- 172.21.15.37:0/489607306 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac541a6ba0 con 0x7fac541065e0 2024-03-26T06:30:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.400+0000 7fac3bfff640 1 -- 172.21.15.37:0/489607306 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fac48044070 con 0x7fac541065e0 2024-03-26T06:30:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.400+0000 7fac5c1fb740 1 Processor -- start 2024-03-26T06:30:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.400+0000 7fac5c1fb740 1 -- start start 2024-03-26T06:30:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef6d40 0x560a5bef5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac3bfff640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac3c0a2a10 0x7fac3c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5bef6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5beff550 0x560a5bf01940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5be5a5f0 con 0x560a5bef5fd0 2024-03-26T06:30:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5be03930 con 0x560a5beff550 2024-03-26T06:30:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5be56690 con 0x560a5bef6d40 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5bef6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac52ffd640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac3c0a2a10 0x7fac3c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5bef6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34232/0 (socket says 172.21.15.37:34232) 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac527fc640 1 -- 172.21.15.37:0/580368578 learned_addr learned my addr 172.21.15.37:0/580368578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac537fe640 1 --2- 172.21.15.37:0/580368578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef6d40 0x560a5bef5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac3bfff640 1 -- 172.21.15.37:0/489607306 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fac480f0990 con 0x7fac541065e0 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac537fe640 1 --2- 172.21.15.37:0/580368578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5beff550 0x560a5bf01940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac527fc640 1 -- 172.21.15.37:0/580368578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef6d40 msgr2=0x560a5bef5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac527fc640 1 --2- 172.21.15.37:0/580368578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef6d40 0x560a5bef5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac527fc640 1 -- 172.21.15.37:0/580368578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5beff550 msgr2=0x560a5bf01940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac527fc640 1 --2- 172.21.15.37:0/580368578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5beff550 0x560a5bf01940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac527fc640 1 -- 172.21.15.37:0/580368578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a5bb6f210 con 0x560a5bef5fd0 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac537fe640 1 --2- 172.21.15.37:0/580368578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5beff550 0x560a5bf01940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:30:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac52ffd640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac3c0a2a10 0x7fac3c0a4ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac4400bbe0 tx=0x7fac4400b390 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac527fc640 1 --2- 172.21.15.37:0/580368578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5bef6450 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fac480ee1f0 tx=0x7fac48003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d1a349932b90b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.401+0000 7fac397fa640 1 -- 172.21.15.37:0/580368578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4804b070 con 0x560a5bef5fd0 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.402+0000 7fac397fa640 1 -- 172.21.15.37:0/580368578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac48005690 con 0x560a5bef5fd0 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.402+0000 7fac397fa640 1 -- 172.21.15.37:0/580368578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac480bd1e0 con 0x560a5bef5fd0 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.402+0000 7fac5c1fb740 1 -- 172.21.15.37:0/580368578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 msgr2=0x560a5bef6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.402+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/580368578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5bef6450 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fac480ee1f0 tx=0x7fac48003990 comp rx=0 tx=0).stop 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.402+0000 7fac5c1fb740 1 -- 172.21.15.37:0/580368578 shutdown_connections 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.402+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/580368578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5beff550 0x560a5bf01940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.402+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/580368578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5bef6450 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.402+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/580368578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef6d40 0x560a5bef5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.402+0000 7fac5c1fb740 1 -- 172.21.15.37:0/580368578 >> 172.21.15.37:0/580368578 conn(0x560a5bebd5b0 msgr2=0x560a5bef0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.402+0000 7fac5c1fb740 1 -- 172.21.15.37:0/580368578 shutdown_connections 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.402+0000 7fac5c1fb740 1 -- 172.21.15.37:0/580368578 wait complete. 2024-03-26T06:30:15.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac5c1fb740 1 -- start start 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac5c1fb740 1 Processor -- start 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5befc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5befcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5beff550 0x560a5befd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5bf02040 con 0x560a5bef5fd0 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5be16d90 con 0x560a5bef6d40 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5bf021c0 con 0x560a5beff550 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac5c1fb740 10 client.0 _ll_register_callbacks cb 0x560a5bb6ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac52ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5befc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac52ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5befc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34238/0 (socket says 172.21.15.37:34238) 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac52ffd640 1 -- 172.21.15.37:0/1226460938 learned_addr learned my addr 172.21.15.37:0/1226460938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac537fe640 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5beff550 0x560a5befd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac527fc640 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5befcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac52ffd640 1 -- 172.21.15.37:0/1226460938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5beff550 msgr2=0x560a5befd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac52ffd640 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5beff550 0x560a5befd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac52ffd640 1 -- 172.21.15.37:0/1226460938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 msgr2=0x560a5befcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac52ffd640 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5befcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.403+0000 7fac52ffd640 1 -- 172.21.15.37:0/1226460938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a5bc3b620 con 0x560a5bef5fd0 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac52ffd640 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5befc5e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fac440116c0 tx=0x7fac44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f599f1c2c545805 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac36ff5640 1 -- 172.21.15.37:0/1226460938 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4401e8f0 con 0x560a5bef5fd0 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac36ff5640 1 -- 172.21.15.37:0/1226460938 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fac4401ea90 con 0x560a5bef5fd0 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac5c1fb740 1 -- 172.21.15.37:0/1226460938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a5be67b30 con 0x560a5bef5fd0 2024-03-26T06:30:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac36ff5640 1 -- 172.21.15.37:0/1226460938 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac4401ec90 con 0x560a5bef5fd0 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac5c1fb740 10 client.16083 fetch_fsmap learned FSMap version 40 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac5c1fb740 1 -- 172.21.15.37:0/1226460938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560a5bf0e360 con 0x560a5bef5fd0 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac36ff5640 1 -- 172.21.15.37:0/1226460938 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560a5be67b30 con 0x560a5bef5fd0 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac36ff5640 1 -- 172.21.15.37:0/1226460938 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fac4401c500 con 0x560a5bef5fd0 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac5c1fb740 10 client.16083 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac5c1fb740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac5c1fb740 20 client.16083 populate_metadata read hostname 'smithi037' 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560a5bf0f870 0x560a5bf11c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac5c1fb740 4 client.16083 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.404+0000 7fac5c1fb740 1 -- 172.21.15.37:0/1226460938 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a5be61e80 con 0x560a5bf0f870 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.405+0000 7fac527fc640 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560a5bf0f870 0x560a5bf11c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.405+0000 7fac36ff5640 10 client.16083 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.405+0000 7fac527fc640 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560a5bf0f870 0x560a5bf11c90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c7642543e21cf5e server_cookie=5a2f4ea046ea3f00 in_seq=0 out_seq=0 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac36ff5640 1 -- 172.21.15.37:0/1226460938 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a5be61e80 con 0x560a5bf0f870 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac36ff5640 10 client.16083 handle_command_reply: tid=0 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 1 client.16083 shutdown 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 1 -- 172.21.15.37:0/1226460938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 msgr2=0x560a5befc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5befc5e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fac440116c0 tx=0x7fac44004730 comp rx=0 tx=0).stop 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 1 -- 172.21.15.37:0/1226460938 shutdown_connections 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560a5bf0f870 0x560a5bf11c90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5beff550 0x560a5befd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5befcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac36ff5640 0 client.16083 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/1226460938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5bef5fd0 0x560a5befc5e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 1 -- 172.21.15.37:0/1226460938 >> 172.21.15.37:0/1226460938 conn(0x560a5bebd5b0 msgr2=0x560a5bef2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 1 -- 172.21.15.37:0/1226460938 shutdown_connections 2024-03-26T06:30:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 1 -- 172.21.15.37:0/1226460938 wait complete. 2024-03-26T06:30:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.406+0000 7fac5c1fb740 20 client.16083 trim_cache size 0 max 16384 2024-03-26T06:30:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.419+0000 7fac5c1fb740 1 Processor -- start 2024-03-26T06:30:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.419+0000 7fac5c1fb740 1 -- start start 2024-03-26T06:30:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef5fd0 0x560a5befed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5bf063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bf06930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac541aee70 con 0x560a5beff550 2024-03-26T06:30:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac541b08b0 con 0x560a5bef6d40 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac541af4d0 con 0x560a5bef5fd0 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bf06930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bf06930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34244/0 (socket says 172.21.15.37:34244) 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac537fe640 1 -- 172.21.15.37:0/3518948655 learned_addr learned my addr 172.21.15.37:0/3518948655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac527fc640 1 --2- 172.21.15.37:0/3518948655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5bf063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac52ffd640 1 --2- 172.21.15.37:0/3518948655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef5fd0 0x560a5befed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac537fe640 1 -- 172.21.15.37:0/3518948655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef5fd0 msgr2=0x560a5befed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac537fe640 1 --2- 172.21.15.37:0/3518948655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef5fd0 0x560a5befed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac537fe640 1 -- 172.21.15.37:0/3518948655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 msgr2=0x560a5bf063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac537fe640 1 --2- 172.21.15.37:0/3518948655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5bf063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac537fe640 1 -- 172.21.15.37:0/3518948655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a5bef52a0 con 0x560a5beff550 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac537fe640 1 --2- 172.21.15.37:0/3518948655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bf06930 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fac4c003ce0 tx=0x7fac4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c67f8572a38db9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac38ff9640 1 -- 172.21.15.37:0/3518948655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c022070 con 0x560a5beff550 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac38ff9640 1 -- 172.21.15.37:0/3518948655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac4c010b60 con 0x560a5beff550 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.420+0000 7fac38ff9640 1 -- 172.21.15.37:0/3518948655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c012b70 con 0x560a5beff550 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.421+0000 7fac5c1fb740 1 -- 172.21.15.37:0/3518948655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 msgr2=0x560a5bf06930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.421+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/3518948655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bf06930 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fac4c003ce0 tx=0x7fac4c004a40 comp rx=0 tx=0).stop 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.421+0000 7fac5c1fb740 1 -- 172.21.15.37:0/3518948655 shutdown_connections 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.421+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/3518948655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bf06930 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.421+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/3518948655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5bf063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.421+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/3518948655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef5fd0 0x560a5befed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.421+0000 7fac5c1fb740 1 -- 172.21.15.37:0/3518948655 >> 172.21.15.37:0/3518948655 conn(0x560a5bebd5b0 msgr2=0x560a5bef07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.421+0000 7fac5c1fb740 1 -- 172.21.15.37:0/3518948655 shutdown_connections 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.421+0000 7fac5c1fb740 1 -- 172.21.15.37:0/3518948655 wait complete. 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.421+0000 7fac5c1fb740 1 -- start start 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.421+0000 7fac5c1fb740 1 Processor -- start 2024-03-26T06:30:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef5fd0 0x560a5bf06660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5bef8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac5c1fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bef9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac541b0190 con 0x560a5beff550 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac541afb30 con 0x560a5bef6d40 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac5c1fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac541ad130 con 0x560a5bef5fd0 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac5c1fb740 10 client.0 _ll_register_callbacks cb 0x7fac44010c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bef9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bef9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34248/0 (socket says 172.21.15.37:34248) 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac537fe640 1 -- 172.21.15.37:0/4256823391 learned_addr learned my addr 172.21.15.37:0/4256823391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac527fc640 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5bef8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac52ffd640 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef5fd0 0x560a5bf06660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac537fe640 1 -- 172.21.15.37:0/4256823391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef5fd0 msgr2=0x560a5bf06660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac537fe640 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef5fd0 0x560a5bf06660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac537fe640 1 -- 172.21.15.37:0/4256823391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 msgr2=0x560a5bef8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac537fe640 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5bef8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac537fe640 1 -- 172.21.15.37:0/4256823391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a5bef9a30 con 0x560a5beff550 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac527fc640 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5bef8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac537fe640 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bef9370 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fac4c003f90 tx=0x7fac4c012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05a56a27475da6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac36ff5640 1 -- 172.21.15.37:0/4256823391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c02c020 con 0x560a5beff550 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.422+0000 7fac36ff5640 1 -- 172.21.15.37:0/4256823391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fac4c010680 con 0x560a5beff550 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac5c1fb740 1 -- 172.21.15.37:0/4256823391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac4c010b60 con 0x560a5beff550 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac36ff5640 1 -- 172.21.15.37:0/4256823391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac4c037050 con 0x560a5beff550 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac5c1fb740 10 client.16089 fetch_fsmap learned FSMap version 40 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac5c1fb740 1 -- 172.21.15.37:0/4256823391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560a5bf66c90 con 0x560a5beff550 2024-03-26T06:30:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac36ff5640 1 -- 172.21.15.37:0/4256823391 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac4c010b60 con 0x560a5beff550 2024-03-26T06:30:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac36ff5640 1 -- 172.21.15.37:0/4256823391 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fac4c00f040 con 0x560a5beff550 2024-03-26T06:30:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac5c1fb740 10 client.16089 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:30:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac5c1fb740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:30:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac5c1fb740 20 client.16089 populate_metadata read hostname 'smithi037' 2024-03-26T06:30:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560a5bf673f0 0x560a5bf69810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac5c1fb740 4 client.16089 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:30:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac5c1fb740 1 -- 172.21.15.37:0/4256823391 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a5bde0710 con 0x560a5bf673f0 2024-03-26T06:30:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.423+0000 7fac52ffd640 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560a5bf673f0 0x560a5bf69810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.424+0000 7fac52ffd640 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560a5bf673f0 0x560a5bf69810 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2136d4f2a906dcf server_cookie=653d3a2098731efc in_seq=0 out_seq=0 2024-03-26T06:30:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.424+0000 7fac36ff5640 10 client.16089 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac36ff5640 1 -- 172.21.15.37:0/4256823391 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560a5bde0710 con 0x560a5bf673f0 2024-03-26T06:30:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac36ff5640 10 client.16089 handle_command_reply: tid=0 2024-03-26T06:30:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 1 client.16089 shutdown 2024-03-26T06:30:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 1 -- 172.21.15.37:0/4256823391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 msgr2=0x560a5bef9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bef9370 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fac4c003f90 tx=0x7fac4c012670 comp rx=0 tx=0).stop 2024-03-26T06:30:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 1 -- 172.21.15.37:0/4256823391 shutdown_connections 2024-03-26T06:30:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560a5bf673f0 0x560a5bf69810 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a5beff550 0x560a5bef9370 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a5bef6d40 0x560a5bef8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 1 --2- 172.21.15.37:0/4256823391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a5bef5fd0 0x560a5bf06660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac36ff5640 0 client.16089 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 1 -- 172.21.15.37:0/4256823391 >> 172.21.15.37:0/4256823391 conn(0x560a5bebd5b0 msgr2=0x560a5beef5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 1 -- 172.21.15.37:0/4256823391 shutdown_connections 2024-03-26T06:30:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 1 -- 172.21.15.37:0/4256823391 wait complete. 2024-03-26T06:30:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.453+0000 7fac5c1fb740 20 client.16089 trim_cache size 0 max 16384 2024-03-26T06:30:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:30:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:30:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:30:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:30:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 -- 172.21.15.37:0/489607306 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac3c0a2a10 msgr2=0x7fac3c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac3c0a2a10 0x7fac3c0a4ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac4400bbe0 tx=0x7fac4400b390 comp rx=0 tx=0).stop 2024-03-26T06:30:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 -- 172.21.15.37:0/489607306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541065e0 msgr2=0x7fac54073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541065e0 0x7fac54073b30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fac48036040 tx=0x7fac4802ce20 comp rx=0 tx=0).stop 2024-03-26T06:30:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 -- 172.21.15.37:0/489607306 shutdown_connections 2024-03-26T06:30:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac3c0a2a10 0x7fac3c0a4ed0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac54074070 0x7fac541a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541065e0 0x7fac54073b30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 --2- 172.21.15.37:0/489607306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac54101510 0x7fac540735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 -- 172.21.15.37:0/489607306 >> 172.21.15.37:0/489607306 conn(0x7fac540fcdb0 msgr2=0x7fac540ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 -- 172.21.15.37:0/489607306 shutdown_connections 2024-03-26T06:30:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:15.454+0000 7fac5953b640 1 -- 172.21.15.37:0/489607306 wait complete. 2024-03-26T06:30:15.509 INFO:tasks.workunit.client.0.smithi037.stdout: 1 252670 4.16 MB/sec execute 230 sec latency 2213.736 ms 2024-03-26T06:30:15.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:15 smithi037 ceph-mon[31974]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T06:30:16.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:15 smithi066 ceph-mon[36504]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T06:30:16.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:15 smithi149 ceph-mon[36681]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T06:30:16.509 INFO:tasks.workunit.client.0.smithi037.stdout: 1 253034 4.14 MB/sec execute 231 sec latency 3012.650 ms 2024-03-26T06:30:16.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:16 smithi037 ceph-mon[31974]: pgmap v223: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-03-26T06:30:17.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:16 smithi066 ceph-mon[36504]: pgmap v223: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-03-26T06:30:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:16 smithi149 ceph-mon[36681]: pgmap v223: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-03-26T06:30:17.509 INFO:tasks.workunit.client.0.smithi037.stdout: 1 255484 4.19 MB/sec execute 232 sec latency 14.601 ms 2024-03-26T06:30:18.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 256087 4.18 MB/sec execute 233 sec latency 371.401 ms 2024-03-26T06:30:18.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:18 smithi037 ceph-mon[31974]: pgmap v224: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-03-26T06:30:19.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:18 smithi066 ceph-mon[36504]: pgmap v224: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-03-26T06:30:19.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:18 smithi149 ceph-mon[36681]: pgmap v224: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-03-26T06:30:19.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 256087 4.16 MB/sec execute 234 sec latency 1371.525 ms 2024-03-26T06:30:20.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 256087 4.15 MB/sec execute 235 sec latency 2371.599 ms 2024-03-26T06:30:21.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:20 smithi066 ceph-mon[36504]: pgmap v225: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-03-26T06:30:21.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:20 smithi149 ceph-mon[36681]: pgmap v225: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-03-26T06:30:21.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:20 smithi037 ceph-mon[31974]: pgmap v225: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-03-26T06:30:21.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 256143 4.13 MB/sec execute 236 sec latency 3175.555 ms 2024-03-26T06:30:22.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 256143 4.11 MB/sec execute 237 sec latency 1158.526 ms 2024-03-26T06:30:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:22 smithi066 ceph-mon[36504]: pgmap v226: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-03-26T06:30:23.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:22 smithi149 ceph-mon[36681]: pgmap v226: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-03-26T06:30:23.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:22 smithi037 ceph-mon[31974]: pgmap v226: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-03-26T06:30:23.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 256143 4.10 MB/sec execute 238 sec latency 2158.643 ms 2024-03-26T06:30:24.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 256143 4.08 MB/sec execute 239 sec latency 3158.733 ms 2024-03-26T06:30:25.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:24 smithi066 ceph-mon[36504]: pgmap v227: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-03-26T06:30:25.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:24 smithi149 ceph-mon[36681]: pgmap v227: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-03-26T06:30:25.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:24 smithi037 ceph-mon[31974]: pgmap v227: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-03-26T06:30:25.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 256143 4.06 MB/sec execute 240 sec latency 4158.838 ms 2024-03-26T06:30:26.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 256493 4.05 MB/sec execute 241 sec latency 4956.560 ms 2024-03-26T06:30:27.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:26 smithi066 ceph-mon[36504]: pgmap v228: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-03-26T06:30:27.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:26 smithi149 ceph-mon[36681]: pgmap v228: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-03-26T06:30:27.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:26 smithi037 ceph-mon[31974]: pgmap v228: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-03-26T06:30:27.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 258949 4.10 MB/sec execute 242 sec latency 16.706 ms 2024-03-26T06:30:28.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 259760 4.09 MB/sec execute 243 sec latency 244.469 ms 2024-03-26T06:30:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.541+0000 7f034cbd4640 1 -- 172.21.15.37:0/4021241028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0348100440 msgr2=0x7f03481008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.541+0000 7f034cbd4640 1 --2- 172.21.15.37:0/4021241028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0348100440 0x7f03481008c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0330009900 tx=0x7f033002bc30 comp rx=0 tx=0).stop 2024-03-26T06:30:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.541+0000 7f034cbd4640 1 -- 172.21.15.37:0/4021241028 shutdown_connections 2024-03-26T06:30:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.541+0000 7f034cbd4640 1 --2- 172.21.15.37:0/4021241028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0348100e00 0x7f034810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.541+0000 7f034cbd4640 1 --2- 172.21.15.37:0/4021241028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0348100440 0x7f03481008c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.541+0000 7f034cbd4640 1 --2- 172.21.15.37:0/4021241028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0348105190 0x7f0348105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.541+0000 7f034cbd4640 1 -- 172.21.15.37:0/4021241028 >> 172.21.15.37:0/4021241028 conn(0x7f0348077040 msgr2=0x7f0348077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.541+0000 7f034cbd4640 1 -- 172.21.15.37:0/4021241028 shutdown_connections 2024-03-26T06:30:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.541+0000 7f034cbd4640 1 -- 172.21.15.37:0/4021241028 wait complete. 2024-03-26T06:30:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f034cbd4640 1 Processor -- start 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f034cbd4640 1 -- start start 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f034cbd4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0348100440 0x7f03481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f034cbd4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0348100e00 0x7f0348198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f034cbd4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0348105190 0x7f03481a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f034cbd4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0348110650 con 0x7f0348100e00 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f034cbd4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03481104d0 con 0x7f0348105190 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f034cbd4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03481107d0 con 0x7f0348100440 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f033ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0348100e00 0x7f0348198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f034659c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0348100440 0x7f03481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f033ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0348100e00 0x7f0348198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51832/0 (socket says 172.21.15.37:51832) 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f033ffff640 1 -- 172.21.15.37:0/636920849 learned_addr learned my addr 172.21.15.37:0/636920849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.542+0000 7f0346d9d640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0348105190 0x7f03481a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.543+0000 7f033ffff640 1 -- 172.21.15.37:0/636920849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0348100440 msgr2=0x7f03481989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.543+0000 7f033ffff640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0348100440 0x7f03481989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.543+0000 7f033ffff640 1 -- 172.21.15.37:0/636920849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0348105190 msgr2=0x7f03481a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.543+0000 7f033ffff640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0348105190 0x7f03481a3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.543+0000 7f033ffff640 1 -- 172.21.15.37:0/636920849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03481a3f70 con 0x7f0348100e00 2024-03-26T06:30:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.543+0000 7f033ffff640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0348100e00 0x7f0348198ef0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0330005940 tx=0x7f0330004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.543+0000 7f033f7fe640 1 -- 172.21.15.37:0/636920849 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0330041070 con 0x7f0348100e00 2024-03-26T06:30:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.543+0000 7f033f7fe640 1 -- 172.21.15.37:0/636920849 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03300054e0 con 0x7f0348100e00 2024-03-26T06:30:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.543+0000 7f034cbd4640 1 -- 172.21.15.37:0/636920849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03481a4140 con 0x7f0348100e00 2024-03-26T06:30:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.543+0000 7f033f7fe640 1 -- 172.21.15.37:0/636920849 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0330003d00 con 0x7f0348100e00 2024-03-26T06:30:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.543+0000 7f034cbd4640 1 -- 172.21.15.37:0/636920849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03481a45a0 con 0x7f0348100e00 2024-03-26T06:30:28.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.545+0000 7f033f7fe640 1 -- 172.21.15.37:0/636920849 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0330044070 con 0x7f0348100e00 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f034f7b8740 1 Processor -- start 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f033f7fe640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f03340a2a10 0x7f03340a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f034f7b8740 1 -- start start 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f0346d9d640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f03340a2a10 0x7f03340a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b89561d40 0x558b89560a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89561450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f033f7fe640 1 -- 172.21.15.37:0/636920849 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f03300f0990 con 0x7f0348100e00 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8956a550 0x558b8956c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f034f7b8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b894c55f0 con 0x558b8956a550 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f034f7b8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8946e930 con 0x558b89560fd0 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f034f7b8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b894c1690 con 0x558b89561d40 2024-03-26T06:30:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f033ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89561450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f0346d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8956a550 0x558b8956c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f033ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89561450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:46630/0 (socket says 172.21.15.37:46630) 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f033ffff640 1 -- 172.21.15.37:0/2207749985 learned_addr learned my addr 172.21.15.37:0/2207749985 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f034659c640 1 --2- 172.21.15.37:0/2207749985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b89561d40 0x558b89560a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f033ffff640 1 -- 172.21.15.37:0/2207749985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b89561d40 msgr2=0x558b89560a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.546+0000 7f033ffff640 1 --2- 172.21.15.37:0/2207749985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b89561d40 0x558b89560a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.547+0000 7f033ffff640 1 -- 172.21.15.37:0/2207749985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8956a550 msgr2=0x558b8956c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.547+0000 7f033ffff640 1 --2- 172.21.15.37:0/2207749985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8956a550 0x558b8956c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.547+0000 7f033ffff640 1 -- 172.21.15.37:0/2207749985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b891da210 con 0x558b89560fd0 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.547+0000 7f0346d9d640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f03340a2a10 0x7f03340a4ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0340005fd0 tx=0x7f0340013b60 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.547+0000 7f033ffff640 1 --2- 172.21.15.37:0/2207749985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89561450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f03300c5370 tx=0x7f03300bcf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=434bda86c4c14a01 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.547+0000 7f033cff9640 1 -- 172.21.15.37:0/2207749985 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033004b040 con 0x558b89560fd0 2024-03-26T06:30:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.547+0000 7f033cff9640 1 -- 172.21.15.37:0/2207749985 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0330005690 con 0x558b89560fd0 2024-03-26T06:30:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.547+0000 7f033cff9640 1 -- 172.21.15.37:0/2207749985 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300dd070 con 0x558b89560fd0 2024-03-26T06:30:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.548+0000 7f034f7b8740 1 -- 172.21.15.37:0/2207749985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 msgr2=0x558b89561450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.548+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2207749985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89561450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f03300c5370 tx=0x7f03300bcf80 comp rx=0 tx=0).stop 2024-03-26T06:30:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.548+0000 7f034f7b8740 1 -- 172.21.15.37:0/2207749985 shutdown_connections 2024-03-26T06:30:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.548+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2207749985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8956a550 0x558b8956c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.548+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2207749985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89561450 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.548+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2207749985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b89561d40 0x558b89560a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.548+0000 7f034f7b8740 1 -- 172.21.15.37:0/2207749985 >> 172.21.15.37:0/2207749985 conn(0x558b895285b0 msgr2=0x558b8955bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.548+0000 7f034f7b8740 1 -- 172.21.15.37:0/2207749985 shutdown_connections 2024-03-26T06:30:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.548+0000 7f034f7b8740 1 -- 172.21.15.37:0/2207749985 wait complete. 2024-03-26T06:30:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.548+0000 7f034f7b8740 1 -- start start 2024-03-26T06:30:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.548+0000 7f034f7b8740 1 Processor -- start 2024-03-26T06:30:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89567640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89567b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8956a550 0x558b895680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034f7b8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8956d040 con 0x558b89561d40 2024-03-26T06:30:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034f7b8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b89481d90 con 0x558b89560fd0 2024-03-26T06:30:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034f7b8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8956d1c0 con 0x558b8956a550 2024-03-26T06:30:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034f7b8740 10 client.0 _ll_register_callbacks cb 0x558b891d9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:30:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f033ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89567b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f033ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89567b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51850/0 (socket says 172.21.15.37:51850) 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89567640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f033ffff640 1 -- 172.21.15.37:0/2546811568 learned_addr learned my addr 172.21.15.37:0/2546811568 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f0346d9d640 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8956a550 0x558b895680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034659c640 1 -- 172.21.15.37:0/2546811568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8956a550 msgr2=0x558b895680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034659c640 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8956a550 0x558b895680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034659c640 1 -- 172.21.15.37:0/2546811568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 msgr2=0x558b89567b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034659c640 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89567b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034659c640 1 -- 172.21.15.37:0/2546811568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b892a6620 con 0x558b89560fd0 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f033ffff640 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89567b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034659c640 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89567640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0338012700 tx=0x7f0338012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65fa076d6c14df00 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f032dffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f032dffb640 1 -- 172.21.15.37:0/2546811568 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338004030 con 0x558b89560fd0 2024-03-26T06:30:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f034f7b8740 1 -- 172.21.15.37:0/2546811568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b894d2b30 con 0x558b89560fd0 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.549+0000 7f032dffb640 1 -- 172.21.15.37:0/2546811568 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f03380041d0 con 0x558b89560fd0 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f032dffb640 1 -- 172.21.15.37:0/2546811568 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03380043d0 con 0x558b89560fd0 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f034f7b8740 10 client.25655 fetch_fsmap learned FSMap version 40 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f034f7b8740 1 -- 172.21.15.37:0/2546811568 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558b89579330 con 0x558b89560fd0 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f032dffb640 1 -- 172.21.15.37:0/2546811568 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558b894d2b30 con 0x558b89560fd0 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f032dffb640 1 -- 172.21.15.37:0/2546811568 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0338003ad0 con 0x558b89560fd0 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f034f7b8740 10 client.25655 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f034f7b8740 10 client.25655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f034f7b8740 20 client.25655 populate_metadata read hostname 'smithi037' 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8957a7a0 0x558b8957cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f034f7b8740 4 client.25655 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f034f7b8740 1 -- 172.21.15.37:0/2546811568 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b894cce80 con 0x558b8957a7a0 2024-03-26T06:30:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f033ffff640 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8957a7a0 0x558b8957cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f033ffff640 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8957a7a0 0x558b8957cc00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=874a2a020ef180e5 server_cookie=2f79fd83a5fe4682 in_seq=0 out_seq=0 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.550+0000 7f032dffb640 10 client.25655 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f032dffb640 1 -- 172.21.15.37:0/2546811568 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b894cce80 con 0x558b8957a7a0 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f032dffb640 10 client.25655 handle_command_reply: tid=0 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 1 client.25655 shutdown 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 1 -- 172.21.15.37:0/2546811568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 msgr2=0x558b89567640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89567640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0338012700 tx=0x7f0338012bc0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 1 -- 172.21.15.37:0/2546811568 shutdown_connections 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8957a7a0 0x558b8957cc00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8956a550 0x558b895680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f032dffb640 0 client.25655 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89567b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2546811568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89567640 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 1 -- 172.21.15.37:0/2546811568 >> 172.21.15.37:0/2546811568 conn(0x558b895285b0 msgr2=0x558b8955da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 1 -- 172.21.15.37:0/2546811568 shutdown_connections 2024-03-26T06:30:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 1 -- 172.21.15.37:0/2546811568 wait complete. 2024-03-26T06:30:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.551+0000 7f034f7b8740 20 client.25655 trim_cache size 0 max 16384 2024-03-26T06:30:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f034f7b8740 1 Processor -- start 2024-03-26T06:30:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f034f7b8740 1 -- start start 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89569d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89571360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8956a550 0x558b895718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f034f7b8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03481acc40 con 0x558b89561d40 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f034f7b8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03481ae680 con 0x558b89560fd0 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f034f7b8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03481ad2a0 con 0x558b8956a550 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f033ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89571360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f033ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89571360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51860/0 (socket says 172.21.15.37:51860) 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f033ffff640 1 -- 172.21.15.37:0/2452129571 learned_addr learned my addr 172.21.15.37:0/2452129571 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f034659c640 1 --2- 172.21.15.37:0/2452129571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89569d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f0346d9d640 1 --2- 172.21.15.37:0/2452129571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8956a550 0x558b895718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f033ffff640 1 -- 172.21.15.37:0/2452129571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8956a550 msgr2=0x558b895718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f033ffff640 1 --2- 172.21.15.37:0/2452129571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8956a550 0x558b895718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f033ffff640 1 -- 172.21.15.37:0/2452129571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 msgr2=0x558b89569d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f033ffff640 1 --2- 172.21.15.37:0/2452129571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89569d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f033ffff640 1 -- 172.21.15.37:0/2452129571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b895602a0 con 0x558b89561d40 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f0346d9d640 1 --2- 172.21.15.37:0/2452129571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8956a550 0x558b895718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f033ffff640 1 --2- 172.21.15.37:0/2452129571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89571360 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f03300037a0 tx=0x7f03300c1180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d34c6f31318c1b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.565+0000 7f032ffff640 1 -- 172.21.15.37:0/2452129571 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033004b040 con 0x558b89561d40 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f032ffff640 1 -- 172.21.15.37:0/2452129571 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f033002ccd0 con 0x558b89561d40 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f032ffff640 1 -- 172.21.15.37:0/2452129571 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300b9820 con 0x558b89561d40 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f034f7b8740 1 -- 172.21.15.37:0/2452129571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 msgr2=0x558b89571360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2452129571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89571360 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f03300037a0 tx=0x7f03300c1180 comp rx=0 tx=0).stop 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f034f7b8740 1 -- 172.21.15.37:0/2452129571 shutdown_connections 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2452129571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8956a550 0x558b895718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2452129571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89561d40 0x558b89571360 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2452129571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b89560fd0 0x558b89569d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f034f7b8740 1 -- 172.21.15.37:0/2452129571 >> 172.21.15.37:0/2452129571 conn(0x558b895285b0 msgr2=0x558b8955b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f034f7b8740 1 -- 172.21.15.37:0/2452129571 shutdown_connections 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f034f7b8740 1 -- 172.21.15.37:0/2452129571 wait complete. 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f034f7b8740 1 -- start start 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.566+0000 7f034f7b8740 1 Processor -- start 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89560fd0 0x558b8956dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b89561d40 0x558b8956e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034f7b8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8956a550 0x558b8956e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034f7b8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03481adf60 con 0x558b89560fd0 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034f7b8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03481ad900 con 0x558b8956a550 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034f7b8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03481aaf00 con 0x558b89561d40 2024-03-26T06:30:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034f7b8740 10 client.0 _ll_register_callbacks cb 0x7f03300035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89560fd0 0x558b8956dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89560fd0 0x558b8956dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51876/0 (socket says 172.21.15.37:51876) 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034659c640 1 -- 172.21.15.37:0/2154721494 learned_addr learned my addr 172.21.15.37:0/2154721494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f0346d9d640 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8956a550 0x558b8956e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f033ffff640 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b89561d40 0x558b8956e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034659c640 1 -- 172.21.15.37:0/2154721494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b89561d40 msgr2=0x558b8956e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034659c640 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b89561d40 0x558b8956e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034659c640 1 -- 172.21.15.37:0/2154721494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8956a550 msgr2=0x558b8956e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034659c640 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8956a550 0x558b8956e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034659c640 1 -- 172.21.15.37:0/2154721494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b895d0d20 con 0x558b89560fd0 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034659c640 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89560fd0 0x558b8956dd10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0338000c00 tx=0x7f033800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c633da05d394041 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f032dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:30:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f032dffb640 1 -- 172.21.15.37:0/2154721494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338003fa0 con 0x558b89560fd0 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f032dffb640 1 -- 172.21.15.37:0/2154721494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0338004140 con 0x558b89560fd0 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.567+0000 7f034f7b8740 1 -- 172.21.15.37:0/2154721494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b8944b710 con 0x558b89560fd0 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f032dffb640 1 -- 172.21.15.37:0/2154721494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0338025950 con 0x558b89560fd0 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f034f7b8740 10 client.16107 fetch_fsmap learned FSMap version 40 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f034f7b8740 1 -- 172.21.15.37:0/2154721494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558b895d2000 con 0x558b89560fd0 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f032dffb640 1 -- 172.21.15.37:0/2154721494 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558b8944b710 con 0x558b89560fd0 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f032dffb640 1 -- 172.21.15.37:0/2154721494 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0338003610 con 0x558b89560fd0 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f034f7b8740 10 client.16107 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f034f7b8740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f034f7b8740 20 client.16107 populate_metadata read hostname 'smithi037' 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b895d27d0 0x558b895d4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f034f7b8740 4 client.16107 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f034f7b8740 1 -- 172.21.15.37:0/2154721494 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558b89497590 con 0x558b895d27d0 2024-03-26T06:30:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f033ffff640 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b895d27d0 0x558b895d4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f033ffff640 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b895d27d0 0x558b895d4bf0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad77d5ec773a0b6 server_cookie=425409f6ad654f63 in_seq=0 out_seq=0 2024-03-26T06:30:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.568+0000 7f032dffb640 10 client.16107 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f032dffb640 1 -- 172.21.15.37:0/2154721494 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558b89497590 con 0x558b895d27d0 2024-03-26T06:30:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f032dffb640 10 client.16107 handle_command_reply: tid=0 2024-03-26T06:30:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f034f7b8740 1 client.16107 shutdown 2024-03-26T06:30:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f034f7b8740 1 -- 172.21.15.37:0/2154721494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89560fd0 msgr2=0x558b8956dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89560fd0 0x558b8956dd10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0338000c00 tx=0x7f033800f790 comp rx=0 tx=0).stop 2024-03-26T06:30:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f034f7b8740 1 -- 172.21.15.37:0/2154721494 shutdown_connections 2024-03-26T06:30:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b895d27d0 0x558b895d4bf0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8956a550 0x558b8956e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f032dffb640 0 client.16107 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b89561d40 0x558b8956e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f034f7b8740 1 --2- 172.21.15.37:0/2154721494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b89560fd0 0x558b8956dd10 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f034f7b8740 1 -- 172.21.15.37:0/2154721494 >> 172.21.15.37:0/2154721494 conn(0x558b895285b0 msgr2=0x558b89565b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.592+0000 7f034f7b8740 1 -- 172.21.15.37:0/2154721494 shutdown_connections 2024-03-26T06:30:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034f7b8740 1 -- 172.21.15.37:0/2154721494 wait complete. 2024-03-26T06:30:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034f7b8740 20 client.16107 trim_cache size 0 max 16384 2024-03-26T06:30:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:30:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:30:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:30:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:30:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 -- 172.21.15.37:0/636920849 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f03340a2a10 msgr2=0x7f03340a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f03340a2a10 0x7f03340a4ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0340005fd0 tx=0x7f0340013b60 comp rx=0 tx=0).stop 2024-03-26T06:30:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 -- 172.21.15.37:0/636920849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0348100e00 msgr2=0x7f0348198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0348100e00 0x7f0348198ef0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0330005940 tx=0x7f0330004270 comp rx=0 tx=0).stop 2024-03-26T06:30:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 -- 172.21.15.37:0/636920849 shutdown_connections 2024-03-26T06:30:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f03340a2a10 0x7f03340a4ed0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0348105190 0x7f03481a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0348100e00 0x7f0348198ef0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 --2- 172.21.15.37:0/636920849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0348100440 0x7f03481989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 -- 172.21.15.37:0/636920849 >> 172.21.15.37:0/636920849 conn(0x7f0348077040 msgr2=0x7f03480fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 -- 172.21.15.37:0/636920849 shutdown_connections 2024-03-26T06:30:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:28.593+0000 7f034cbd4640 1 -- 172.21.15.37:0/636920849 wait complete. 2024-03-26T06:30:29.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:28 smithi066 ceph-mon[36504]: pgmap v229: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 597 KiB/s wr, 75 op/s 2024-03-26T06:30:29.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:28 smithi149 ceph-mon[36681]: pgmap v229: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 597 KiB/s wr, 75 op/s 2024-03-26T06:30:29.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:28 smithi037 ceph-mon[31974]: pgmap v229: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 597 KiB/s wr, 75 op/s 2024-03-26T06:30:29.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 259760 4.07 MB/sec execute 244 sec latency 1244.583 ms 2024-03-26T06:30:30.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 259760 4.06 MB/sec execute 245 sec latency 2244.642 ms 2024-03-26T06:30:31.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:30 smithi066 ceph-mon[36504]: pgmap v230: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 597 KiB/s wr, 75 op/s 2024-03-26T06:30:31.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:30 smithi149 ceph-mon[36681]: pgmap v230: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 597 KiB/s wr, 75 op/s 2024-03-26T06:30:31.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:30 smithi037 ceph-mon[31974]: pgmap v230: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 597 KiB/s wr, 75 op/s 2024-03-26T06:30:31.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 260124 4.05 MB/sec execute 246 sec latency 3041.707 ms 2024-03-26T06:30:32.510 INFO:tasks.workunit.client.0.smithi037.stdout: 1 262318 4.08 MB/sec execute 247 sec latency 17.216 ms 2024-03-26T06:30:33.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:32 smithi066 ceph-mon[36504]: pgmap v231: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 934 KiB/s wr, 127 op/s 2024-03-26T06:30:33.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:32 smithi149 ceph-mon[36681]: pgmap v231: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 934 KiB/s wr, 127 op/s 2024-03-26T06:30:33.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:32 smithi037 ceph-mon[31974]: pgmap v231: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 934 KiB/s wr, 127 op/s 2024-03-26T06:30:33.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 263648 4.09 MB/sec execute 248 sec latency 16.107 ms 2024-03-26T06:30:34.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 265065 4.10 MB/sec execute 249 sec latency 14.090 ms 2024-03-26T06:30:35.001 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:34 smithi037 ceph-mon[31974]: pgmap v232: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 393 KiB/s wr, 70 op/s 2024-03-26T06:30:35.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:34 smithi066 ceph-mon[36504]: pgmap v232: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 393 KiB/s wr, 70 op/s 2024-03-26T06:30:35.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:34 smithi149 ceph-mon[36681]: pgmap v232: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 393 KiB/s wr, 70 op/s 2024-03-26T06:30:35.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 266536 4.13 MB/sec execute 250 sec latency 35.429 ms 2024-03-26T06:30:36.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 267154 4.12 MB/sec execute 251 sec latency 614.031 ms 2024-03-26T06:30:37.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:36 smithi066 ceph-mon[36504]: pgmap v233: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 875 KiB/s wr, 135 op/s 2024-03-26T06:30:37.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:36 smithi149 ceph-mon[36681]: pgmap v233: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 875 KiB/s wr, 135 op/s 2024-03-26T06:30:37.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:36 smithi037 ceph-mon[31974]: pgmap v233: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 875 KiB/s wr, 135 op/s 2024-03-26T06:30:37.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 269418 4.15 MB/sec execute 252 sec latency 16.571 ms 2024-03-26T06:30:38.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 270269 4.15 MB/sec execute 253 sec latency 270.637 ms 2024-03-26T06:30:39.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:38 smithi066 ceph-mon[36504]: pgmap v234: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 818 KiB/s wr, 117 op/s 2024-03-26T06:30:39.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:38 smithi149 ceph-mon[36681]: pgmap v234: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 818 KiB/s wr, 117 op/s 2024-03-26T06:30:39.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:38 smithi037 ceph-mon[31974]: pgmap v234: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 818 KiB/s wr, 117 op/s 2024-03-26T06:30:39.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 270269 4.14 MB/sec execute 254 sec latency 1270.743 ms 2024-03-26T06:30:40.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 270269 4.12 MB/sec execute 255 sec latency 2270.877 ms 2024-03-26T06:30:41.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:40 smithi066 ceph-mon[36504]: pgmap v235: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 818 KiB/s wr, 117 op/s 2024-03-26T06:30:41.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:40 smithi149 ceph-mon[36681]: pgmap v235: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 818 KiB/s wr, 117 op/s 2024-03-26T06:30:41.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:40 smithi037 ceph-mon[31974]: pgmap v235: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 818 KiB/s wr, 117 op/s 2024-03-26T06:30:41.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 270595 4.11 MB/sec execute 256 sec latency 3069.288 ms 2024-03-26T06:30:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.683+0000 7fe6e511b640 1 -- 172.21.15.37:0/3247211521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e0105190 msgr2=0x7fe6e0105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.683+0000 7fe6e511b640 1 --2- 172.21.15.37:0/3247211521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e0105190 0x7fe6e0105570 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009900 tx=0x7fe6d802bc30 comp rx=0 tx=0).stop 2024-03-26T06:30:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.683+0000 7fe6e511b640 1 -- 172.21.15.37:0/3247211521 shutdown_connections 2024-03-26T06:30:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.683+0000 7fe6e511b640 1 --2- 172.21.15.37:0/3247211521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6e0100e00 0x7fe6e010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.683+0000 7fe6e511b640 1 --2- 172.21.15.37:0/3247211521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e0100440 0x7fe6e01008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.683+0000 7fe6e511b640 1 --2- 172.21.15.37:0/3247211521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e0105190 0x7fe6e0105570 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.683+0000 7fe6e511b640 1 -- 172.21.15.37:0/3247211521 >> 172.21.15.37:0/3247211521 conn(0x7fe6e0077040 msgr2=0x7fe6e0077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.683+0000 7fe6e511b640 1 -- 172.21.15.37:0/3247211521 shutdown_connections 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.683+0000 7fe6e511b640 1 -- 172.21.15.37:0/3247211521 wait complete. 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6e511b640 1 Processor -- start 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6e511b640 1 -- start start 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6e511b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6e0100440 0x7fe6e019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6e511b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e0100e00 0x7fe6e019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6e511b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e0105190 0x7fe6e0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6e511b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0110650 con 0x7fe6e0105190 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6e511b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e01104d0 con 0x7fe6e0100440 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6e511b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e01107d0 con 0x7fe6e0100e00 2024-03-26T06:30:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6de59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e0100e00 0x7fe6e019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6df59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e0105190 0x7fe6e0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6ded9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6e0100440 0x7fe6e019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6de59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e0100e00 0x7fe6e019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:40412/0 (socket says 172.21.15.37:40412) 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6de59c640 1 -- 172.21.15.37:0/2633994083 learned_addr learned my addr 172.21.15.37:0/2633994083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6de59c640 1 -- 172.21.15.37:0/2633994083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6e0100440 msgr2=0x7fe6e019d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6de59c640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6e0100440 0x7fe6e019d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6de59c640 1 -- 172.21.15.37:0/2633994083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e0105190 msgr2=0x7fe6e0197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6de59c640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e0105190 0x7fe6e0197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.684+0000 7fe6de59c640 1 -- 172.21.15.37:0/2633994083 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0198190 con 0x7fe6e0100e00 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.685+0000 7fe6de59c640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e0100e00 0x7fe6e019dda0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9f0 tx=0x7fe6c800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.685+0000 7fe6d7fff640 1 -- 172.21.15.37:0/2633994083 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800cd80 con 0x7fe6e0100e00 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.685+0000 7fe6d7fff640 1 -- 172.21.15.37:0/2633994083 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8005500 con 0x7fe6e0100e00 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.685+0000 7fe6e511b640 1 -- 172.21.15.37:0/2633994083 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e0198480 con 0x7fe6e0100e00 2024-03-26T06:30:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.685+0000 7fe6d7fff640 1 -- 172.21.15.37:0/2633994083 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8010690 con 0x7fe6e0100e00 2024-03-26T06:30:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.685+0000 7fe6e511b640 1 -- 172.21.15.37:0/2633994083 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e01a4720 con 0x7fe6e0100e00 2024-03-26T06:30:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.687+0000 7fe6d7fff640 1 -- 172.21.15.37:0/2633994083 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe6c80046c0 con 0x7fe6e0100e00 2024-03-26T06:30:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6e7dd4740 1 Processor -- start 2024-03-26T06:30:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6e7dd4740 1 -- start start 2024-03-26T06:30:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6d7fff640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe6cc0a2a90 0x7fe6cc0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6df59e640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe6cc0a2a90 0x7fe6cc0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 0x56116ca43a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6d7fff640 1 -- 172.21.15.37:0/2633994083 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe6c80cf230 con 0x7fe6e0100e00 2024-03-26T06:30:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca43fd0 0x56116ca44450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca4d550 0x56116ca4f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116c9a85f0 con 0x56116ca44d40 2024-03-26T06:30:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116c951930 con 0x56116ca4d550 2024-03-26T06:30:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116c9a4690 con 0x56116ca43fd0 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6ded9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 0x56116ca43a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6de59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca43fd0 0x56116ca44450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6ded9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 0x56116ca43a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54500/0 (socket says 172.21.15.37:54500) 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6ded9d640 1 -- 172.21.15.37:0/2596491901 learned_addr learned my addr 172.21.15.37:0/2596491901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.688+0000 7fe6df59e640 1 --2- 172.21.15.37:0/2596491901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca4d550 0x56116ca4f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.689+0000 7fe6df59e640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe6cc0a2a90 0x7fe6cc0a4f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005fd0 tx=0x7fe6d0009500 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.689+0000 7fe6de59c640 1 -- 172.21.15.37:0/2596491901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca4d550 msgr2=0x56116ca4f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.689+0000 7fe6de59c640 1 --2- 172.21.15.37:0/2596491901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca4d550 0x56116ca4f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.689+0000 7fe6de59c640 1 -- 172.21.15.37:0/2596491901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 msgr2=0x56116ca43a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.689+0000 7fe6de59c640 1 --2- 172.21.15.37:0/2596491901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 0x56116ca43a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.689+0000 7fe6de59c640 1 -- 172.21.15.37:0/2596491901 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56116c6bd210 con 0x56116ca43fd0 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.689+0000 7fe6de59c640 1 --2- 172.21.15.37:0/2596491901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca43fd0 0x56116ca44450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80cc920 tx=0x7fe6c809b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddc8aae1cb767f50 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.689+0000 7fe6d57fa640 1 -- 172.21.15.37:0/2596491901 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80b4070 con 0x56116ca43fd0 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.689+0000 7fe6d57fa640 1 -- 172.21.15.37:0/2596491901 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c809f980 con 0x56116ca43fd0 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.689+0000 7fe6d57fa640 1 -- 172.21.15.37:0/2596491901 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c809ba90 con 0x56116ca43fd0 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2596491901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca43fd0 msgr2=0x56116ca44450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/2596491901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca43fd0 0x56116ca44450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80cc920 tx=0x7fe6c809b430 comp rx=0 tx=0).stop 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2596491901 shutdown_connections 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/2596491901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca4d550 0x56116ca4f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/2596491901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca43fd0 0x56116ca44450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/2596491901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 0x56116ca43a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2596491901 >> 172.21.15.37:0/2596491901 conn(0x56116ca0b5b0 msgr2=0x56116ca3ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2596491901 shutdown_connections 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2596491901 wait complete. 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 -- start start 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 Processor -- start 2024-03-26T06:30:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca43fd0 0x56116ca4a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 0x56116ca4ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca4d550 0x56116ca4b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116ca50280 con 0x56116ca44d40 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116c964d90 con 0x56116ca43fd0 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116ca50400 con 0x56116ca4d550 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.690+0000 7fe6e7dd4740 10 client.0 _ll_register_callbacks cb 0x56116c6bca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6de59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 0x56116ca4ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6df59e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca4d550 0x56116ca4b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6ded9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca43fd0 0x56116ca4a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6de59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 0x56116ca4ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54504/0 (socket says 172.21.15.37:54504) 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6de59c640 1 -- 172.21.15.37:0/2453848102 learned_addr learned my addr 172.21.15.37:0/2453848102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6df59e640 1 -- 172.21.15.37:0/2453848102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca43fd0 msgr2=0x56116ca4a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6df59e640 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca43fd0 0x56116ca4a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6df59e640 1 -- 172.21.15.37:0/2453848102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 msgr2=0x56116ca4ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6df59e640 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 0x56116ca4ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6df59e640 1 -- 172.21.15.37:0/2453848102 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56116c789620 con 0x56116ca4d550 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6de59c640 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 0x56116ca4ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6df59e640 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca4d550 0x56116ca4b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x56116ca44a70 tx=0x7fe6d0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=860e91383487f73d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6c67fc640 1 -- 172.21.15.37:0/2453848102 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0004b20 con 0x56116ca4d550 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2453848102 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6c809f980 con 0x56116ca4d550 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.691+0000 7fe6c67fc640 1 -- 172.21.15.37:0/2453848102 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe6d0004cc0 con 0x56116ca4d550 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6c67fc640 1 -- 172.21.15.37:0/2453848102 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6d001c6f0 con 0x56116ca4d550 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6e7dd4740 10 client.25534 fetch_fsmap learned FSMap version 40 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2453848102 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56116ca5c650 con 0x56116ca4d550 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6c67fc640 1 -- 172.21.15.37:0/2453848102 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c809f980 con 0x56116ca4d550 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6c67fc640 1 -- 172.21.15.37:0/2453848102 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe6d00055f0 con 0x56116ca4d550 2024-03-26T06:30:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6e7dd4740 10 client.25534 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6e7dd4740 10 client.25534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6e7dd4740 20 client.25534 populate_metadata read hostname 'smithi037' 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56116ca5db60 0x56116ca5ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6e7dd4740 4 client.25534 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2453848102 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56116c9b5b30 con 0x56116ca5db60 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6ded9d640 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56116ca5db60 0x56116ca5ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6ded9d640 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56116ca5db60 0x56116ca5ff80 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83fc18bc2789ad45 server_cookie=b2b11ed149fae18b in_seq=0 out_seq=0 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.692+0000 7fe6c67fc640 10 client.25534 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6c67fc640 1 -- 172.21.15.37:0/2453848102 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56116c9b5b30 con 0x56116ca5db60 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6c67fc640 10 client.25534 handle_command_reply: tid=0 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 1 client.25534 shutdown 2024-03-26T06:30:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2453848102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca4d550 msgr2=0x56116ca4b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca4d550 0x56116ca4b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x56116ca44a70 tx=0x7fe6d0004ab0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2453848102 shutdown_connections 2024-03-26T06:30:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56116ca5db60 0x56116ca5ff80 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca4d550 0x56116ca4b0a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6c67fc640 0 client.25534 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca44d40 0x56116ca4ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/2453848102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca43fd0 0x56116ca4a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2453848102 >> 172.21.15.37:0/2453848102 conn(0x56116ca0b5b0 msgr2=0x56116ca40a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2453848102 shutdown_connections 2024-03-26T06:30:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/2453848102 wait complete. 2024-03-26T06:30:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.693+0000 7fe6e7dd4740 20 client.25534 trim_cache size 0 max 16384 2024-03-26T06:30:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6e7dd4740 1 Processor -- start 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6e7dd4740 1 -- start start 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca43fd0 0x56116ca4cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca44d40 0x56116ca54670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca4d550 0x56116ca54bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e01acc60 con 0x56116ca4d550 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e01ae6a0 con 0x56116ca44d40 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e01ad2c0 con 0x56116ca43fd0 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6df59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca4d550 0x56116ca54bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6df59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca4d550 0x56116ca54bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54520/0 (socket says 172.21.15.37:54520) 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6ded9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca43fd0 0x56116ca4cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6df59e640 1 -- 172.21.15.37:0/3044535281 learned_addr learned my addr 172.21.15.37:0/3044535281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6de59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca44d40 0x56116ca54670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6df59e640 1 -- 172.21.15.37:0/3044535281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca43fd0 msgr2=0x56116ca4cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6df59e640 1 --2- 172.21.15.37:0/3044535281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca43fd0 0x56116ca4cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6df59e640 1 -- 172.21.15.37:0/3044535281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca44d40 msgr2=0x56116ca54670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6df59e640 1 --2- 172.21.15.37:0/3044535281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca44d40 0x56116ca54670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6df59e640 1 -- 172.21.15.37:0/3044535281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56116ca65170 con 0x56116ca4d550 2024-03-26T06:30:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.707+0000 7fe6de59c640 1 --2- 172.21.15.37:0/3044535281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca44d40 0x56116ca54670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:30:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.708+0000 7fe6df59e640 1 --2- 172.21.15.37:0/3044535281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca4d550 0x56116ca54bb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005990 tx=0x7fe6d0004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cd7f0d3d6d1c1f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.708+0000 7fe6d4ff9640 1 -- 172.21.15.37:0/3044535281 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0034070 con 0x56116ca4d550 2024-03-26T06:30:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.708+0000 7fe6d4ff9640 1 -- 172.21.15.37:0/3044535281 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6d0003e30 con 0x56116ca4d550 2024-03-26T06:30:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.708+0000 7fe6d4ff9640 1 -- 172.21.15.37:0/3044535281 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0027bd0 con 0x56116ca4d550 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.708+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/3044535281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca4d550 msgr2=0x56116ca54bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.708+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/3044535281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca4d550 0x56116ca54bb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005990 tx=0x7fe6d0004850 comp rx=0 tx=0).stop 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/3044535281 shutdown_connections 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/3044535281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca4d550 0x56116ca54bb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/3044535281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca44d40 0x56116ca54670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/3044535281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca43fd0 0x56116ca4cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/3044535281 >> 172.21.15.37:0/3044535281 conn(0x56116ca0b5b0 msgr2=0x56116ca3e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/3044535281 shutdown_connections 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/3044535281 wait complete. 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 -- start start 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 Processor -- start 2024-03-26T06:30:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca43fd0 0x56116ca47350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca44d40 0x56116ca47890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca4d550 0x56116ca47dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e01adf80 con 0x56116ca43fd0 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e01ad920 con 0x56116ca4d550 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e01aaf20 con 0x56116ca44d40 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6e7dd4740 10 client.0 _ll_register_callbacks cb 0x7fe6c80056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6de59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca44d40 0x56116ca47890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.709+0000 7fe6ded9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca43fd0 0x56116ca47350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6de59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca44d40 0x56116ca47890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:40436/0 (socket says 172.21.15.37:40436) 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6de59c640 1 -- 172.21.15.37:0/811958847 learned_addr learned my addr 172.21.15.37:0/811958847 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6ded9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca43fd0 0x56116ca47350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54534/0 (socket says 172.21.15.37:54534) 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6df59e640 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca4d550 0x56116ca47dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6ded9d640 1 -- 172.21.15.37:0/811958847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca44d40 msgr2=0x56116ca47890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6ded9d640 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca44d40 0x56116ca47890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6ded9d640 1 -- 172.21.15.37:0/811958847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca4d550 msgr2=0x56116ca47dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6ded9d640 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca4d550 0x56116ca47dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6ded9d640 1 -- 172.21.15.37:0/811958847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56116cab8520 con 0x56116ca43fd0 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6de59c640 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca44d40 0x56116ca47890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6ded9d640 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca43fd0 0x56116ca47350 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8036040 tx=0x7fe6d8004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff4448cfbb2c9d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6c67fc640 1 -- 172.21.15.37:0/811958847 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d802cab0 con 0x56116ca43fd0 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6c67fc640 1 -- 172.21.15.37:0/811958847 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe6d802cc50 con 0x56116ca43fd0 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/811958847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6d0003e30 con 0x56116ca43fd0 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6c67fc640 1 -- 172.21.15.37:0/811958847 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6d8049410 con 0x56116ca43fd0 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6e7dd4740 10 client.16125 fetch_fsmap learned FSMap version 40 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.710+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/811958847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56116cab9870 con 0x56116ca43fd0 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.711+0000 7fe6c67fc640 1 -- 172.21.15.37:0/811958847 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6d0003e30 con 0x56116ca43fd0 2024-03-26T06:30:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.711+0000 7fe6c67fc640 1 -- 172.21.15.37:0/811958847 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe6d8004f50 con 0x56116ca43fd0 2024-03-26T06:30:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.711+0000 7fe6e7dd4740 10 client.16125 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:30:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.711+0000 7fe6e7dd4740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:30:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.711+0000 7fe6e7dd4740 20 client.16125 populate_metadata read hostname 'smithi037' 2024-03-26T06:30:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.711+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56116caba030 0x56116cabc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.711+0000 7fe6e7dd4740 4 client.16125 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:30:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.711+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/811958847 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56116c9afe80 con 0x56116caba030 2024-03-26T06:30:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.711+0000 7fe6de59c640 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56116caba030 0x56116cabc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.711+0000 7fe6c67fc640 10 client.16125 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.711+0000 7fe6de59c640 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56116caba030 0x56116cabc450 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c59dbe60d9d497 server_cookie=5a7e569eced9ca62 in_seq=0 out_seq=0 2024-03-26T06:30:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6c67fc640 1 -- 172.21.15.37:0/811958847 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56116c9afe80 con 0x56116caba030 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6c67fc640 10 client.16125 handle_command_reply: tid=0 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 1 client.16125 shutdown 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/811958847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca43fd0 msgr2=0x56116ca47350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca43fd0 0x56116ca47350 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8036040 tx=0x7fe6d8004200 comp rx=0 tx=0).stop 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/811958847 shutdown_connections 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56116caba030 0x56116cabc450 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56116ca4d550 0x56116ca47dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56116ca44d40 0x56116ca47890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6c67fc640 0 client.16125 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 1 --2- 172.21.15.37:0/811958847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56116ca43fd0 0x56116ca47350 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/811958847 >> 172.21.15.37:0/811958847 conn(0x56116ca0b5b0 msgr2=0x56116ca48b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/811958847 shutdown_connections 2024-03-26T06:30:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 1 -- 172.21.15.37:0/811958847 wait complete. 2024-03-26T06:30:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.744+0000 7fe6e7dd4740 20 client.16125 trim_cache size 0 max 16384 2024-03-26T06:30:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:30:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:30:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:30:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:30:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 -- 172.21.15.37:0/2633994083 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe6cc0a2a90 msgr2=0x7fe6cc0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe6cc0a2a90 0x7fe6cc0a4f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005fd0 tx=0x7fe6d0009500 comp rx=0 tx=0).stop 2024-03-26T06:30:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 -- 172.21.15.37:0/2633994083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e0100e00 msgr2=0x7fe6e019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e0100e00 0x7fe6e019dda0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9f0 tx=0x7fe6c800edf0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 -- 172.21.15.37:0/2633994083 shutdown_connections 2024-03-26T06:30:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe6cc0a2a90 0x7fe6cc0a4f50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e0105190 0x7fe6e0197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e0100e00 0x7fe6e019dda0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 --2- 172.21.15.37:0/2633994083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6e0100440 0x7fe6e019d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 -- 172.21.15.37:0/2633994083 >> 172.21.15.37:0/2633994083 conn(0x7fe6e0077040 msgr2=0x7fe6e0068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 -- 172.21.15.37:0/2633994083 shutdown_connections 2024-03-26T06:30:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:41.745+0000 7fe6e511b640 1 -- 172.21.15.37:0/2633994083 wait complete. 2024-03-26T06:30:42.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 272941 4.15 MB/sec execute 257 sec latency 21.715 ms 2024-03-26T06:30:43.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:42 smithi066 ceph-mon[36504]: pgmap v236: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-03-26T06:30:43.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:42 smithi149 ceph-mon[36681]: pgmap v236: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-03-26T06:30:43.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:42 smithi037 ceph-mon[31974]: pgmap v236: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-03-26T06:30:43.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 273818 4.15 MB/sec execute 258 sec latency 288.049 ms 2024-03-26T06:30:44.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 273818 4.13 MB/sec execute 259 sec latency 1288.134 ms 2024-03-26T06:30:45.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:44 smithi066 ceph-mon[36504]: pgmap v237: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-26T06:30:45.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:44 smithi149 ceph-mon[36681]: pgmap v237: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-26T06:30:45.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:44 smithi037 ceph-mon[31974]: pgmap v237: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-26T06:30:45.511 INFO:tasks.workunit.client.0.smithi037.stdout: 1 273818 4.12 MB/sec execute 260 sec latency 2288.183 ms 2024-03-26T06:30:46.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 274130 4.10 MB/sec execute 261 sec latency 3093.032 ms 2024-03-26T06:30:47.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:46 smithi066 ceph-mon[36504]: pgmap v238: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-03-26T06:30:47.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:46 smithi149 ceph-mon[36681]: pgmap v238: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-03-26T06:30:47.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:46 smithi037 ceph-mon[31974]: pgmap v238: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-03-26T06:30:47.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 276361 4.14 MB/sec execute 262 sec latency 15.643 ms 2024-03-26T06:30:48.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 277416 4.14 MB/sec execute 263 sec latency 167.111 ms 2024-03-26T06:30:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:48 smithi066 ceph-mon[36504]: pgmap v239: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-03-26T06:30:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:48 smithi149 ceph-mon[36681]: pgmap v239: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-03-26T06:30:49.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:48 smithi037 ceph-mon[31974]: pgmap v239: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-03-26T06:30:49.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 277416 4.13 MB/sec execute 264 sec latency 1167.216 ms 2024-03-26T06:30:50.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 277416 4.11 MB/sec execute 265 sec latency 2167.303 ms 2024-03-26T06:30:51.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:50 smithi066 ceph-mon[36504]: pgmap v240: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-03-26T06:30:51.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:50 smithi149 ceph-mon[36681]: pgmap v240: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-03-26T06:30:51.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:50 smithi037 ceph-mon[31974]: pgmap v240: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-03-26T06:30:51.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 277762 4.10 MB/sec execute 266 sec latency 2965.952 ms 2024-03-26T06:30:52.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 280069 4.13 MB/sec execute 267 sec latency 21.430 ms 2024-03-26T06:30:53.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:52 smithi066 ceph-mon[36504]: pgmap v241: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-03-26T06:30:53.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:52 smithi149 ceph-mon[36681]: pgmap v241: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-03-26T06:30:53.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:52 smithi037 ceph-mon[31974]: pgmap v241: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-03-26T06:30:53.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 281426 4.14 MB/sec execute 268 sec latency 11.340 ms 2024-03-26T06:30:54.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 283448 4.17 MB/sec execute 269 sec latency 16.694 ms 2024-03-26T06:30:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.837+0000 7f0ef33a2640 1 -- 172.21.15.37:0/2536767551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eec100be0 msgr2=0x7f0eec104c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.837+0000 7f0ef33a2640 1 --2- 172.21.15.37:0/2536767551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eec100be0 0x7f0eec104c20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009900 tx=0x7f0edc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:30:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.837+0000 7f0ef33a2640 1 -- 172.21.15.37:0/2536767551 shutdown_connections 2024-03-26T06:30:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.837+0000 7f0ef33a2640 1 --2- 172.21.15.37:0/2536767551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eec105300 0x7f0eec10ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.837+0000 7f0ef33a2640 1 --2- 172.21.15.37:0/2536767551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eec100be0 0x7f0eec104c20 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.837+0000 7f0ef33a2640 1 --2- 172.21.15.37:0/2536767551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0eec100230 0x7f0eec100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.837+0000 7f0ef33a2640 1 -- 172.21.15.37:0/2536767551 >> 172.21.15.37:0/2536767551 conn(0x7f0eec0fbaf0 msgr2=0x7f0eec0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.837+0000 7f0ef33a2640 1 -- 172.21.15.37:0/2536767551 shutdown_connections 2024-03-26T06:30:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.837+0000 7f0ef33a2640 1 -- 172.21.15.37:0/2536767551 wait complete. 2024-03-26T06:30:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef33a2640 1 Processor -- start 2024-03-26T06:30:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef33a2640 1 -- start start 2024-03-26T06:30:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef33a2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eec100230 0x7f0eec101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef33a2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0eec105300 0x7f0eec102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef33a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eec102740 0x7f0eec1a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef33a2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec110370 con 0x7f0eec102740 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef33a2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1101f0 con 0x7f0eec100230 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef33a2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1104f0 con 0x7f0eec105300 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef2ba1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eec102740 0x7f0eec1a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef2ba1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eec102740 0x7f0eec1a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39150/0 (socket says 172.21.15.37:39150) 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef1b9f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0eec105300 0x7f0eec102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef2ba1640 1 -- 172.21.15.37:0/3045181482 learned_addr learned my addr 172.21.15.37:0/3045181482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef23a0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eec100230 0x7f0eec101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef2ba1640 1 -- 172.21.15.37:0/3045181482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0eec105300 msgr2=0x7f0eec102200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.838+0000 7f0ef2ba1640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0eec105300 0x7f0eec102200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.839+0000 7f0ef2ba1640 1 -- 172.21.15.37:0/3045181482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eec100230 msgr2=0x7f0eec101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.839+0000 7f0ef2ba1640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eec100230 0x7f0eec101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.839+0000 7f0ef2ba1640 1 -- 172.21.15.37:0/3045181482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec1a6330 con 0x7f0eec102740 2024-03-26T06:30:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.839+0000 7f0ef2ba1640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eec102740 0x7f0eec1a5c90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400eb10 tx=0x7f0ee400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.839+0000 7f0ee37fe640 1 -- 172.21.15.37:0/3045181482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4004b20 con 0x7f0eec102740 2024-03-26T06:30:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.839+0000 7f0ef33a2640 1 -- 172.21.15.37:0/3045181482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eec1a6620 con 0x7f0eec102740 2024-03-26T06:30:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.839+0000 7f0ee37fe640 1 -- 172.21.15.37:0/3045181482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ee4004cc0 con 0x7f0eec102740 2024-03-26T06:30:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.839+0000 7f0ee37fe640 1 -- 172.21.15.37:0/3045181482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4010760 con 0x7f0eec102740 2024-03-26T06:30:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.839+0000 7f0ef33a2640 1 -- 172.21.15.37:0/3045181482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec1a6b40 con 0x7f0eec102740 2024-03-26T06:30:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef71e4740 1 Processor -- start 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ee37fe640 1 -- 172.21.15.37:0/3045181482 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f0ee4005020 con 0x7f0eec102740 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef71e4740 1 -- start start 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b2723f20 0x55c8b2722c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b27231b0 0x55c8b2723630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b272eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b26875f0 con 0x55c8b2723f20 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b2630930 con 0x55c8b272c730 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b2683690 con 0x55c8b27231b0 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef1b9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b2723f20 0x55c8b2722c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef1b9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b2723f20 0x55c8b2722c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39162/0 (socket says 172.21.15.37:39162) 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/1189275735 learned_addr learned my addr 172.21.15.37:0/1189275735 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ee37fe640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0ed80a2960 0x7f0ed80a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1189275735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b272eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.842+0000 7f0ef23a0640 1 --2- 172.21.15.37:0/1189275735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b27231b0 0x55c8b2723630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ee37fe640 1 -- 172.21.15.37:0/3045181482 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0ee40cf3d0 con 0x7f0eec102740 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ef2ba1640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0ed80a2960 0x7f0ed80a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/1189275735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b27231b0 msgr2=0x55c8b2723630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1189275735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b27231b0 0x55c8b2723630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/1189275735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 msgr2=0x55c8b272eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1189275735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b272eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/1189275735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b239c210 con 0x55c8b2723f20 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1189275735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b2723f20 0x55c8b2722c70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0edc005ec0 tx=0x7f0edc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb25d52f152cedb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ee0bfe640 1 -- 172.21.15.37:0/1189275735 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc004020 con 0x55c8b2723f20 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ef2ba1640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0ed80a2960 0x7f0ed80a4e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40cd120 tx=0x7f0ee4005b40 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ee0bfe640 1 -- 172.21.15.37:0/1189275735 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0edc004ba0 con 0x55c8b2723f20 2024-03-26T06:30:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.843+0000 7f0ee0bfe640 1 -- 172.21.15.37:0/1189275735 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc005200 con 0x55c8b2723f20 2024-03-26T06:30:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.844+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1189275735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b2723f20 msgr2=0x55c8b2722c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.844+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1189275735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b2723f20 0x55c8b2722c70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0edc005ec0 tx=0x7f0edc005dd0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.844+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1189275735 shutdown_connections 2024-03-26T06:30:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.844+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1189275735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b272eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.844+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1189275735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b27231b0 0x55c8b2723630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.844+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1189275735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b2723f20 0x55c8b2722c70 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.844+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1189275735 >> 172.21.15.37:0/1189275735 conn(0x55c8b26ea5b0 msgr2=0x55c8b271dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.844+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1189275735 shutdown_connections 2024-03-26T06:30:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.844+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1189275735 wait complete. 2024-03-26T06:30:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.844+0000 7f0ef71e4740 1 -- start start 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.844+0000 7f0ef71e4740 1 Processor -- start 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2729760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2729ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b272a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b272f3a0 con 0x55c8b27231b0 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b2643d90 con 0x55c8b272c730 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b272f520 con 0x55c8b2723f20 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef71e4740 10 client.0 _ll_register_callbacks cb 0x55c8b239ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef1b9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2729760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef1b9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2729760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39172/0 (socket says 172.21.15.37:39172) 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/1313726152 learned_addr learned my addr 172.21.15.37:0/1313726152 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef23a0640 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2729ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b272a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/1313726152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 msgr2=0x55c8b2729ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2729ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/1313726152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 msgr2=0x55c8b272a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b272a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/1313726152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8b2468620 con 0x55c8b27231b0 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2729760 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0edc004b50 tx=0x7f0edc004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53aa21d43e7ee158 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ed5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1313726152 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc043030 con 0x55c8b27231b0 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1313726152 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0edc0051a0 con 0x55c8b27231b0 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.845+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1313726152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0edc004ba0 con 0x55c8b27231b0 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1313726152 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0edc0053a0 con 0x55c8b27231b0 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ef71e4740 10 client.16137 fetch_fsmap learned FSMap version 40 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1313726152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c8b273b680 con 0x55c8b27231b0 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1313726152 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0edc004ba0 con 0x55c8b27231b0 2024-03-26T06:30:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1313726152 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0edc034440 con 0x55c8b27231b0 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ef71e4740 10 client.16137 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ef71e4740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ef71e4740 20 client.16137 populate_metadata read hostname 'smithi037' 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c8b273c9d0 0x55c8b273edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ef71e4740 4 client.16137 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1313726152 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8b2694b30 con 0x55c8b273c9d0 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.846+0000 7f0ef23a0640 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c8b273c9d0 0x55c8b273edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ef23a0640 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c8b273c9d0 0x55c8b273edc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3196975ae4fc0778 server_cookie=68c336422d3328c8 in_seq=0 out_seq=0 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ed5ffb640 10 client.16137 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1313726152 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c8b2694b30 con 0x55c8b273c9d0 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ed5ffb640 10 client.16137 handle_command_reply: tid=0 2024-03-26T06:30:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ef71e4740 1 client.16137 shutdown 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1313726152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 msgr2=0x55c8b2729760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2729760 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0edc004b50 tx=0x7f0edc004ec0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1313726152 shutdown_connections 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c8b273c9d0 0x55c8b273edc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b272a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2729ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1313726152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2729760 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1313726152 >> 172.21.15.37:0/1313726152 conn(0x55c8b26ea5b0 msgr2=0x55c8b271fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.847+0000 7f0ed5ffb640 0 client.16137 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.848+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1313726152 shutdown_connections 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.848+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1313726152 wait complete. 2024-03-26T06:30:54.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.848+0000 7f0ef71e4740 20 client.16137 trim_cache size 0 max 16384 2024-03-26T06:30:54.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.861+0000 7f0ef71e4740 1 Processor -- start 2024-03-26T06:30:54.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.861+0000 7f0ef71e4740 1 -- start start 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.861+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b27231b0 0x55c8b272be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.861+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2728030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b272c730 0x55c8b2728570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1aed40 con 0x55c8b272c730 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1b0780 con 0x55c8b27231b0 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1af3a0 con 0x55c8b2723f20 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef1b9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b272c730 0x55c8b2728570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef23a0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2728030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef1b9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b272c730 0x55c8b2728570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39182/0 (socket says 172.21.15.37:39182) 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/3965952522 learned_addr learned my addr 172.21.15.37:0/3965952522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/3965952522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b27231b0 0x55c8b272be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/3965952522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 msgr2=0x55c8b2728030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/3965952522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2728030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/3965952522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b27231b0 msgr2=0x55c8b272be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/3965952522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b27231b0 0x55c8b272be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/3965952522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b27182f0 con 0x55c8b272c730 2024-03-26T06:30:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/3965952522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b272c730 0x55c8b2728570 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0edc005940 tx=0x7f0edc02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820594ae70b7aab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ed7fff640 1 -- 172.21.15.37:0/3965952522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc043040 con 0x55c8b272c730 2024-03-26T06:30:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ed7fff640 1 -- 172.21.15.37:0/3965952522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0edc009520 con 0x55c8b272c730 2024-03-26T06:30:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.862+0000 7f0ed7fff640 1 -- 172.21.15.37:0/3965952522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc034440 con 0x55c8b272c730 2024-03-26T06:30:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.863+0000 7f0ef71e4740 1 -- 172.21.15.37:0/3965952522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b272c730 msgr2=0x55c8b2728570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.863+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/3965952522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b272c730 0x55c8b2728570 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0edc005940 tx=0x7f0edc02c650 comp rx=0 tx=0).stop 2024-03-26T06:30:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.863+0000 7f0ef71e4740 1 -- 172.21.15.37:0/3965952522 shutdown_connections 2024-03-26T06:30:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.863+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/3965952522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b272c730 0x55c8b2728570 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.863+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/3965952522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2728030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.863+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/3965952522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b27231b0 0x55c8b272be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.863+0000 7f0ef71e4740 1 -- 172.21.15.37:0/3965952522 >> 172.21.15.37:0/3965952522 conn(0x55c8b26ea5b0 msgr2=0x55c8b2724a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.863+0000 7f0ef71e4740 1 -- 172.21.15.37:0/3965952522 shutdown_connections 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.863+0000 7f0ef71e4740 1 -- 172.21.15.37:0/3965952522 wait complete. 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.863+0000 7f0ef71e4740 1 -- start start 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef71e4740 1 Processor -- start 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2733ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2731ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef71e4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b2732400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1b0060 con 0x55c8b27231b0 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1afa00 con 0x55c8b272c730 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef71e4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1ad000 con 0x55c8b2723f20 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef71e4740 10 client.0 _ll_register_callbacks cb 0x7f0edc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:30:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef1b9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2733ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef1b9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2733ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39196/0 (socket says 172.21.15.37:39196) 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef1b9f640 1 -- 172.21.15.37:0/1850990830 learned_addr learned my addr 172.21.15.37:0/1850990830 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef23a0640 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2731ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b2732400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef23a0640 1 -- 172.21.15.37:0/1850990830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 msgr2=0x55c8b2732400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef23a0640 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b2732400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef23a0640 1 -- 172.21.15.37:0/1850990830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 msgr2=0x55c8b2733ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef23a0640 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2733ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef23a0640 1 -- 172.21.15.37:0/1850990830 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8b2722480 con 0x55c8b2723f20 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2733ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ef23a0640 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2731ec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c8b273f330 tx=0x7f0ee800bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d636e4f20309702c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.864+0000 7f0ed5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:30:54.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1850990830 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80037f0 con 0x55c8b2723f20 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1850990830 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0edc009520 con 0x55c8b2723f20 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1850990830 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0ee8003990 con 0x55c8b2723f20 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1850990830 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ee801f8a0 con 0x55c8b2723f20 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ef71e4740 10 client.25558 fetch_fsmap learned FSMap version 40 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1850990830 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c8b2793a80 con 0x55c8b2723f20 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1850990830 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0edc009520 con 0x55c8b2723f20 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1850990830 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0ee8026070 con 0x55c8b2723f20 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ef71e4740 10 client.25558 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ef71e4740 10 client.25558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ef71e4740 20 client.25558 populate_metadata read hostname 'smithi037' 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c8b2794240 0x55c8b2796660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ef71e4740 4 client.25558 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:30:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1850990830 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c8b2722ea0 con 0x55c8b2794240 2024-03-26T06:30:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.865+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c8b2794240 0x55c8b2796660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:30:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.866+0000 7f0ef1b9f640 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c8b2794240 0x55c8b2796660 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0dc90e61deaeacd server_cookie=400f4351d3dd725d in_seq=0 out_seq=0 2024-03-26T06:30:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.866+0000 7f0ed5ffb640 10 client.25558 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ed5ffb640 1 -- 172.21.15.37:0/1850990830 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c8b2722ea0 con 0x55c8b2794240 2024-03-26T06:30:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ed5ffb640 10 client.25558 handle_command_reply: tid=0 2024-03-26T06:30:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 1 client.25558 shutdown 2024-03-26T06:30:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1850990830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 msgr2=0x55c8b2731ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2731ec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c8b273f330 tx=0x7f0ee800bec0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1850990830 shutdown_connections 2024-03-26T06:30:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c8b2794240 0x55c8b2796660 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c8b272c730 0x55c8b2732400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ed5ffb640 0 client.25558 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:30:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b2723f20 0x55c8b2731ec0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 1 --2- 172.21.15.37:0/1850990830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8b27231b0 0x55c8b2733ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1850990830 >> 172.21.15.37:0/1850990830 conn(0x55c8b26ea5b0 msgr2=0x55c8b2727ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1850990830 shutdown_connections 2024-03-26T06:30:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 1 -- 172.21.15.37:0/1850990830 wait complete. 2024-03-26T06:30:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.900+0000 7f0ef71e4740 20 client.25558 trim_cache size 0 max 16384 2024-03-26T06:30:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:30:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:30:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:30:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:30:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 -- 172.21.15.37:0/3045181482 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0ed80a2960 msgr2=0x7f0ed80a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0ed80a2960 0x7f0ed80a4e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40cd120 tx=0x7f0ee4005b40 comp rx=0 tx=0).stop 2024-03-26T06:30:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 -- 172.21.15.37:0/3045181482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eec102740 msgr2=0x7f0eec1a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:30:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eec102740 0x7f0eec1a5c90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400eb10 tx=0x7f0ee400c2d0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 -- 172.21.15.37:0/3045181482 shutdown_connections 2024-03-26T06:30:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f0ed80a2960 0x7f0ed80a4e20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0eec102740 0x7f0eec1a5c90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0eec105300 0x7f0eec102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 --2- 172.21.15.37:0/3045181482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eec100230 0x7f0eec101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:30:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 -- 172.21.15.37:0/3045181482 >> 172.21.15.37:0/3045181482 conn(0x7f0eec0fbaf0 msgr2=0x7f0eec0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:30:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 -- 172.21.15.37:0/3045181482 shutdown_connections 2024-03-26T06:30:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:30:54.901+0000 7f0ef33a2640 1 -- 172.21.15.37:0/3045181482 wait complete. 2024-03-26T06:30:55.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:54 smithi066 ceph-mon[36504]: pgmap v242: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-26T06:30:55.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:54 smithi149 ceph-mon[36681]: pgmap v242: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-26T06:30:55.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:54 smithi037 ceph-mon[31974]: pgmap v242: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-26T06:30:55.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 284575 4.18 MB/sec execute 270 sec latency 88.233 ms 2024-03-26T06:30:56.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 284939 4.17 MB/sec execute 271 sec latency 883.050 ms 2024-03-26T06:30:57.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:56 smithi066 ceph-mon[36504]: pgmap v243: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-03-26T06:30:57.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:56 smithi149 ceph-mon[36681]: pgmap v243: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-03-26T06:30:57.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:56 smithi037 ceph-mon[31974]: pgmap v243: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 214 op/s 2024-03-26T06:30:57.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 286276 4.17 MB/sec execute 272 sec latency 16.567 ms 2024-03-26T06:30:58.512 INFO:tasks.workunit.client.0.smithi037.stdout: 1 287869 4.20 MB/sec execute 273 sec latency 10.831 ms 2024-03-26T06:30:59.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:30:58 smithi066 ceph-mon[36504]: pgmap v244: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-26T06:30:59.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:30:58 smithi149 ceph-mon[36681]: pgmap v244: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-26T06:30:59.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:30:58 smithi037 ceph-mon[31974]: pgmap v244: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-26T06:30:59.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 288055 4.19 MB/sec execute 274 sec latency 870.741 ms 2024-03-26T06:31:00.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 288055 4.17 MB/sec execute 275 sec latency 1870.852 ms 2024-03-26T06:31:01.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:00 smithi066 ceph-mon[36504]: pgmap v245: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-26T06:31:01.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:00 smithi149 ceph-mon[36681]: pgmap v245: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-26T06:31:01.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:00 smithi037 ceph-mon[31974]: pgmap v245: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-26T06:31:01.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 288553 4.16 MB/sec execute 276 sec latency 2253.131 ms 2024-03-26T06:31:02.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 290965 4.20 MB/sec execute 277 sec latency 14.255 ms 2024-03-26T06:31:03.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:02 smithi066 ceph-mon[36504]: pgmap v246: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-03-26T06:31:03.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:02 smithi149 ceph-mon[36681]: pgmap v246: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-03-26T06:31:03.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:02 smithi037 ceph-mon[31974]: pgmap v246: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-03-26T06:31:03.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 292285 4.21 MB/sec execute 278 sec latency 13.808 ms 2024-03-26T06:31:04.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 293847 4.22 MB/sec execute 279 sec latency 14.640 ms 2024-03-26T06:31:05.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:04 smithi066 ceph-mon[36504]: pgmap v247: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-03-26T06:31:05.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:04 smithi149 ceph-mon[36681]: pgmap v247: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-03-26T06:31:05.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:04 smithi037 ceph-mon[31974]: pgmap v247: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.2 MiB/s wr, 150 op/s 2024-03-26T06:31:05.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 295147 4.23 MB/sec execute 280 sec latency 22.582 ms 2024-03-26T06:31:06.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:05 smithi066 ceph-mon[36504]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T06:31:06.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:05 smithi066 ceph-mon[36504]: pgmap v249: 97 pgs: 36 active+clean+snaptrim_wait, 41 active+clean, 20 active+clean+snaptrim; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-26T06:31:06.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:05 smithi149 ceph-mon[36681]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T06:31:06.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:05 smithi149 ceph-mon[36681]: pgmap v249: 97 pgs: 36 active+clean+snaptrim_wait, 41 active+clean, 20 active+clean+snaptrim; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-26T06:31:06.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:05 smithi037 ceph-mon[31974]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T06:31:06.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:05 smithi037 ceph-mon[31974]: pgmap v249: 97 pgs: 36 active+clean+snaptrim_wait, 41 active+clean, 20 active+clean+snaptrim; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-26T06:31:06.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 297191 4.26 MB/sec execute 281 sec latency 21.354 ms 2024-03-26T06:31:07.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 298640 4.27 MB/sec execute 282 sec latency 50.264 ms 2024-03-26T06:31:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.992+0000 7f7c72a8c640 1 -- 172.21.15.37:0/3875633302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c6c105190 msgr2=0x7f7c6c105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.992+0000 7f7c72a8c640 1 --2- 172.21.15.37:0/3875633302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c6c105190 0x7f7c6c105570 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009ca0 tx=0x7f7c6402c0f0 comp rx=0 tx=0).stop 2024-03-26T06:31:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.992+0000 7f7c72a8c640 1 -- 172.21.15.37:0/3875633302 shutdown_connections 2024-03-26T06:31:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.992+0000 7f7c72a8c640 1 --2- 172.21.15.37:0/3875633302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c100e00 0x7f7c6c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.992+0000 7f7c72a8c640 1 --2- 172.21.15.37:0/3875633302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c100440 0x7f7c6c1008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.992+0000 7f7c72a8c640 1 --2- 172.21.15.37:0/3875633302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c6c105190 0x7f7c6c105570 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c72a8c640 1 -- 172.21.15.37:0/3875633302 >> 172.21.15.37:0/3875633302 conn(0x7f7c6c077040 msgr2=0x7f7c6c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c72a8c640 1 -- 172.21.15.37:0/3875633302 shutdown_connections 2024-03-26T06:31:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c72a8c640 1 -- 172.21.15.37:0/3875633302 wait complete. 2024-03-26T06:31:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c72a8c640 1 Processor -- start 2024-03-26T06:31:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c72a8c640 1 -- start start 2024-03-26T06:31:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c72a8c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c100440 0x7f7c6c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c72a8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c6c100e00 0x7f7c6c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c72a8c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c105190 0x7f7c6c1a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c72a8c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c110650 con 0x7f7c6c100e00 2024-03-26T06:31:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c72a8c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1104d0 con 0x7f7c6c100440 2024-03-26T06:31:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c72a8c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1107d0 con 0x7f7c6c105190 2024-03-26T06:31:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.993+0000 7f7c6bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c6c100e00 0x7f7c6c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c6bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c6c100e00 0x7f7c6c198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43142/0 (socket says 172.21.15.37:43142) 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c6bfff640 1 -- 172.21.15.37:0/854858235 learned_addr learned my addr 172.21.15.37:0/854858235 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c71029640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c105190 0x7f7c6c1a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c70828640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c100440 0x7f7c6c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c6bfff640 1 -- 172.21.15.37:0/854858235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c105190 msgr2=0x7f7c6c1a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c6bfff640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c105190 0x7f7c6c1a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c6bfff640 1 -- 172.21.15.37:0/854858235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c100440 msgr2=0x7f7c6c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c6bfff640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c100440 0x7f7c6c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c6bfff640 1 -- 172.21.15.37:0/854858235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c1a3f90 con 0x7f7c6c100e00 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c6bfff640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c6c100e00 0x7f7c6c198f10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000e9f0 tx=0x7f7c6000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c69ffb640 1 -- 172.21.15.37:0/854858235 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000cd80 con 0x7f7c6c100e00 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c69ffb640 1 -- 172.21.15.37:0/854858235 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c60005500 con 0x7f7c6c100e00 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c69ffb640 1 -- 172.21.15.37:0/854858235 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c60010690 con 0x7f7c6c100e00 2024-03-26T06:31:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c72a8c640 1 -- 172.21.15.37:0/854858235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6c1a4160 con 0x7f7c6c100e00 2024-03-26T06:31:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.994+0000 7f7c72a8c640 1 -- 172.21.15.37:0/854858235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6c1a4630 con 0x7f7c6c100e00 2024-03-26T06:31:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.997+0000 7f7c69ffb640 1 -- 172.21.15.37:0/854858235 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7c600046c0 con 0x7f7c6c100e00 2024-03-26T06:31:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.997+0000 7f7c75674740 1 Processor -- start 2024-03-26T06:31:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.997+0000 7f7c75674740 1 -- start start 2024-03-26T06:31:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c69ffb640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7c540a2a10 0x7f7c540a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c75674740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c75674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c75674740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058bc550 0x5566058be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c75674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566058175f0 con 0x5566058b2fd0 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c75674740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566057c0930 con 0x5566058bc550 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c75674740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556605813690 con 0x5566058b3d40 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c6bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c71029640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7c540a2a10 0x7f7c540a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c6bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43152/0 (socket says 172.21.15.37:43152) 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c6bfff640 1 -- 172.21.15.37:0/3850108438 learned_addr learned my addr 172.21.15.37:0/3850108438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c71029640 1 --2- 172.21.15.37:0/3850108438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058bc550 0x5566058be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c69ffb640 1 -- 172.21.15.37:0/854858235 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f7c600cf320 con 0x7f7c6c100e00 2024-03-26T06:31:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c70828640 1 --2- 172.21.15.37:0/3850108438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c6bfff640 1 -- 172.21.15.37:0/3850108438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 msgr2=0x5566058b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c6bfff640 1 --2- 172.21.15.37:0/3850108438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c6bfff640 1 -- 172.21.15.37:0/3850108438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058bc550 msgr2=0x5566058be940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c6bfff640 1 --2- 172.21.15.37:0/3850108438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058bc550 0x5566058be940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c6bfff640 1 -- 172.21.15.37:0/3850108438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55660552c210 con 0x5566058b2fd0 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c71029640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7c540a2a10 0x7f7c540a4ed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c005fd0 tx=0x7f7c5c00bd60 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c6bfff640 1 --2- 172.21.15.37:0/3850108438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058b3450 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7c60098330 tx=0x7f7c60005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7952b1c4662016 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.998+0000 7f7c52ffd640 1 -- 172.21.15.37:0/3850108438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c600ae720 con 0x5566058b2fd0 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.999+0000 7f7c52ffd640 1 -- 172.21.15.37:0/3850108438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c6009c060 con 0x5566058b2fd0 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.999+0000 7f7c52ffd640 1 -- 172.21.15.37:0/3850108438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c600ae360 con 0x5566058b2fd0 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.999+0000 7f7c75674740 1 -- 172.21.15.37:0/3850108438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 msgr2=0x5566058b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.999+0000 7f7c75674740 1 --2- 172.21.15.37:0/3850108438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058b3450 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7c60098330 tx=0x7f7c60005740 comp rx=0 tx=0).stop 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.999+0000 7f7c75674740 1 -- 172.21.15.37:0/3850108438 shutdown_connections 2024-03-26T06:31:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.999+0000 7f7c75674740 1 --2- 172.21.15.37:0/3850108438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058bc550 0x5566058be940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.999+0000 7f7c75674740 1 --2- 172.21.15.37:0/3850108438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058b3450 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.999+0000 7f7c75674740 1 --2- 172.21.15.37:0/3850108438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.999+0000 7f7c75674740 1 -- 172.21.15.37:0/3850108438 >> 172.21.15.37:0/3850108438 conn(0x55660587a5b0 msgr2=0x5566058adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.999+0000 7f7c75674740 1 -- 172.21.15.37:0/3850108438 shutdown_connections 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:07.999+0000 7f7c75674740 1 -- 172.21.15.37:0/3850108438 wait complete. 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c75674740 1 -- start start 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c75674740 1 Processor -- start 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c75674740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b2fd0 0x5566058b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c75674740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c75674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c75674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566058bf220 con 0x5566058bc550 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c75674740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566057d3d90 con 0x5566058b2fd0 2024-03-26T06:31:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c75674740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566058bf3a0 con 0x5566058b3d40 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c75674740 10 client.0 _ll_register_callbacks cb 0x55660552ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c71029640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c71029640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058ba0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43166/0 (socket says 172.21.15.37:43166) 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c71029640 1 -- 172.21.15.37:0/2404050079 learned_addr learned my addr 172.21.15.37:0/2404050079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c70828640 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b2fd0 0x5566058b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.000+0000 7f7c6bfff640 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c71029640 1 -- 172.21.15.37:0/2404050079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 msgr2=0x5566058b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c71029640 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c71029640 1 -- 172.21.15.37:0/2404050079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b2fd0 msgr2=0x5566058b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c71029640 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b2fd0 0x5566058b9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c71029640 1 -- 172.21.15.37:0/2404050079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566055f8620 con 0x5566058bc550 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c71029640 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058ba0c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0126a0 tx=0x7f7c5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee9e1c31e23b4f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2404050079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c005220 con 0x5566058bc550 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2404050079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f7c5c0053c0 con 0x5566058bc550 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c75674740 1 -- 172.21.15.37:0/2404050079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c6009c060 con 0x5566058bc550 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2404050079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c5c0055c0 con 0x5566058bc550 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c75674740 10 client.16155 fetch_fsmap learned FSMap version 40 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c75674740 1 -- 172.21.15.37:0/2404050079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5566058cb5c0 con 0x5566058bc550 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2404050079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c6009c060 con 0x5566058bc550 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.001+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2404050079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7c5c01fbe0 con 0x5566058bc550 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.002+0000 7f7c75674740 10 client.16155 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.002+0000 7f7c75674740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.002+0000 7f7c75674740 20 client.16155 populate_metadata read hostname 'smithi037' 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.002+0000 7f7c75674740 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5566058cca80 0x5566058cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.002+0000 7f7c75674740 4 client.16155 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:31:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.002+0000 7f7c75674740 1 -- 172.21.15.37:0/2404050079 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556605824b30 con 0x5566058cca80 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.002+0000 7f7c70828640 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5566058cca80 0x5566058cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.002+0000 7f7c4bfff640 10 client.16155 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.002+0000 7f7c70828640 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5566058cca80 0x5566058cef00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1209f38e50e57e server_cookie=2f440a987b17cd5d in_seq=0 out_seq=0 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2404050079 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556605824b30 con 0x5566058cca80 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c4bfff640 10 client.16155 handle_command_reply: tid=0 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 1 client.16155 shutdown 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 1 -- 172.21.15.37:0/2404050079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 msgr2=0x5566058ba0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058ba0c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0126a0 tx=0x7f7c5c004730 comp rx=0 tx=0).stop 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 1 -- 172.21.15.37:0/2404050079 shutdown_connections 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5566058cca80 0x5566058cef00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058ba0c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 1 --2- 172.21.15.37:0/2404050079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b2fd0 0x5566058b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c4bfff640 0 client.16155 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 1 -- 172.21.15.37:0/2404050079 >> 172.21.15.37:0/2404050079 conn(0x55660587a5b0 msgr2=0x5566058afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 1 -- 172.21.15.37:0/2404050079 shutdown_connections 2024-03-26T06:31:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 1 -- 172.21.15.37:0/2404050079 wait complete. 2024-03-26T06:31:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.003+0000 7f7c75674740 20 client.16155 trim_cache size 0 max 16384 2024-03-26T06:31:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c75674740 1 Processor -- start 2024-03-26T06:31:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c75674740 1 -- start start 2024-03-26T06:31:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c75674740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b2fd0 0x5566058bbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c75674740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058c3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c75674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c75674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1acc80 con 0x5566058bc550 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c75674740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1ae6c0 con 0x5566058b2fd0 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c75674740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1ad2e0 con 0x5566058b3d40 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c71029640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c71029640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43180/0 (socket says 172.21.15.37:43180) 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c71029640 1 -- 172.21.15.37:0/2061019225 learned_addr learned my addr 172.21.15.37:0/2061019225 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c6bfff640 1 --2- 172.21.15.37:0/2061019225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c70828640 1 --2- 172.21.15.37:0/2061019225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b2fd0 0x5566058bbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c71029640 1 -- 172.21.15.37:0/2061019225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 msgr2=0x5566058c3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c71029640 1 --2- 172.21.15.37:0/2061019225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058c3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c71029640 1 -- 172.21.15.37:0/2061019225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b2fd0 msgr2=0x5566058bbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c71029640 1 --2- 172.21.15.37:0/2061019225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b2fd0 0x5566058bbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c71029640 1 -- 172.21.15.37:0/2061019225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566058b44d0 con 0x5566058bc550 2024-03-26T06:31:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c70828640 1 --2- 172.21.15.37:0/2061019225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b2fd0 0x5566058bbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.017+0000 7f7c71029640 1 --2- 172.21.15.37:0/2061019225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058c3b50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00b360 tx=0x7f7c5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4186cf78ed4ebf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c527fc640 1 -- 172.21.15.37:0/2061019225 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c033030 con 0x5566058bc550 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c527fc640 1 -- 172.21.15.37:0/2061019225 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c5c00fde0 con 0x5566058bc550 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c527fc640 1 -- 172.21.15.37:0/2061019225 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c01fc00 con 0x5566058bc550 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c75674740 1 -- 172.21.15.37:0/2061019225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 msgr2=0x5566058c3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c75674740 1 --2- 172.21.15.37:0/2061019225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058c3b50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00b360 tx=0x7f7c5c0042f0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c75674740 1 -- 172.21.15.37:0/2061019225 shutdown_connections 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c75674740 1 --2- 172.21.15.37:0/2061019225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058bc550 0x5566058c3b50 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c75674740 1 --2- 172.21.15.37:0/2061019225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058b3d40 0x5566058c3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c75674740 1 --2- 172.21.15.37:0/2061019225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b2fd0 0x5566058bbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c75674740 1 -- 172.21.15.37:0/2061019225 >> 172.21.15.37:0/2061019225 conn(0x55660587a5b0 msgr2=0x5566058ad7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c75674740 1 -- 172.21.15.37:0/2061019225 shutdown_connections 2024-03-26T06:31:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c75674740 1 -- 172.21.15.37:0/2061019225 wait complete. 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.018+0000 7f7c75674740 1 -- start start 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c75674740 1 Processor -- start 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c75674740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058c3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c75674740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b3d40 0x5566058b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c75674740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058bc550 0x5566058b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c75674740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1adfa0 con 0x5566058b2fd0 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c75674740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1ad940 con 0x5566058b3d40 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c75674740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1aaf40 con 0x5566058bc550 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c75674740 10 client.0 _ll_register_callbacks cb 0x7f7c6000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c70828640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c70828640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058c3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43196/0 (socket says 172.21.15.37:43196) 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c70828640 1 -- 172.21.15.37:0/2147976408 learned_addr learned my addr 172.21.15.37:0/2147976408 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c6bfff640 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b3d40 0x5566058b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c71029640 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058bc550 0x5566058b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c70828640 1 -- 172.21.15.37:0/2147976408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058bc550 msgr2=0x5566058b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c70828640 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058bc550 0x5566058b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c70828640 1 -- 172.21.15.37:0/2147976408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b3d40 msgr2=0x5566058b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c70828640 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b3d40 0x5566058b5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c70828640 1 -- 172.21.15.37:0/2147976408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566058b6aa0 con 0x5566058b2fd0 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.019+0000 7f7c71029640 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058bc550 0x5566058b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c70828640 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058c3880 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009ca0 tx=0x7f7c64032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b782b214ed0b524 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2147976408 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64039040 con 0x5566058b2fd0 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2147976408 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f7c64032790 con 0x5566058b2fd0 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c75674740 1 -- 172.21.15.37:0/2147976408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c5c00fde0 con 0x5566058b2fd0 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2147976408 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c64056050 con 0x5566058b2fd0 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c75674740 10 client.16161 fetch_fsmap learned FSMap version 40 2024-03-26T06:31:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c75674740 1 -- 172.21.15.37:0/2147976408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556605923f50 con 0x5566058b2fd0 2024-03-26T06:31:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2147976408 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c5c00fde0 con 0x5566058b2fd0 2024-03-26T06:31:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2147976408 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7c640044e0 con 0x5566058b2fd0 2024-03-26T06:31:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c75674740 10 client.16161 fetch_fsmap finished waiting for FSMap version 40 2024-03-26T06:31:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c75674740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:31:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c75674740 20 client.16161 populate_metadata read hostname 'smithi037' 2024-03-26T06:31:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c75674740 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5566059246f0 0x556605926b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c75674740 4 client.16161 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:31:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.020+0000 7f7c75674740 1 -- 172.21.15.37:0/2147976408 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55660581ee80 con 0x5566059246f0 2024-03-26T06:31:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.021+0000 7f7c6bfff640 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5566059246f0 0x556605926b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.021+0000 7f7c6bfff640 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5566059246f0 0x556605926b10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=816e2edecbd7b81f server_cookie=8f3d7926dcc658bf in_seq=0 out_seq=0 2024-03-26T06:31:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.021+0000 7f7c4bfff640 10 client.16161 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c4bfff640 1 -- 172.21.15.37:0/2147976408 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55660581ee80 con 0x5566059246f0 2024-03-26T06:31:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c4bfff640 10 client.16161 handle_command_reply: tid=0 2024-03-26T06:31:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 1 client.16161 shutdown 2024-03-26T06:31:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 1 -- 172.21.15.37:0/2147976408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 msgr2=0x5566058c3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058c3880 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009ca0 tx=0x7f7c64032e70 comp rx=0 tx=0).stop 2024-03-26T06:31:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 1 -- 172.21.15.37:0/2147976408 shutdown_connections 2024-03-26T06:31:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5566059246f0 0x556605926b10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c4bfff640 0 client.16161 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566058bc550 0x5566058b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5566058b3d40 0x5566058b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 1 --2- 172.21.15.37:0/2147976408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566058b2fd0 0x5566058c3880 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 1 -- 172.21.15.37:0/2147976408 >> 172.21.15.37:0/2147976408 conn(0x55660587a5b0 msgr2=0x5566058b7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 1 -- 172.21.15.37:0/2147976408 shutdown_connections 2024-03-26T06:31:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 1 -- 172.21.15.37:0/2147976408 wait complete. 2024-03-26T06:31:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.057+0000 7f7c75674740 20 client.16161 trim_cache size 0 max 16384 2024-03-26T06:31:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:31:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:31:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:31:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:31:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 -- 172.21.15.37:0/854858235 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7c540a2a10 msgr2=0x7f7c540a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7c540a2a10 0x7f7c540a4ed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c005fd0 tx=0x7f7c5c00bd60 comp rx=0 tx=0).stop 2024-03-26T06:31:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 -- 172.21.15.37:0/854858235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c6c100e00 msgr2=0x7f7c6c198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c6c100e00 0x7f7c6c198f10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000e9f0 tx=0x7f7c6000edf0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 -- 172.21.15.37:0/854858235 shutdown_connections 2024-03-26T06:31:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7c540a2a10 0x7f7c540a4ed0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c105190 0x7f7c6c1a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c6c100e00 0x7f7c6c198f10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 --2- 172.21.15.37:0/854858235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c6c100440 0x7f7c6c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 -- 172.21.15.37:0/854858235 >> 172.21.15.37:0/854858235 conn(0x7f7c6c077040 msgr2=0x7f7c6c0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 -- 172.21.15.37:0/854858235 shutdown_connections 2024-03-26T06:31:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:08.058+0000 7f7c72a8c640 1 -- 172.21.15.37:0/854858235 wait complete. 2024-03-26T06:31:08.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:07 smithi037 ceph-mon[31974]: pgmap v250: 97 pgs: 36 active+clean+snaptrim_wait, 41 active+clean, 20 active+clean+snaptrim; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-26T06:31:08.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:07 smithi066 ceph-mon[36504]: pgmap v250: 97 pgs: 36 active+clean+snaptrim_wait, 41 active+clean, 20 active+clean+snaptrim; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-26T06:31:08.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:07 smithi149 ceph-mon[36681]: pgmap v250: 97 pgs: 36 active+clean+snaptrim_wait, 41 active+clean, 20 active+clean+snaptrim; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-26T06:31:08.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 298640 4.25 MB/sec execute 283 sec latency 1050.347 ms 2024-03-26T06:31:09.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 298640 4.24 MB/sec execute 284 sec latency 2050.456 ms 2024-03-26T06:31:10.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 298640 4.22 MB/sec execute 285 sec latency 3050.497 ms 2024-03-26T06:31:11.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:10 smithi066 ceph-mon[36504]: pgmap v251: 97 pgs: 36 active+clean+snaptrim_wait, 41 active+clean, 20 active+clean+snaptrim; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-26T06:31:11.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:10 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:31:11.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:10 smithi149 ceph-mon[36681]: pgmap v251: 97 pgs: 36 active+clean+snaptrim_wait, 41 active+clean, 20 active+clean+snaptrim; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-26T06:31:11.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:10 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:31:11.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:10 smithi037 ceph-mon[31974]: pgmap v251: 97 pgs: 36 active+clean+snaptrim_wait, 41 active+clean, 20 active+clean+snaptrim; 107 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-26T06:31:11.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:10 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:31:11.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 298764 4.21 MB/sec execute 286 sec latency 3849.150 ms 2024-03-26T06:31:12.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:11 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:31:12.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:11 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:31:12.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:11 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:31:12.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:11 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:31:12.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:11 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:31:12.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:11 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:31:12.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:11 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:31:12.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:11 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:31:12.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:11 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:31:12.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 298764 4.20 MB/sec execute 287 sec latency 1120.170 ms 2024-03-26T06:31:13.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:12 smithi066 ceph-mon[36504]: pgmap v252: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 241 op/s 2024-03-26T06:31:13.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:12 smithi149 ceph-mon[36681]: pgmap v252: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 241 op/s 2024-03-26T06:31:13.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:12 smithi037 ceph-mon[31974]: pgmap v252: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 241 op/s 2024-03-26T06:31:13.513 INFO:tasks.workunit.client.0.smithi037.stdout: 1 298764 4.18 MB/sec execute 288 sec latency 2120.287 ms 2024-03-26T06:31:14.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 298764 4.17 MB/sec execute 289 sec latency 3120.377 ms 2024-03-26T06:31:15.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:14 smithi066 ceph-mon[36504]: pgmap v253: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 241 op/s 2024-03-26T06:31:15.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:14 smithi066 ceph-mon[36504]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T06:31:15.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:14 smithi149 ceph-mon[36681]: pgmap v253: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 241 op/s 2024-03-26T06:31:15.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:14 smithi149 ceph-mon[36681]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T06:31:15.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:14 smithi037 ceph-mon[31974]: pgmap v253: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 241 op/s 2024-03-26T06:31:15.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:14 smithi037 ceph-mon[31974]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T06:31:15.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 298764 4.15 MB/sec execute 290 sec latency 4120.477 ms 2024-03-26T06:31:16.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:15 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:31:16.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:15 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:31:16.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:15 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:31:16.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 299116 4.14 MB/sec execute 291 sec latency 4921.164 ms 2024-03-26T06:31:17.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:16 smithi066 ceph-mon[36504]: pgmap v255: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T06:31:17.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:16 smithi149 ceph-mon[36681]: pgmap v255: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T06:31:17.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:16 smithi037 ceph-mon[31974]: pgmap v255: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T06:31:17.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 301517 4.18 MB/sec execute 292 sec latency 15.924 ms 2024-03-26T06:31:18.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 302189 4.17 MB/sec execute 293 sec latency 333.907 ms 2024-03-26T06:31:19.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:18 smithi066 ceph-mon[36504]: pgmap v256: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T06:31:19.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:18 smithi149 ceph-mon[36681]: pgmap v256: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T06:31:19.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:18 smithi037 ceph-mon[31974]: pgmap v256: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T06:31:19.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 302189 4.16 MB/sec execute 294 sec latency 1333.955 ms 2024-03-26T06:31:20.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 302189 4.15 MB/sec execute 295 sec latency 2334.061 ms 2024-03-26T06:31:21.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:20 smithi066 ceph-mon[36504]: pgmap v257: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T06:31:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.149+0000 7f66f7a80640 1 -- 172.21.15.37:0/1308197018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f006a710 msgr2=0x7f66f0105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.149+0000 7f66f7a80640 1 --2- 172.21.15.37:0/1308197018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f006a710 0x7f66f0105f80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009900 tx=0x7f66e002bc30 comp rx=0 tx=0).stop 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.149+0000 7f66f7a80640 1 -- 172.21.15.37:0/1308197018 shutdown_connections 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.149+0000 7f66f7a80640 1 --2- 172.21.15.37:0/1308197018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f01066b0 0x7f66f010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.149+0000 7f66f7a80640 1 --2- 172.21.15.37:0/1308197018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f006a710 0x7f66f0105f80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.149+0000 7f66f7a80640 1 --2- 172.21.15.37:0/1308197018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f0069d60 0x7f66f006a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.149+0000 7f66f7a80640 1 -- 172.21.15.37:0/1308197018 >> 172.21.15.37:0/1308197018 conn(0x7f66f0076fc0 msgr2=0x7f66f00773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.149+0000 7f66f7a80640 1 -- 172.21.15.37:0/1308197018 shutdown_connections 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.149+0000 7f66f7a80640 1 -- 172.21.15.37:0/1308197018 wait complete. 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f7a80640 1 Processor -- start 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f7a80640 1 -- start start 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f7a80640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0069d60 0x7f66f019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f7a80640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f01066b0 0x7f66f019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f7a80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f0197910 0x7f66f0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f7a80640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0111690 con 0x7f66f0197910 2024-03-26T06:31:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f7a80640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0111510 con 0x7f66f0069d60 2024-03-26T06:31:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f7a80640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0111810 con 0x7f66f01066b0 2024-03-26T06:31:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f601d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f0197910 0x7f66f0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f601d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f0197910 0x7f66f0197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49122/0 (socket says 172.21.15.37:49122) 2024-03-26T06:31:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f601d640 1 -- 172.21.15.37:0/3130177525 learned_addr learned my addr 172.21.15.37:0/3130177525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f581c640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0069d60 0x7f66f019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f501b640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f01066b0 0x7f66f019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f601d640 1 -- 172.21.15.37:0/3130177525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f01066b0 msgr2=0x7f66f019dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f601d640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f01066b0 0x7f66f019dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f601d640 1 -- 172.21.15.37:0/3130177525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0069d60 msgr2=0x7f66f019d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f601d640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0069d60 0x7f66f019d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f601d640 1 -- 172.21.15.37:0/3130177525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0198430 con 0x7f66f0197910 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.150+0000 7f66f581c640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0069d60 0x7f66f019d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.151+0000 7f66f601d640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f0197910 0x7f66f0197dc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f66e800eb10 tx=0x7f66e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.151+0000 7f66e6ffd640 1 -- 172.21.15.37:0/3130177525 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8004b20 con 0x7f66f0197910 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.151+0000 7f66e6ffd640 1 -- 172.21.15.37:0/3130177525 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66e8004cc0 con 0x7f66f0197910 2024-03-26T06:31:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.151+0000 7f66f7a80640 1 -- 172.21.15.37:0/3130177525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f0198720 con 0x7f66f0197910 2024-03-26T06:31:21.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.151+0000 7f66e6ffd640 1 -- 172.21.15.37:0/3130177525 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8010760 con 0x7f66f0197910 2024-03-26T06:31:21.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.151+0000 7f66f7a80640 1 -- 172.21.15.37:0/3130177525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f01a4920 con 0x7f66f0197910 2024-03-26T06:31:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.153+0000 7f66e6ffd640 1 -- 172.21.15.37:0/3130177525 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f66e8005020 con 0x7f66f0197910 2024-03-26T06:31:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66fa65c740 1 Processor -- start 2024-03-26T06:31:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66fa65c740 1 -- start start 2024-03-26T06:31:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66e6ffd640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f66dc0a2a10 0x7f66dc0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 0x56456bf81a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 0x56456bf82450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf8d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66fa65c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456bee65f0 con 0x56456bf81fd0 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66fa65c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456be8f930 con 0x56456bf8b550 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66fa65c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456bee2690 con 0x56456bf82d40 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66f501b640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f66dc0a2a10 0x7f66dc0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66f601d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf8d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66f501b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 0x56456bf82450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66f581c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 0x56456bf81a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66f501b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 0x56456bf82450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49132/0 (socket says 172.21.15.37:49132) 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66f601d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf8d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:58840/0 (socket says 172.21.15.37:58840) 2024-03-26T06:31:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66f501b640 1 -- 172.21.15.37:0/2851739689 learned_addr learned my addr 172.21.15.37:0/2851739689 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66f581c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 0x56456bf81a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:33504/0 (socket says 172.21.15.37:33504) 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.154+0000 7f66e6ffd640 1 -- 172.21.15.37:0/3130177525 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f66e80cf580 con 0x7f66f0197910 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.155+0000 7f66f581c640 1 -- 172.21.15.37:0/2851739689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 msgr2=0x56456bf8d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.155+0000 7f66f581c640 1 --2- 172.21.15.37:0/2851739689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf8d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.155+0000 7f66f581c640 1 -- 172.21.15.37:0/2851739689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 msgr2=0x56456bf82450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.155+0000 7f66f581c640 1 --2- 172.21.15.37:0/2851739689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 0x56456bf82450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.155+0000 7f66f581c640 1 -- 172.21.15.37:0/2851739689 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56456bbfb210 con 0x56456bf82d40 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.155+0000 7f66f601d640 1 --2- 172.21.15.37:0/2851739689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf8d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.155+0000 7f66f501b640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f66dc0a2a10 0x7f66dc0a4ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66e0005ec0 tx=0x7f66e0005dd0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.155+0000 7f66f581c640 1 --2- 172.21.15.37:0/2851739689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 0x56456bf81a90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66f0198a50 tx=0x7f66ec00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3590e8cda3052e25 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.155+0000 7f66dbfff640 1 -- 172.21.15.37:0/2851739689 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec015070 con 0x56456bf82d40 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.155+0000 7f66dbfff640 1 -- 172.21.15.37:0/2851739689 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66ec01a940 con 0x56456bf82d40 2024-03-26T06:31:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.155+0000 7f66dbfff640 1 -- 172.21.15.37:0/2851739689 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0118b0 con 0x56456bf82d40 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.156+0000 7f66fa65c740 1 -- 172.21.15.37:0/2851739689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 msgr2=0x56456bf81a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.156+0000 7f66fa65c740 1 --2- 172.21.15.37:0/2851739689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 0x56456bf81a90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66f0198a50 tx=0x7f66ec00f730 comp rx=0 tx=0).stop 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.156+0000 7f66fa65c740 1 -- 172.21.15.37:0/2851739689 shutdown_connections 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.156+0000 7f66fa65c740 1 --2- 172.21.15.37:0/2851739689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf8d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.156+0000 7f66fa65c740 1 --2- 172.21.15.37:0/2851739689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 0x56456bf82450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.156+0000 7f66fa65c740 1 --2- 172.21.15.37:0/2851739689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 0x56456bf81a90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.156+0000 7f66fa65c740 1 -- 172.21.15.37:0/2851739689 >> 172.21.15.37:0/2851739689 conn(0x56456bf495b0 msgr2=0x56456bf7ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.156+0000 7f66fa65c740 1 -- 172.21.15.37:0/2851739689 shutdown_connections 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.156+0000 7f66fa65c740 1 -- 172.21.15.37:0/2851739689 wait complete. 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.156+0000 7f66fa65c740 1 -- start start 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66fa65c740 1 Processor -- start 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf81fd0 0x56456bf88540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf82d40 0x56456bf88a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf88fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66fa65c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456bf8e280 con 0x56456bf82d40 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66fa65c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456bea2d90 con 0x56456bf8b550 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66fa65c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456bf8e400 con 0x56456bf81fd0 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66fa65c740 10 client.0 _ll_register_callbacks cb 0x56456bbfaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66f501b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf82d40 0x56456bf88a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66f501b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf82d40 0x56456bf88a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49142/0 (socket says 172.21.15.37:49142) 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66f501b640 1 -- 172.21.15.37:0/3716360465 learned_addr learned my addr 172.21.15.37:0/3716360465 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66f581c640 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf81fd0 0x56456bf88540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66f601d640 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf88fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66f501b640 1 -- 172.21.15.37:0/3716360465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf81fd0 msgr2=0x56456bf88540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66f501b640 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf81fd0 0x56456bf88540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66f501b640 1 -- 172.21.15.37:0/3716360465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 msgr2=0x56456bf88fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66f501b640 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf88fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.157+0000 7f66f501b640 1 -- 172.21.15.37:0/3716360465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56456bcc7620 con 0x56456bf82d40 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66f501b640 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf82d40 0x56456bf88a80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f66e002c170 tx=0x7f66e0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985c6f10578ff419 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66d97fa640 1 -- 172.21.15.37:0/3716360465 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0042640 con 0x56456bf82d40 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66d97fa640 1 -- 172.21.15.37:0/3716360465 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f66e00427e0 con 0x56456bf82d40 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66fa65c740 1 -- 172.21.15.37:0/3716360465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66ec01a940 con 0x56456bf82d40 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66d97fa640 1 -- 172.21.15.37:0/3716360465 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66e005c050 con 0x56456bf82d40 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66fa65c740 10 client.16173 fetch_fsmap learned FSMap version 41 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66fa65c740 1 -- 172.21.15.37:0/3716360465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56456bf9a510 con 0x56456bf82d40 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66d97fa640 1 -- 172.21.15.37:0/3716360465 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66ec01a940 con 0x56456bf82d40 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66d97fa640 1 -- 172.21.15.37:0/3716360465 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f66e0034880 con 0x56456bf82d40 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66fa65c740 10 client.16173 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66fa65c740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:31:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66fa65c740 20 client.16173 populate_metadata read hostname 'smithi037' 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66fa65c740 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56456bf9b9d0 0x56456bf9de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66fa65c740 4 client.16173 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.158+0000 7f66fa65c740 1 -- 172.21.15.37:0/3716360465 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56456bef3b30 con 0x56456bf9b9d0 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.159+0000 7f66f581c640 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56456bf9b9d0 0x56456bf9de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.159+0000 7f66d97fa640 10 client.16173 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.159+0000 7f66f581c640 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56456bf9b9d0 0x56456bf9de00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56074b9819cc00c7 server_cookie=32216936968003f9 in_seq=0 out_seq=0 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66d97fa640 1 -- 172.21.15.37:0/3716360465 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56456bef3b30 con 0x56456bf9b9d0 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66d97fa640 10 client.16173 handle_command_reply: tid=0 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 1 client.16173 shutdown 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 1 -- 172.21.15.37:0/3716360465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf82d40 msgr2=0x56456bf88a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf82d40 0x56456bf88a80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f66e002c170 tx=0x7f66e0000bc0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 1 -- 172.21.15.37:0/3716360465 shutdown_connections 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56456bf9b9d0 0x56456bf9de00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf88fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf82d40 0x56456bf88a80 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66d97fa640 0 client.16173 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 1 --2- 172.21.15.37:0/3716360465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf81fd0 0x56456bf88540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 1 -- 172.21.15.37:0/3716360465 >> 172.21.15.37:0/3716360465 conn(0x56456bf495b0 msgr2=0x56456bf7ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 1 -- 172.21.15.37:0/3716360465 shutdown_connections 2024-03-26T06:31:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 1 -- 172.21.15.37:0/3716360465 wait complete. 2024-03-26T06:31:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.160+0000 7f66fa65c740 20 client.16173 trim_cache size 0 max 16384 2024-03-26T06:31:21.172 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:20 smithi149 ceph-mon[36681]: pgmap v257: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.173+0000 7f66fa65c740 1 Processor -- start 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.173+0000 7f66fa65c740 1 -- start start 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.173+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf81fd0 0x56456bf857a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.173+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf82d40 0x56456bf85ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.173+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf8b550 0x56456bf89c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.173+0000 7f66fa65c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01acc20 con 0x56456bf8b550 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.173+0000 7f66fa65c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01ae5d0 con 0x56456bf82d40 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.173+0000 7f66fa65c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01ad280 con 0x56456bf81fd0 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.173+0000 7f66f601d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf8b550 0x56456bf89c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.173+0000 7f66f601d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf8b550 0x56456bf89c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49152/0 (socket says 172.21.15.37:49152) 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.173+0000 7f66f601d640 1 -- 172.21.15.37:0/3182451881 learned_addr learned my addr 172.21.15.37:0/3182451881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.174+0000 7f66f581c640 1 --2- 172.21.15.37:0/3182451881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf81fd0 0x56456bf857a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.174+0000 7f66f501b640 1 --2- 172.21.15.37:0/3182451881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf82d40 0x56456bf85ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.174+0000 7f66f601d640 1 -- 172.21.15.37:0/3182451881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf81fd0 msgr2=0x56456bf857a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.174+0000 7f66f601d640 1 --2- 172.21.15.37:0/3182451881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf81fd0 0x56456bf857a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.174+0000 7f66f601d640 1 -- 172.21.15.37:0/3182451881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf82d40 msgr2=0x56456bf85ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.174+0000 7f66f601d640 1 --2- 172.21.15.37:0/3182451881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf82d40 0x56456bf85ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.174+0000 7f66f601d640 1 -- 172.21.15.37:0/3182451881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56456bf812a0 con 0x56456bf8b550 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.174+0000 7f66f601d640 1 --2- 172.21.15.37:0/3182451881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf8b550 0x56456bf89c60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f66e8005880 tx=0x7f66e809ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f16b8177cc79b9c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.174+0000 7f66db7fe640 1 -- 172.21.15.37:0/3182451881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e809c280 con 0x56456bf8b550 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.174+0000 7f66db7fe640 1 -- 172.21.15.37:0/3182451881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66e8066190 con 0x56456bf8b550 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.174+0000 7f66db7fe640 1 -- 172.21.15.37:0/3182451881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e80ccef0 con 0x56456bf8b550 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 -- 172.21.15.37:0/3182451881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf8b550 msgr2=0x56456bf89c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 --2- 172.21.15.37:0/3182451881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf8b550 0x56456bf89c60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f66e8005880 tx=0x7f66e809ba80 comp rx=0 tx=0).stop 2024-03-26T06:31:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 -- 172.21.15.37:0/3182451881 shutdown_connections 2024-03-26T06:31:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 --2- 172.21.15.37:0/3182451881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf8b550 0x56456bf89c60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 --2- 172.21.15.37:0/3182451881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf82d40 0x56456bf85ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 --2- 172.21.15.37:0/3182451881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf81fd0 0x56456bf857a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 -- 172.21.15.37:0/3182451881 >> 172.21.15.37:0/3182451881 conn(0x56456bf495b0 msgr2=0x56456bf86e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 -- 172.21.15.37:0/3182451881 shutdown_connections 2024-03-26T06:31:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 -- 172.21.15.37:0/3182451881 wait complete. 2024-03-26T06:31:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 -- start start 2024-03-26T06:31:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 Processor -- start 2024-03-26T06:31:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 0x56456bfec320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 0x56456bfec860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf919d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01adf40 con 0x56456bf81fd0 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01ad8e0 con 0x56456bf8b550 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01aaee0 con 0x56456bf82d40 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.175+0000 7f66fa65c740 10 client.0 _ll_register_callbacks cb 0x7f66ec00b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f581c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 0x56456bfec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f581c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 0x56456bfec320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49164/0 (socket says 172.21.15.37:49164) 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f581c640 1 -- 172.21.15.37:0/4060404658 learned_addr learned my addr 172.21.15.37:0/4060404658 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f501b640 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 0x56456bfec860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f601d640 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf919d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f581c640 1 -- 172.21.15.37:0/4060404658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 msgr2=0x56456bfec860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f581c640 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 0x56456bfec860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f581c640 1 -- 172.21.15.37:0/4060404658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 msgr2=0x56456bf919d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f581c640 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf919d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f581c640 1 -- 172.21.15.37:0/4060404658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56456bf920c0 con 0x56456bf81fd0 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f501b640 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 0x56456bfec860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66f581c640 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 0x56456bfec320 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f66f0198a50 tx=0x7f66ec00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bcc497392aae24 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66d97fa640 1 -- 172.21.15.37:0/4060404658 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec02a070 con 0x56456bf81fd0 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66d97fa640 1 -- 172.21.15.37:0/4060404658 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f66ec005450 con 0x56456bf81fd0 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66fa65c740 1 -- 172.21.15.37:0/4060404658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66e8066190 con 0x56456bf81fd0 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66d97fa640 1 -- 172.21.15.37:0/4060404658 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66ec036050 con 0x56456bf81fd0 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66fa65c740 10 client.16179 fetch_fsmap learned FSMap version 41 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66fa65c740 1 -- 172.21.15.37:0/4060404658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56456bff3b30 con 0x56456bf81fd0 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.176+0000 7f66d97fa640 1 -- 172.21.15.37:0/4060404658 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66e8066190 con 0x56456bf81fd0 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.177+0000 7f66d97fa640 1 -- 172.21.15.37:0/4060404658 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f66ec026ce0 con 0x56456bf81fd0 2024-03-26T06:31:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.177+0000 7f66fa65c740 10 client.16179 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T06:31:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.177+0000 7f66fa65c740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:31:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.177+0000 7f66fa65c740 20 client.16179 populate_metadata read hostname 'smithi037' 2024-03-26T06:31:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.177+0000 7f66fa65c740 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56456bff3fb0 0x56456bff63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.177+0000 7f66fa65c740 4 client.16179 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:31:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.177+0000 7f66fa65c740 1 -- 172.21.15.37:0/4060404658 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56456beede80 con 0x56456bff3fb0 2024-03-26T06:31:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.177+0000 7f66f501b640 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56456bff3fb0 0x56456bff63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.177+0000 7f66f501b640 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56456bff3fb0 0x56456bff63d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b678d26692ae4fc server_cookie=5382d33b15e99c50 in_seq=0 out_seq=0 2024-03-26T06:31:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.177+0000 7f66d97fa640 10 client.16179 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:21.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:20 smithi037 ceph-mon[31974]: pgmap v257: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T06:31:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66d97fa640 1 -- 172.21.15.37:0/4060404658 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56456beede80 con 0x56456bff3fb0 2024-03-26T06:31:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66d97fa640 10 client.16179 handle_command_reply: tid=0 2024-03-26T06:31:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 1 client.16179 shutdown 2024-03-26T06:31:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 1 -- 172.21.15.37:0/4060404658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 msgr2=0x56456bfec320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 0x56456bfec320 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f66f0198a50 tx=0x7f66ec00b780 comp rx=0 tx=0).stop 2024-03-26T06:31:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 1 -- 172.21.15.37:0/4060404658 shutdown_connections 2024-03-26T06:31:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x56456bff3fb0 0x56456bff63d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56456bf8b550 0x56456bf919d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56456bf82d40 0x56456bfec860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 1 --2- 172.21.15.37:0/4060404658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56456bf81fd0 0x56456bfec320 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66d97fa640 0 client.16179 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 1 -- 172.21.15.37:0/4060404658 >> 172.21.15.37:0/4060404658 conn(0x56456bf495b0 msgr2=0x56456bf7b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 1 -- 172.21.15.37:0/4060404658 shutdown_connections 2024-03-26T06:31:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 1 -- 172.21.15.37:0/4060404658 wait complete. 2024-03-26T06:31:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66fa65c740 20 client.16179 trim_cache size 0 max 16384 2024-03-26T06:31:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:31:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:31:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:31:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:31:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 -- 172.21.15.37:0/3130177525 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f66dc0a2a10 msgr2=0x7f66dc0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f66dc0a2a10 0x7f66dc0a4ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66e0005ec0 tx=0x7f66e0005dd0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 -- 172.21.15.37:0/3130177525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f0197910 msgr2=0x7f66f0197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f0197910 0x7f66f0197dc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f66e800eb10 tx=0x7f66e800c2d0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 -- 172.21.15.37:0/3130177525 shutdown_connections 2024-03-26T06:31:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f66dc0a2a10 0x7f66dc0a4ed0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f0197910 0x7f66f0197dc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f01066b0 0x7f66f019dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 --2- 172.21.15.37:0/3130177525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0069d60 0x7f66f019d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 -- 172.21.15.37:0/3130177525 >> 172.21.15.37:0/3130177525 conn(0x7f66f0076fc0 msgr2=0x7f66f00fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 -- 172.21.15.37:0/3130177525 shutdown_connections 2024-03-26T06:31:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:21.207+0000 7f66f7a80640 1 -- 172.21.15.37:0/3130177525 wait complete. 2024-03-26T06:31:21.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 302245 4.13 MB/sec execute 296 sec latency 3132.262 ms 2024-03-26T06:31:22.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 302245 4.12 MB/sec execute 297 sec latency 1164.341 ms 2024-03-26T06:31:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:22 smithi066 ceph-mon[36504]: pgmap v258: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-03-26T06:31:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:22 smithi149 ceph-mon[36681]: pgmap v258: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-03-26T06:31:23.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:22 smithi037 ceph-mon[31974]: pgmap v258: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-03-26T06:31:23.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 302245 4.10 MB/sec execute 298 sec latency 2164.429 ms 2024-03-26T06:31:24.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 302245 4.09 MB/sec execute 299 sec latency 3164.532 ms 2024-03-26T06:31:25.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:24 smithi066 ceph-mon[36504]: pgmap v259: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-03-26T06:31:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:24 smithi149 ceph-mon[36681]: pgmap v259: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-03-26T06:31:25.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:24 smithi037 ceph-mon[31974]: pgmap v259: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-03-26T06:31:25.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 302245 4.08 MB/sec execute 300 sec latency 4164.633 ms 2024-03-26T06:31:26.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:25 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:31:26.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:25 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:31:26.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:25 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:31:26.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 302595 4.07 MB/sec execute 301 sec latency 4962.212 ms 2024-03-26T06:31:27.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:26 smithi066 ceph-mon[36504]: pgmap v260: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-03-26T06:31:27.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:26 smithi149 ceph-mon[36681]: pgmap v260: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-03-26T06:31:27.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:26 smithi037 ceph-mon[31974]: pgmap v260: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-03-26T06:31:27.514 INFO:tasks.workunit.client.0.smithi037.stdout: 1 304878 4.10 MB/sec execute 302 sec latency 22.484 ms 2024-03-26T06:31:28.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 306221 4.11 MB/sec execute 303 sec latency 12.780 ms 2024-03-26T06:31:29.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:28 smithi066 ceph-mon[36504]: pgmap v261: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 922 KiB/s wr, 82 op/s 2024-03-26T06:31:29.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:28 smithi149 ceph-mon[36681]: pgmap v261: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 922 KiB/s wr, 82 op/s 2024-03-26T06:31:29.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:28 smithi037 ceph-mon[31974]: pgmap v261: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 922 KiB/s wr, 82 op/s 2024-03-26T06:31:29.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 308682 4.14 MB/sec execute 304 sec latency 14.755 ms 2024-03-26T06:31:30.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 309403 4.14 MB/sec execute 305 sec latency 270.757 ms 2024-03-26T06:31:31.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:30 smithi066 ceph-mon[36504]: pgmap v262: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 922 KiB/s wr, 82 op/s 2024-03-26T06:31:31.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:30 smithi149 ceph-mon[36681]: pgmap v262: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 922 KiB/s wr, 82 op/s 2024-03-26T06:31:31.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:30 smithi037 ceph-mon[31974]: pgmap v262: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 922 KiB/s wr, 82 op/s 2024-03-26T06:31:31.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 309768 4.13 MB/sec execute 306 sec latency 1067.193 ms 2024-03-26T06:31:32.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 312140 4.17 MB/sec execute 307 sec latency 15.572 ms 2024-03-26T06:31:33.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:32 smithi066 ceph-mon[36504]: pgmap v263: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-26T06:31:33.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:32 smithi149 ceph-mon[36681]: pgmap v263: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-26T06:31:33.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:32 smithi037 ceph-mon[31974]: pgmap v263: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-26T06:31:33.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 312822 4.16 MB/sec execute 308 sec latency 308.587 ms 2024-03-26T06:31:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.304+0000 7f4b7b57e640 1 -- 172.21.15.37:0/206255593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b74100440 msgr2=0x7f4b741008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.304+0000 7f4b7b57e640 1 --2- 172.21.15.37:0/206255593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b74100440 0x7f4b741008c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009900 tx=0x7f4b6402bc30 comp rx=0 tx=0).stop 2024-03-26T06:31:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.304+0000 7f4b7b57e640 1 -- 172.21.15.37:0/206255593 shutdown_connections 2024-03-26T06:31:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.304+0000 7f4b7b57e640 1 --2- 172.21.15.37:0/206255593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b74100e00 0x7f4b7410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.304+0000 7f4b7b57e640 1 --2- 172.21.15.37:0/206255593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b74100440 0x7f4b741008c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.304+0000 7f4b7b57e640 1 --2- 172.21.15.37:0/206255593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b74105190 0x7f4b74105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.304+0000 7f4b7b57e640 1 -- 172.21.15.37:0/206255593 >> 172.21.15.37:0/206255593 conn(0x7f4b74077040 msgr2=0x7f4b74077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.304+0000 7f4b7b57e640 1 -- 172.21.15.37:0/206255593 shutdown_connections 2024-03-26T06:31:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.305+0000 7f4b7b57e640 1 -- 172.21.15.37:0/206255593 wait complete. 2024-03-26T06:31:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.305+0000 7f4b7b57e640 1 Processor -- start 2024-03-26T06:31:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.305+0000 7f4b7b57e640 1 -- start start 2024-03-26T06:31:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.305+0000 7f4b7b57e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b74100440 0x7f4b74198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.305+0000 7f4b7b57e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b74100e00 0x7f4b74198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.305+0000 7f4b7b57e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b74105190 0x7f4b741a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.305+0000 7f4b7b57e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741106a0 con 0x7f4b74100440 2024-03-26T06:31:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.305+0000 7f4b7b57e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74110520 con 0x7f4b74100e00 2024-03-26T06:31:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.305+0000 7f4b7b57e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74110820 con 0x7f4b74105190 2024-03-26T06:31:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.305+0000 7f4b7931a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b74100440 0x7f4b74198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.305+0000 7f4b7931a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b74100440 0x7f4b74198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36316/0 (socket says 172.21.15.37:36316) 2024-03-26T06:31:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b7931a640 1 -- 172.21.15.37:0/615931219 learned_addr learned my addr 172.21.15.37:0/615931219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b78b19640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b74100e00 0x7f4b74198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b79b1b640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b74105190 0x7f4b741a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b7931a640 1 -- 172.21.15.37:0/615931219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b74105190 msgr2=0x7f4b741a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b7931a640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b74105190 0x7f4b741a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b7931a640 1 -- 172.21.15.37:0/615931219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b74100e00 msgr2=0x7f4b74198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b7931a640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b74100e00 0x7f4b74198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b7931a640 1 -- 172.21.15.37:0/615931219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b741a3fc0 con 0x7f4b74100440 2024-03-26T06:31:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b78b19640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b74100e00 0x7f4b74198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:31:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b7931a640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b74100440 0x7f4b74198a00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000d770 tx=0x7f4b7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b6a7fc640 1 -- 172.21.15.37:0/615931219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70017070 con 0x7f4b74100440 2024-03-26T06:31:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b7b57e640 1 -- 172.21.15.37:0/615931219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b741a4190 con 0x7f4b74100440 2024-03-26T06:31:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b6a7fc640 1 -- 172.21.15.37:0/615931219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b70005500 con 0x7f4b74100440 2024-03-26T06:31:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b6a7fc640 1 -- 172.21.15.37:0/615931219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70004e00 con 0x7f4b74100440 2024-03-26T06:31:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.306+0000 7f4b7b57e640 1 -- 172.21.15.37:0/615931219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b741a4660 con 0x7f4b74100440 2024-03-26T06:31:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b6a7fc640 1 -- 172.21.15.37:0/615931219 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4b70004fa0 con 0x7f4b74100440 2024-03-26T06:31:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b7ddfd740 1 Processor -- start 2024-03-26T06:31:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b7ddfd740 1 -- start start 2024-03-26T06:31:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0585d40 0x5618e0584a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0584fd0 0x5618e0585450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e058e550 0x5618e0590940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e04e95f0 con 0x5618e0584fd0 2024-03-26T06:31:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e0492930 con 0x5618e0585d40 2024-03-26T06:31:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e04e5690 con 0x5618e058e550 2024-03-26T06:31:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b7931a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0584fd0 0x5618e0585450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b79b1b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0585d40 0x5618e0584a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b7931a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0584fd0 0x5618e0585450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36322/0 (socket says 172.21.15.37:36322) 2024-03-26T06:31:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b7931a640 1 -- 172.21.15.37:0/1758811168 learned_addr learned my addr 172.21.15.37:0/1758811168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b78b19640 1 --2- 172.21.15.37:0/1758811168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e058e550 0x5618e0590940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.310+0000 7f4b6a7fc640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b600a2a10 0x7f4b600a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b79b1b640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b600a2a10 0x7f4b600a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b78b19640 1 -- 172.21.15.37:0/1758811168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0585d40 msgr2=0x5618e0584a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b78b19640 1 --2- 172.21.15.37:0/1758811168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0585d40 0x5618e0584a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b78b19640 1 -- 172.21.15.37:0/1758811168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0584fd0 msgr2=0x5618e0585450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b78b19640 1 --2- 172.21.15.37:0/1758811168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0584fd0 0x5618e0585450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b78b19640 1 -- 172.21.15.37:0/1758811168 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618e01fe210 con 0x5618e058e550 2024-03-26T06:31:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b7931a640 1 --2- 172.21.15.37:0/1758811168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0584fd0 0x5618e0585450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:31:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b6a7fc640 1 -- 172.21.15.37:0/615931219 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4b700ce780 con 0x7f4b74100440 2024-03-26T06:31:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b78b19640 1 --2- 172.21.15.37:0/1758811168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e058e550 0x5618e0590940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b6402c170 tx=0x7f4b64005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3b11212cd98502 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b79b1b640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b600a2a10 0x7f4b600a4ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009fd0 tx=0x7f4b6c004ab0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b5f7fe640 1 -- 172.21.15.37:0/1758811168 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64042070 con 0x5618e058e550 2024-03-26T06:31:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b5f7fe640 1 -- 172.21.15.37:0/1758811168 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b64004c70 con 0x5618e058e550 2024-03-26T06:31:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.311+0000 7f4b5f7fe640 1 -- 172.21.15.37:0/1758811168 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640051e0 con 0x5618e058e550 2024-03-26T06:31:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.312+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1758811168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e058e550 msgr2=0x5618e0590940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.312+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/1758811168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e058e550 0x5618e0590940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b6402c170 tx=0x7f4b64005d50 comp rx=0 tx=0).stop 2024-03-26T06:31:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.312+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1758811168 shutdown_connections 2024-03-26T06:31:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.312+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/1758811168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e058e550 0x5618e0590940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.312+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/1758811168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0584fd0 0x5618e0585450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.312+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/1758811168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0585d40 0x5618e0584a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.312+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1758811168 >> 172.21.15.37:0/1758811168 conn(0x5618e054c5b0 msgr2=0x5618e057fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.312+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1758811168 shutdown_connections 2024-03-26T06:31:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.312+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1758811168 wait complete. 2024-03-26T06:31:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.312+0000 7f4b7ddfd740 1 -- start start 2024-03-26T06:31:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.312+0000 7f4b7ddfd740 1 Processor -- start 2024-03-26T06:31:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e058b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 0x5618e058bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e0591220 con 0x5618e058e550 2024-03-26T06:31:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e04a5d90 con 0x5618e0584fd0 2024-03-26T06:31:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e05913a0 con 0x5618e0585d40 2024-03-26T06:31:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b79b1b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b79b1b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36326/0 (socket says 172.21.15.37:36326) 2024-03-26T06:31:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b79b1b640 1 -- 172.21.15.37:0/1596075950 learned_addr learned my addr 172.21.15.37:0/1596075950 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b7931a640 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e058b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b7ddfd740 10 client.0 _ll_register_callbacks cb 0x5618e01fda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:31:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.313+0000 7f4b78b19640 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 0x5618e058bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.314+0000 7f4b79b1b640 1 -- 172.21.15.37:0/1596075950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 msgr2=0x5618e058bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.314+0000 7f4b79b1b640 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 0x5618e058bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.314+0000 7f4b79b1b640 1 -- 172.21.15.37:0/1596075950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 msgr2=0x5618e058b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.314+0000 7f4b79b1b640 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e058b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.314+0000 7f4b79b1b640 1 -- 172.21.15.37:0/1596075950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618e02ca620 con 0x5618e058e550 2024-03-26T06:31:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.314+0000 7f4b78b19640 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 0x5618e058bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:31:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.314+0000 7f4b79b1b640 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058c020 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009f70 tx=0x7f4b6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38be3857bdd20ef1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.314+0000 7f4b5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:31:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.314+0000 7f4b5cff9640 1 -- 172.21.15.37:0/1596075950 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0045a0 con 0x5618e058e550 2024-03-26T06:31:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.314+0000 7f4b5cff9640 1 -- 172.21.15.37:0/1596075950 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4b6c004740 con 0x5618e058e550 2024-03-26T06:31:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.314+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1596075950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618e04f6b30 con 0x5618e058e550 2024-03-26T06:31:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.315+0000 7f4b5cff9640 1 -- 172.21.15.37:0/1596075950 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6c004b20 con 0x5618e058e550 2024-03-26T06:31:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.315+0000 7f4b7ddfd740 10 client.16191 fetch_fsmap learned FSMap version 42 2024-03-26T06:31:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.315+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1596075950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5618e059d580 con 0x5618e058e550 2024-03-26T06:31:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.315+0000 7f4b5cff9640 1 -- 172.21.15.37:0/1596075950 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5618e04f6b30 con 0x5618e058e550 2024-03-26T06:31:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.315+0000 7f4b5cff9640 1 -- 172.21.15.37:0/1596075950 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4b6c01f040 con 0x5618e058e550 2024-03-26T06:31:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.315+0000 7f4b7ddfd740 10 client.16191 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:31:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.315+0000 7f4b7ddfd740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:31:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.315+0000 7f4b7ddfd740 20 client.16191 populate_metadata read hostname 'smithi037' 2024-03-26T06:31:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.316+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5618e059ea40 0x5618e05a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.316+0000 7f4b7ddfd740 4 client.16191 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:31:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.316+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1596075950 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618e04f0e80 con 0x5618e059ea40 2024-03-26T06:31:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.316+0000 7f4b7931a640 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5618e059ea40 0x5618e05a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.316+0000 7f4b5cff9640 10 client.16191 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.316+0000 7f4b7931a640 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5618e059ea40 0x5618e05a0e30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a215176c2517bcaa server_cookie=cf86feec117bd994 in_seq=0 out_seq=0 2024-03-26T06:31:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b5cff9640 1 -- 172.21.15.37:0/1596075950 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5618e04f0e80 con 0x5618e059ea40 2024-03-26T06:31:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b5cff9640 10 client.16191 handle_command_reply: tid=0 2024-03-26T06:31:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b7ddfd740 1 client.16191 shutdown 2024-03-26T06:31:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1596075950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 msgr2=0x5618e058c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058c020 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009f70 tx=0x7f4b6c004a40 comp rx=0 tx=0).stop 2024-03-26T06:31:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1596075950 shutdown_connections 2024-03-26T06:31:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5618e059ea40 0x5618e05a0e30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058c020 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 0x5618e058bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b5cff9640 0 client.16191 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/1596075950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e058b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1596075950 >> 172.21.15.37:0/1596075950 conn(0x5618e054c5b0 msgr2=0x5618e0581a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.317+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1596075950 shutdown_connections 2024-03-26T06:31:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.318+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/1596075950 wait complete. 2024-03-26T06:31:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.318+0000 7f4b7ddfd740 20 client.16191 trim_cache size 0 max 16384 2024-03-26T06:31:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.331+0000 7f4b7ddfd740 1 Processor -- start 2024-03-26T06:31:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.331+0000 7f4b7ddfd740 1 -- start start 2024-03-26T06:31:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.331+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e058dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.331+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 0x5618e0589e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.331+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.331+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741aa9b0 con 0x5618e058e550 2024-03-26T06:31:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.331+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741ac330 con 0x5618e0584fd0 2024-03-26T06:31:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.331+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741ab010 con 0x5618e0585d40 2024-03-26T06:31:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.331+0000 7f4b79b1b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b79b1b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36332/0 (socket says 172.21.15.37:36332) 2024-03-26T06:31:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b79b1b640 1 -- 172.21.15.37:0/4158136306 learned_addr learned my addr 172.21.15.37:0/4158136306 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b7931a640 1 --2- 172.21.15.37:0/4158136306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e058dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b78b19640 1 --2- 172.21.15.37:0/4158136306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 0x5618e0589e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b79b1b640 1 -- 172.21.15.37:0/4158136306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 msgr2=0x5618e0589e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b79b1b640 1 --2- 172.21.15.37:0/4158136306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 0x5618e0589e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b79b1b640 1 -- 172.21.15.37:0/4158136306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 msgr2=0x5618e058dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b79b1b640 1 --2- 172.21.15.37:0/4158136306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e058dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b79b1b640 1 -- 172.21.15.37:0/4158136306 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618e0589af0 con 0x5618e058e550 2024-03-26T06:31:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b78b19640 1 --2- 172.21.15.37:0/4158136306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 0x5618e0589e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:31:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b79b1b640 1 --2- 172.21.15.37:0/4158136306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058a3b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c006010 tx=0x7f4b6c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d95fe55ae9e4d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b5effd640 1 -- 172.21.15.37:0/4158136306 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0053d0 con 0x5618e058e550 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b5effd640 1 -- 172.21.15.37:0/4158136306 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c03a030 con 0x5618e058e550 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.332+0000 7f4b5effd640 1 -- 172.21.15.37:0/4158136306 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0046e0 con 0x5618e058e550 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.333+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4158136306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 msgr2=0x5618e058a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.333+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/4158136306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058a3b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c006010 tx=0x7f4b6c000f40 comp rx=0 tx=0).stop 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.333+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4158136306 shutdown_connections 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.333+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/4158136306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e058e550 0x5618e058a3b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.333+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/4158136306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e0585d40 0x5618e0589e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.333+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/4158136306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e058dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.333+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4158136306 >> 172.21.15.37:0/4158136306 conn(0x5618e054c5b0 msgr2=0x5618e057f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.333+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4158136306 shutdown_connections 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.333+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4158136306 wait complete. 2024-03-26T06:31:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.333+0000 7f4b7ddfd740 1 -- start start 2024-03-26T06:31:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.333+0000 7f4b7ddfd740 1 Processor -- start 2024-03-26T06:31:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e0595810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0585d40 0x5618e0593c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b7ddfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e058e550 0x5618e0594140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741abcd0 con 0x5618e0585d40 2024-03-26T06:31:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741ab670 con 0x5618e0584fd0 2024-03-26T06:31:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b7ddfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741a8c90 con 0x5618e058e550 2024-03-26T06:31:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b7ddfd740 10 client.0 _ll_register_callbacks cb 0x7f4b64009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:31:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b78b19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0585d40 0x5618e0593c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b78b19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0585d40 0x5618e0593c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36338/0 (socket says 172.21.15.37:36338) 2024-03-26T06:31:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b78b19640 1 -- 172.21.15.37:0/4062857485 learned_addr learned my addr 172.21.15.37:0/4062857485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b7931a640 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e0595810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b79b1b640 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e058e550 0x5618e0594140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b78b19640 1 -- 172.21.15.37:0/4062857485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e058e550 msgr2=0x5618e0594140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b78b19640 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e058e550 0x5618e0594140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b78b19640 1 -- 172.21.15.37:0/4062857485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 msgr2=0x5618e0595810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b78b19640 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e0595810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b78b19640 1 -- 172.21.15.37:0/4062857485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618e05947e0 con 0x5618e0585d40 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b78b19640 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0585d40 0x5618e0593c00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4b64005ec0 tx=0x7f4b64005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380a3e29f0c632ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.334+0000 7f4b5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b5cff9640 1 -- 172.21.15.37:0/4062857485 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64042040 con 0x5618e0585d40 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b5cff9640 1 -- 172.21.15.37:0/4062857485 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4b64004230 con 0x5618e0585d40 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4062857485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b6c03a030 con 0x5618e0585d40 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b5cff9640 1 -- 172.21.15.37:0/4062857485 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b64004430 con 0x5618e0585d40 2024-03-26T06:31:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b7ddfd740 10 client.16197 fetch_fsmap learned FSMap version 42 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4062857485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5618e05f5e40 con 0x5618e0585d40 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b5cff9640 1 -- 172.21.15.37:0/4062857485 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c03a030 con 0x5618e0585d40 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b5cff9640 1 -- 172.21.15.37:0/4062857485 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4b64003bf0 con 0x5618e0585d40 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b7ddfd740 10 client.16197 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b7ddfd740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b7ddfd740 20 client.16197 populate_metadata read hostname 'smithi037' 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5618e05f6170 0x5618e05f8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b7ddfd740 4 client.16197 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4062857485 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5618e046f710 con 0x5618e05f6170 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.335+0000 7f4b7931a640 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5618e05f6170 0x5618e05f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.336+0000 7f4b5cff9640 10 client.16197 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.336+0000 7f4b7931a640 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5618e05f6170 0x5618e05f8560 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=498e03766ed3f5cd server_cookie=a17d10012d59d620 in_seq=0 out_seq=0 2024-03-26T06:31:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b5cff9640 1 -- 172.21.15.37:0/4062857485 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5618e046f710 con 0x5618e05f6170 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b5cff9640 10 client.16197 handle_command_reply: tid=0 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 1 client.16197 shutdown 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4062857485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0585d40 msgr2=0x5618e0593c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0585d40 0x5618e0593c00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4b64005ec0 tx=0x7f4b64005850 comp rx=0 tx=0).stop 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4062857485 shutdown_connections 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5618e05f6170 0x5618e05f8560 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618e058e550 0x5618e0594140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618e0585d40 0x5618e0593c00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b5cff9640 0 client.16197 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 1 --2- 172.21.15.37:0/4062857485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e0584fd0 0x5618e0595810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4062857485 >> 172.21.15.37:0/4062857485 conn(0x5618e054c5b0 msgr2=0x5618e0589d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4062857485 shutdown_connections 2024-03-26T06:31:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 1 -- 172.21.15.37:0/4062857485 wait complete. 2024-03-26T06:31:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.371+0000 7f4b7ddfd740 20 client.16197 trim_cache size 0 max 16384 2024-03-26T06:31:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:31:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:31:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:31:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:31:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 -- 172.21.15.37:0/615931219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b600a2a10 msgr2=0x7f4b600a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b600a2a10 0x7f4b600a4ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009fd0 tx=0x7f4b6c004ab0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 -- 172.21.15.37:0/615931219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b74100440 msgr2=0x7f4b74198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b74100440 0x7f4b74198a00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000d770 tx=0x7f4b7000dc30 comp rx=0 tx=0).stop 2024-03-26T06:31:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 -- 172.21.15.37:0/615931219 shutdown_connections 2024-03-26T06:31:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b600a2a10 0x7f4b600a4ed0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b74105190 0x7f4b741a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b74100e00 0x7f4b74198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 --2- 172.21.15.37:0/615931219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b74100440 0x7f4b74198a00 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 -- 172.21.15.37:0/615931219 >> 172.21.15.37:0/615931219 conn(0x7f4b74077040 msgr2=0x7f4b740fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 -- 172.21.15.37:0/615931219 shutdown_connections 2024-03-26T06:31:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:34.372+0000 7f4b7b57e640 1 -- 172.21.15.37:0/615931219 wait complete. 2024-03-26T06:31:34.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 312822 4.14 MB/sec execute 309 sec latency 1308.664 ms 2024-03-26T06:31:35.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:34 smithi066 ceph-mon[36504]: pgmap v264: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-03-26T06:31:35.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:34 smithi149 ceph-mon[36681]: pgmap v264: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-03-26T06:31:35.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:34 smithi037 ceph-mon[31974]: pgmap v264: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-03-26T06:31:35.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 312822 4.13 MB/sec execute 310 sec latency 2308.761 ms 2024-03-26T06:31:36.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313134 4.12 MB/sec execute 311 sec latency 3124.878 ms 2024-03-26T06:31:37.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:36 smithi066 ceph-mon[36504]: pgmap v265: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-03-26T06:31:37.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:36 smithi149 ceph-mon[36681]: pgmap v265: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-03-26T06:31:37.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:36 smithi037 ceph-mon[31974]: pgmap v265: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-03-26T06:31:37.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 315377 4.15 MB/sec execute 312 sec latency 18.507 ms 2024-03-26T06:31:38.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 316495 4.15 MB/sec execute 313 sec latency 138.590 ms 2024-03-26T06:31:39.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:38 smithi066 ceph-mon[36504]: pgmap v266: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-03-26T06:31:39.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:38 smithi149 ceph-mon[36681]: pgmap v266: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-03-26T06:31:39.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:38 smithi037 ceph-mon[31974]: pgmap v266: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-03-26T06:31:39.515 INFO:tasks.workunit.client.0.smithi037.stdout: 1 316495 4.14 MB/sec execute 314 sec latency 1138.713 ms 2024-03-26T06:31:40.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 316495 4.13 MB/sec execute 315 sec latency 2138.786 ms 2024-03-26T06:31:41.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:40 smithi066 ceph-mon[36504]: pgmap v267: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-03-26T06:31:41.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:40 smithi149 ceph-mon[36681]: pgmap v267: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-03-26T06:31:41.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:40 smithi037 ceph-mon[31974]: pgmap v267: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-03-26T06:31:41.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 316860 4.12 MB/sec execute 316 sec latency 2932.246 ms 2024-03-26T06:31:42.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 319212 4.15 MB/sec execute 317 sec latency 20.278 ms 2024-03-26T06:31:43.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:42 smithi066 ceph-mon[36504]: pgmap v268: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2024-03-26T06:31:43.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:42 smithi149 ceph-mon[36681]: pgmap v268: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2024-03-26T06:31:43.201 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:42 smithi037 ceph-mon[31974]: pgmap v268: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2024-03-26T06:31:43.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 319969 4.15 MB/sec execute 318 sec latency 292.774 ms 2024-03-26T06:31:44.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:43 smithi066 ceph-mon[36504]: pgmap v269: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-03-26T06:31:44.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:43 smithi149 ceph-mon[36681]: pgmap v269: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-03-26T06:31:44.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:43 smithi037 ceph-mon[31974]: pgmap v269: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-03-26T06:31:44.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 319969 4.13 MB/sec execute 319 sec latency 1292.878 ms 2024-03-26T06:31:45.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 319969 4.12 MB/sec execute 320 sec latency 2293.014 ms 2024-03-26T06:31:46.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:45 smithi037 ceph-mon[31974]: pgmap v270: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-03-26T06:31:46.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:45 smithi066 ceph-mon[36504]: pgmap v270: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-03-26T06:31:46.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:45 smithi149 ceph-mon[36681]: pgmap v270: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-03-26T06:31:46.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 320319 4.11 MB/sec execute 321 sec latency 3087.031 ms 2024-03-26T06:31:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.464+0000 7fecc6159640 1 -- 172.21.15.37:0/1147175825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc0100440 msgr2=0x7fecc0100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.464+0000 7fecc6159640 1 --2- 172.21.15.37:0/1147175825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc0100440 0x7fecc0100820 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009900 tx=0x7fecb002bc30 comp rx=0 tx=0).stop 2024-03-26T06:31:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.464+0000 7fecc6159640 1 -- 172.21.15.37:0/1147175825 shutdown_connections 2024-03-26T06:31:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.464+0000 7fecc6159640 1 --2- 172.21.15.37:0/1147175825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc0105510 0x7fecc010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.464+0000 7fecc6159640 1 --2- 172.21.15.37:0/1147175825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc0100df0 0x7fecc0104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.464+0000 7fecc6159640 1 --2- 172.21.15.37:0/1147175825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc0100440 0x7fecc0100820 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.464+0000 7fecc6159640 1 -- 172.21.15.37:0/1147175825 >> 172.21.15.37:0/1147175825 conn(0x7fecc0077340 msgr2=0x7fecc0077750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:47.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.465+0000 7fecc6159640 1 -- 172.21.15.37:0/1147175825 shutdown_connections 2024-03-26T06:31:47.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.465+0000 7fecc6159640 1 -- 172.21.15.37:0/1147175825 wait complete. 2024-03-26T06:31:47.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.465+0000 7fecc6159640 1 Processor -- start 2024-03-26T06:31:47.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.465+0000 7fecc6159640 1 -- start start 2024-03-26T06:31:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecc6159640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc0100440 0x7fecc019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecc6159640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc0105510 0x7fecc019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecc6159640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc01968a0 0x7fecc0196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecc6159640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc01105c0 con 0x7fecc0100440 2024-03-26T06:31:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecc6159640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc0110440 con 0x7fecc01968a0 2024-03-26T06:31:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecc6159640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc0110740 con 0x7fecc0105510 2024-03-26T06:31:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc0100440 0x7fecc019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc0100440 0x7fecc019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34210/0 (socket says 172.21.15.37:34210) 2024-03-26T06:31:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbf7fe640 1 -- 172.21.15.37:0/660189803 learned_addr learned my addr 172.21.15.37:0/660189803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbffff640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc01968a0 0x7fecc0196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbeffd640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc0105510 0x7fecc019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbf7fe640 1 -- 172.21.15.37:0/660189803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc0105510 msgr2=0x7fecc019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc0105510 0x7fecc019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbf7fe640 1 -- 172.21.15.37:0/660189803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc01968a0 msgr2=0x7fecc0196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc01968a0 0x7fecc0196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbf7fe640 1 -- 172.21.15.37:0/660189803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc0197430 con 0x7fecc0100440 2024-03-26T06:31:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbffff640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc01968a0 0x7fecc0196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:31:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc0100440 0x7fecc019c6e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fecb0006010 tx=0x7fecb002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbcff9640 1 -- 172.21.15.37:0/660189803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0041070 con 0x7fecc0100440 2024-03-26T06:31:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecc6159640 1 -- 172.21.15.37:0/660189803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc01976c0 con 0x7fecc0100440 2024-03-26T06:31:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbcff9640 1 -- 172.21.15.37:0/660189803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecb0004450 con 0x7fecc0100440 2024-03-26T06:31:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecbcff9640 1 -- 172.21.15.37:0/660189803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0003e00 con 0x7fecc0100440 2024-03-26T06:31:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.466+0000 7fecc6159640 1 -- 172.21.15.37:0/660189803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc01a3800 con 0x7fecc0100440 2024-03-26T06:31:47.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.469+0000 7fecbcff9640 1 -- 172.21.15.37:0/660189803 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fecb0044070 con 0x7fecc0100440 2024-03-26T06:31:47.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.469+0000 7fecc8e06740 1 Processor -- start 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.469+0000 7fecc8e06740 1 -- start start 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.469+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7bbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.469+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bc1e0 0x5600cc7bc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.469+0000 7fecbcff9640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7feca80a2a10 0x7feca80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.469+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7c5760 0x5600cc7c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.469+0000 7fecc8e06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600cc71e5f0 con 0x5600cc7bcf50 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.469+0000 7fecc8e06740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600cc6c7930 con 0x5600cc7c5760 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.469+0000 7fecc8e06740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600cc71a690 con 0x5600cc7bc1e0 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.469+0000 7fecbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7bbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bc1e0 0x5600cc7bc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7bbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34224/0 (socket says 172.21.15.37:34224) 2024-03-26T06:31:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbffff640 1 -- 172.21.15.37:0/3903166885 learned_addr learned my addr 172.21.15.37:0/3903166885 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbcff9640 1 -- 172.21.15.37:0/660189803 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fecb00f07b0 con 0x7fecc0100440 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbffff640 1 --2- 172.21.15.37:0/3903166885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7c5760 0x5600cc7c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbeffd640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7feca80a2a10 0x7feca80a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbf7fe640 1 -- 172.21.15.37:0/3903166885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7c5760 msgr2=0x5600cc7c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/3903166885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7c5760 0x5600cc7c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbf7fe640 1 -- 172.21.15.37:0/3903166885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 msgr2=0x5600cc7bbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/3903166885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7bbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbf7fe640 1 -- 172.21.15.37:0/3903166885 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600cc433210 con 0x5600cc7bc1e0 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbffff640 1 --2- 172.21.15.37:0/3903166885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbeffd640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7feca80a2a10 0x7feca80a4ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fecc0104e90 tx=0x7fecb4005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/3903166885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bc1e0 0x5600cc7bc660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fecb00ee010 tx=0x7fecb00b8bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d9032c615ac09ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7feca5ffb640 1 -- 172.21.15.37:0/3903166885 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb004b030 con 0x5600cc7bc1e0 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7feca5ffb640 1 -- 172.21.15.37:0/3903166885 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecb002cd20 con 0x5600cc7bc1e0 2024-03-26T06:31:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.470+0000 7feca5ffb640 1 -- 172.21.15.37:0/3903166885 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00bc660 con 0x5600cc7bc1e0 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.471+0000 7fecc8e06740 1 -- 172.21.15.37:0/3903166885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bc1e0 msgr2=0x5600cc7bc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.471+0000 7fecc8e06740 1 --2- 172.21.15.37:0/3903166885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bc1e0 0x5600cc7bc660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fecb00ee010 tx=0x7fecb00b8bd0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.471+0000 7fecc8e06740 1 -- 172.21.15.37:0/3903166885 shutdown_connections 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.471+0000 7fecc8e06740 1 --2- 172.21.15.37:0/3903166885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7c5760 0x5600cc7c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.471+0000 7fecc8e06740 1 --2- 172.21.15.37:0/3903166885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bc1e0 0x5600cc7bc660 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.471+0000 7fecc8e06740 1 --2- 172.21.15.37:0/3903166885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.471+0000 7fecc8e06740 1 -- 172.21.15.37:0/3903166885 >> 172.21.15.37:0/3903166885 conn(0x5600cc7815b0 msgr2=0x5600cc7b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.471+0000 7fecc8e06740 1 -- 172.21.15.37:0/3903166885 shutdown_connections 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.471+0000 7fecc8e06740 1 -- 172.21.15.37:0/3903166885 wait complete. 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.471+0000 7fecc8e06740 1 -- start start 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecc8e06740 1 Processor -- start 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bc1e0 0x5600cc7c2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7c5760 0x5600cc7c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecc8e06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600cc7c8250 con 0x5600cc7bcf50 2024-03-26T06:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecc8e06740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600cc6dad90 con 0x5600cc7c5760 2024-03-26T06:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecc8e06740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600cc7c83d0 con 0x5600cc7bc1e0 2024-03-26T06:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecc8e06740 10 client.0 _ll_register_callbacks cb 0x5600cc432a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecbeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecbeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7c2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34234/0 (socket says 172.21.15.37:34234) 2024-03-26T06:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecbeffd640 1 -- 172.21.15.37:0/1356617077 learned_addr learned my addr 172.21.15.37:0/1356617077 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bc1e0 0x5600cc7c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecbffff640 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7c5760 0x5600cc7c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecbeffd640 1 -- 172.21.15.37:0/1356617077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bc1e0 msgr2=0x5600cc7c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecbeffd640 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bc1e0 0x5600cc7c2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecbeffd640 1 -- 172.21.15.37:0/1356617077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7c5760 msgr2=0x5600cc7c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecbeffd640 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7c5760 0x5600cc7c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.472+0000 7fecbeffd640 1 -- 172.21.15.37:0/1356617077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600cc4ff620 con 0x5600cc7bcf50 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fecbeffd640 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7c2d40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fecb4003920 tx=0x7fecb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7b3fab4137461f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fec9effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fec9effd640 1 -- 172.21.15.37:0/1356617077 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb400ecc0 con 0x5600cc7bcf50 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fec9effd640 1 -- 172.21.15.37:0/1356617077 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fecb401e420 con 0x5600cc7bcf50 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fecc8e06740 1 -- 172.21.15.37:0/1356617077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecb002cd20 con 0x5600cc7bcf50 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fec9effd640 1 -- 172.21.15.37:0/1356617077 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecb400ee60 con 0x5600cc7bcf50 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fecc8e06740 10 client.16209 fetch_fsmap learned FSMap version 42 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fecc8e06740 1 -- 172.21.15.37:0/1356617077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5600cc7d4530 con 0x5600cc7bcf50 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fec9effd640 1 -- 172.21.15.37:0/1356617077 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecb002cd20 con 0x5600cc7bcf50 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fec9effd640 1 -- 172.21.15.37:0/1356617077 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fecb40267d0 con 0x5600cc7bcf50 2024-03-26T06:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fecc8e06740 10 client.16209 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fecc8e06740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fecc8e06740 20 client.16209 populate_metadata read hostname 'smithi037' 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fecc8e06740 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600cc7d59f0 0x5600cc7d7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fecc8e06740 4 client.16209 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.473+0000 7fecc8e06740 1 -- 172.21.15.37:0/1356617077 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600cc72bb30 con 0x5600cc7d59f0 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.474+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600cc7d59f0 0x5600cc7d7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.474+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600cc7d59f0 0x5600cc7d7de0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe6044e8cb53267 server_cookie=b75e771ad8440816 in_seq=0 out_seq=0 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.474+0000 7fec9effd640 10 client.16209 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.474+0000 7fec9effd640 1 -- 172.21.15.37:0/1356617077 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600cc72bb30 con 0x5600cc7d59f0 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.474+0000 7fec9effd640 10 client.16209 handle_command_reply: tid=0 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 1 client.16209 shutdown 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 1 -- 172.21.15.37:0/1356617077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 msgr2=0x5600cc7c2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7c2d40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fecb4003920 tx=0x7fecb4004730 comp rx=0 tx=0).stop 2024-03-26T06:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 1 -- 172.21.15.37:0/1356617077 shutdown_connections 2024-03-26T06:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600cc7d59f0 0x5600cc7d7de0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7c5760 0x5600cc7c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bcf50 0x5600cc7c2d40 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fec9effd640 0 client.16209 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 1 --2- 172.21.15.37:0/1356617077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bc1e0 0x5600cc7c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 1 -- 172.21.15.37:0/1356617077 >> 172.21.15.37:0/1356617077 conn(0x5600cc7815b0 msgr2=0x5600cc7b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 1 -- 172.21.15.37:0/1356617077 shutdown_connections 2024-03-26T06:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 1 -- 172.21.15.37:0/1356617077 wait complete. 2024-03-26T06:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.475+0000 7fecc8e06740 20 client.16209 trim_cache size 0 max 16384 2024-03-26T06:31:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecc8e06740 1 Processor -- start 2024-03-26T06:31:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecc8e06740 1 -- start start 2024-03-26T06:31:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bc1e0 0x5600cc7c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bcf50 0x5600cc7c1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7c5760 0x5600cc7c1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecc8e06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc01aaa70 con 0x5600cc7bc1e0 2024-03-26T06:31:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecc8e06740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc01ac3f0 con 0x5600cc7bcf50 2024-03-26T06:31:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecc8e06740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc01ab0d0 con 0x5600cc7c5760 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bc1e0 0x5600cc7c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bc1e0 0x5600cc7c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34246/0 (socket says 172.21.15.37:34246) 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecbf7fe640 1 -- 172.21.15.37:0/3035442302 learned_addr learned my addr 172.21.15.37:0/3035442302 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecbffff640 1 --2- 172.21.15.37:0/3035442302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7c5760 0x5600cc7c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.493+0000 7fecbeffd640 1 --2- 172.21.15.37:0/3035442302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bcf50 0x5600cc7c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.494+0000 7fecbeffd640 1 -- 172.21.15.37:0/3035442302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7c5760 msgr2=0x5600cc7c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.494+0000 7fecbeffd640 1 --2- 172.21.15.37:0/3035442302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7c5760 0x5600cc7c1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.494+0000 7fecbeffd640 1 -- 172.21.15.37:0/3035442302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bc1e0 msgr2=0x5600cc7c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.494+0000 7fecbeffd640 1 --2- 172.21.15.37:0/3035442302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bc1e0 0x5600cc7c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.494+0000 7fecbeffd640 1 -- 172.21.15.37:0/3035442302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600cc7bb4b0 con 0x5600cc7bcf50 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.494+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/3035442302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bc1e0 0x5600cc7c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:31:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.494+0000 7fecbffff640 1 --2- 172.21.15.37:0/3035442302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7c5760 0x5600cc7c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.494+0000 7fecbeffd640 1 --2- 172.21.15.37:0/3035442302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bcf50 0x5600cc7c1130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fecb4026c70 tx=0x7fecb4011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37cdf09083a237cf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.494+0000 7feca57fa640 1 -- 172.21.15.37:0/3035442302 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4033040 con 0x5600cc7bcf50 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.494+0000 7feca57fa640 1 -- 172.21.15.37:0/3035442302 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecb403c070 con 0x5600cc7bcf50 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.494+0000 7feca57fa640 1 -- 172.21.15.37:0/3035442302 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4037850 con 0x5600cc7bcf50 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 -- 172.21.15.37:0/3035442302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bcf50 msgr2=0x5600cc7c1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 --2- 172.21.15.37:0/3035442302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bcf50 0x5600cc7c1130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fecb4026c70 tx=0x7fecb4011920 comp rx=0 tx=0).stop 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 -- 172.21.15.37:0/3035442302 shutdown_connections 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 --2- 172.21.15.37:0/3035442302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7c5760 0x5600cc7c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 --2- 172.21.15.37:0/3035442302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bcf50 0x5600cc7c1130 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 --2- 172.21.15.37:0/3035442302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7bc1e0 0x5600cc7c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 -- 172.21.15.37:0/3035442302 >> 172.21.15.37:0/3035442302 conn(0x5600cc7815b0 msgr2=0x5600cc7b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 -- 172.21.15.37:0/3035442302 shutdown_connections 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 -- 172.21.15.37:0/3035442302 wait complete. 2024-03-26T06:31:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 -- start start 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 Processor -- start 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bc1e0 0x5600cc7cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bcf50 0x5600cc7bf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.495+0000 7fecc8e06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7c5760 0x5600cc7bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecc8e06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc01abd90 con 0x5600cc7c5760 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecc8e06740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc01ab730 con 0x5600cc7bc1e0 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecc8e06740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc01a8d30 con 0x5600cc7bcf50 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecc8e06740 10 client.0 _ll_register_callbacks cb 0x7fecb0005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7c5760 0x5600cc7bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bc1e0 0x5600cc7cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7c5760 0x5600cc7bf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34256/0 (socket says 172.21.15.37:34256) 2024-03-26T06:31:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbffff640 1 -- 172.21.15.37:0/109715230 learned_addr learned my addr 172.21.15.37:0/109715230 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbeffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bcf50 0x5600cc7bf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bc1e0 0x5600cc7cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:33720/0 (socket says 172.21.15.37:33720) 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbffff640 1 -- 172.21.15.37:0/109715230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bcf50 msgr2=0x5600cc7bf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbffff640 1 --2- 172.21.15.37:0/109715230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bcf50 0x5600cc7bf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbffff640 1 -- 172.21.15.37:0/109715230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bc1e0 msgr2=0x5600cc7cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbffff640 1 --2- 172.21.15.37:0/109715230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bc1e0 0x5600cc7cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbffff640 1 -- 172.21.15.37:0/109715230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600cc7bfc90 con 0x5600cc7c5760 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecbffff640 1 --2- 172.21.15.37:0/109715230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7c5760 0x5600cc7bf5f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x5600cc7bcc80 tx=0x7fecb8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da118c5703e68652 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fec9effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fec9effd640 1 -- 172.21.15.37:0/109715230 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8003b80 con 0x5600cc7c5760 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fec9effd640 1 -- 172.21.15.37:0/109715230 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fecb8003d20 con 0x5600cc7c5760 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecc8e06740 1 -- 172.21.15.37:0/109715230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecb403c070 con 0x5600cc7c5760 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fec9effd640 1 -- 172.21.15.37:0/109715230 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecb802c050 con 0x5600cc7c5760 2024-03-26T06:31:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecc8e06740 10 client.16215 fetch_fsmap learned FSMap version 42 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.496+0000 7fecc8e06740 1 -- 172.21.15.37:0/109715230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5600cc7b5490 con 0x5600cc7c5760 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.497+0000 7fec9effd640 1 -- 172.21.15.37:0/109715230 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecb403c070 con 0x5600cc7c5760 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.497+0000 7fec9effd640 1 -- 172.21.15.37:0/109715230 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fecb801f480 con 0x5600cc7c5760 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.497+0000 7fecc8e06740 10 client.16215 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.497+0000 7fecc8e06740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.497+0000 7fecc8e06740 20 client.16215 populate_metadata read hostname 'smithi037' 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.497+0000 7fecc8e06740 1 --2- 172.21.15.37:0/109715230 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600cc7b5c10 0x5600cc7b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.497+0000 7fecc8e06740 4 client.16215 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.497+0000 7fecc8e06740 1 -- 172.21.15.37:0/109715230 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5600cc725e80 con 0x5600cc7b5c10 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.497+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/109715230 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600cc7b5c10 0x5600cc7b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.497+0000 7fec9effd640 10 client.16215 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.497+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/109715230 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600cc7b5c10 0x5600cc7b6020 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ab0e02279d43791 server_cookie=b8d2c745c8e93702 in_seq=0 out_seq=0 2024-03-26T06:31:47.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 322603 4.14 MB/sec execute 322 sec latency 22.584 ms 2024-03-26T06:31:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fec9effd640 1 -- 172.21.15.37:0/109715230 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5600cc725e80 con 0x5600cc7b5c10 2024-03-26T06:31:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fec9effd640 10 client.16215 handle_command_reply: tid=0 2024-03-26T06:31:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fecc8e06740 1 client.16215 shutdown 2024-03-26T06:31:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fecc8e06740 1 -- 172.21.15.37:0/109715230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7c5760 msgr2=0x5600cc7bf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fecc8e06740 1 --2- 172.21.15.37:0/109715230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7c5760 0x5600cc7bf5f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x5600cc7bcc80 tx=0x7fecb8009500 comp rx=0 tx=0).stop 2024-03-26T06:31:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fecc8e06740 1 -- 172.21.15.37:0/109715230 shutdown_connections 2024-03-26T06:31:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fecc8e06740 1 --2- 172.21.15.37:0/109715230 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600cc7b5c10 0x5600cc7b6020 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fec9effd640 0 client.16215 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:31:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fecc8e06740 1 --2- 172.21.15.37:0/109715230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600cc7c5760 0x5600cc7bf5f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fecc8e06740 1 --2- 172.21.15.37:0/109715230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600cc7bcf50 0x5600cc7bf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fecc8e06740 1 --2- 172.21.15.37:0/109715230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600cc7bc1e0 0x5600cc7cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.528+0000 7fecc8e06740 1 -- 172.21.15.37:0/109715230 >> 172.21.15.37:0/109715230 conn(0x5600cc7815b0 msgr2=0x5600cc7c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc8e06740 1 -- 172.21.15.37:0/109715230 shutdown_connections 2024-03-26T06:31:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc8e06740 1 -- 172.21.15.37:0/109715230 wait complete. 2024-03-26T06:31:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc8e06740 20 client.16215 trim_cache size 0 max 16384 2024-03-26T06:31:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:31:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:31:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:31:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:31:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc6159640 1 -- 172.21.15.37:0/660189803 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7feca80a2a10 msgr2=0x7feca80a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc6159640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7feca80a2a10 0x7feca80a4ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fecc0104e90 tx=0x7fecb4005f90 comp rx=0 tx=0).stop 2024-03-26T06:31:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc6159640 1 -- 172.21.15.37:0/660189803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc0100440 msgr2=0x7fecc019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:31:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc6159640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc0100440 0x7fecc019c6e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fecb0006010 tx=0x7fecb002cb30 comp rx=0 tx=0).stop 2024-03-26T06:31:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc6159640 1 -- 172.21.15.37:0/660189803 shutdown_connections 2024-03-26T06:31:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc6159640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7feca80a2a10 0x7feca80a4ed0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc6159640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc01968a0 0x7fecc0196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc6159640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc0105510 0x7fecc019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc6159640 1 --2- 172.21.15.37:0/660189803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc0100440 0x7fecc019c6e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:31:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc6159640 1 -- 172.21.15.37:0/660189803 >> 172.21.15.37:0/660189803 conn(0x7fecc0077340 msgr2=0x7fecc00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:31:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.529+0000 7fecc6159640 1 -- 172.21.15.37:0/660189803 shutdown_connections 2024-03-26T06:31:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:31:47.530+0000 7fecc6159640 1 -- 172.21.15.37:0/660189803 wait complete. 2024-03-26T06:31:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:47 smithi037 ceph-mon[31974]: pgmap v271: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-03-26T06:31:48.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:47 smithi066 ceph-mon[36504]: pgmap v271: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-03-26T06:31:48.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:47 smithi149 ceph-mon[36681]: pgmap v271: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-03-26T06:31:48.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323918 4.15 MB/sec execute 323 sec latency 26.879 ms 2024-03-26T06:31:49.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 325671 4.16 MB/sec execute 324 sec latency 15.251 ms 2024-03-26T06:31:50.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:49 smithi037 ceph-mon[31974]: pgmap v272: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-03-26T06:31:50.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:49 smithi066 ceph-mon[36504]: pgmap v272: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-03-26T06:31:50.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:49 smithi149 ceph-mon[36681]: pgmap v272: 97 pgs: 97 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-03-26T06:31:50.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 327019 4.18 MB/sec execute 325 sec latency 14.823 ms 2024-03-26T06:31:51.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 328666 4.18 MB/sec execute 326 sec latency 20.123 ms 2024-03-26T06:31:52.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:51 smithi037 ceph-mon[31974]: pgmap v273: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-03-26T06:31:52.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:51 smithi066 ceph-mon[36504]: pgmap v273: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-03-26T06:31:52.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:51 smithi149 ceph-mon[36681]: pgmap v273: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-03-26T06:31:52.516 INFO:tasks.workunit.client.0.smithi037.stdout: 1 330153 4.21 MB/sec execute 327 sec latency 163.986 ms 2024-03-26T06:31:53.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 330552 4.20 MB/sec execute 328 sec latency 544.038 ms 2024-03-26T06:31:54.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:53 smithi066 ceph-mon[36504]: pgmap v274: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-03-26T06:31:54.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:53 smithi149 ceph-mon[36681]: pgmap v274: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-03-26T06:31:54.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:53 smithi037 ceph-mon[31974]: pgmap v274: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-03-26T06:31:54.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 330552 4.18 MB/sec execute 329 sec latency 1544.144 ms 2024-03-26T06:31:55.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 330552 4.17 MB/sec execute 330 sec latency 2544.250 ms 2024-03-26T06:31:56.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:55 smithi066 ceph-mon[36504]: pgmap v275: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-03-26T06:31:56.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:55 smithi149 ceph-mon[36681]: pgmap v275: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-03-26T06:31:56.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:55 smithi037 ceph-mon[31974]: pgmap v275: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 198 op/s 2024-03-26T06:31:56.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 330864 4.16 MB/sec execute 331 sec latency 3342.690 ms 2024-03-26T06:31:57.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 333059 4.18 MB/sec execute 332 sec latency 14.629 ms 2024-03-26T06:31:58.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:57 smithi066 ceph-mon[36504]: pgmap v276: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-03-26T06:31:58.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:57 smithi149 ceph-mon[36681]: pgmap v276: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-03-26T06:31:58.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:57 smithi037 ceph-mon[31974]: pgmap v276: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-03-26T06:31:58.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 334219 4.19 MB/sec execute 333 sec latency 104.374 ms 2024-03-26T06:31:59.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 334219 4.18 MB/sec execute 334 sec latency 1104.455 ms 2024-03-26T06:32:00.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:31:59 smithi066 ceph-mon[36504]: pgmap v277: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-03-26T06:32:00.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:31:59 smithi149 ceph-mon[36681]: pgmap v277: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-03-26T06:32:00.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:31:59 smithi037 ceph-mon[31974]: pgmap v277: 97 pgs: 97 active+clean; 116 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-03-26T06:32:00.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 334219 4.17 MB/sec execute 335 sec latency 2104.558 ms 2024-03-26T06:32:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.629+0000 7fc6c3b05640 1 -- 172.21.15.37:0/3354596544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6bc101510 msgr2=0x7fc6bc1018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.629+0000 7fc6c3b05640 1 --2- 172.21.15.37:0/3354596544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6bc101510 0x7fc6bc1018f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009ca0 tx=0x7fc6ac02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.629+0000 7fc6c3b05640 1 -- 172.21.15.37:0/3354596544 shutdown_connections 2024-03-26T06:32:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.629+0000 7fc6c3b05640 1 --2- 172.21.15.37:0/3354596544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6bc1065e0 0x7fc6bc10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.629+0000 7fc6c3b05640 1 --2- 172.21.15.37:0/3354596544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc101ec0 0x7fc6bc105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.629+0000 7fc6c3b05640 1 --2- 172.21.15.37:0/3354596544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6bc101510 0x7fc6bc1018f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.629+0000 7fc6c3b05640 1 -- 172.21.15.37:0/3354596544 >> 172.21.15.37:0/3354596544 conn(0x7fc6bc0fcdb0 msgr2=0x7fc6bc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.629+0000 7fc6c3b05640 1 -- 172.21.15.37:0/3354596544 shutdown_connections 2024-03-26T06:32:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.629+0000 7fc6c3b05640 1 -- 172.21.15.37:0/3354596544 wait complete. 2024-03-26T06:32:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.629+0000 7fc6c3b05640 1 Processor -- start 2024-03-26T06:32:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.629+0000 7fc6c3b05640 1 -- start start 2024-03-26T06:32:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c3b05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6bc101510 0x7fc6bc199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c3b05640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6bc1065e0 0x7fc6bc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c3b05640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc19e5f0 0x7fc6bc19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c3b05640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc111690 con 0x7fc6bc101510 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c3b05640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc111510 con 0x7fc6bc1065e0 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c3b05640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc111810 con 0x7fc6bc19e5f0 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c18a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6bc101510 0x7fc6bc199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c18a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6bc101510 0x7fc6bc199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35604/0 (socket says 172.21.15.37:35604) 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2950169667 learned_addr learned my addr 172.21.15.37:0/2950169667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c10a0640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6bc1065e0 0x7fc6bc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c20a2640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc19e5f0 0x7fc6bc19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2950169667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc19e5f0 msgr2=0x7fc6bc19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc19e5f0 0x7fc6bc19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2950169667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6bc1065e0 msgr2=0x7fc6bc19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6bc1065e0 0x7fc6bc19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2950169667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc19f0e0 con 0x7fc6bc101510 2024-03-26T06:32:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.630+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6bc101510 0x7fc6bc199d30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac005e30 tx=0x7fc6ac032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.631+0000 7fc6baffd640 1 -- 172.21.15.37:0/2950169667 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac041020 con 0x7fc6bc101510 2024-03-26T06:32:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.631+0000 7fc6c3b05640 1 -- 172.21.15.37:0/2950169667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc19f370 con 0x7fc6bc101510 2024-03-26T06:32:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.631+0000 7fc6baffd640 1 -- 172.21.15.37:0/2950169667 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac034d90 con 0x7fc6bc101510 2024-03-26T06:32:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.631+0000 7fc6baffd640 1 -- 172.21.15.37:0/2950169667 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac039040 con 0x7fc6bc101510 2024-03-26T06:32:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.631+0000 7fc6c3b05640 1 -- 172.21.15.37:0/2950169667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc1a6b90 con 0x7fc6bc101510 2024-03-26T06:32:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6baffd640 1 -- 172.21.15.37:0/2950169667 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc6ac044020 con 0x7fc6bc101510 2024-03-26T06:32:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c67bd740 1 Processor -- start 2024-03-26T06:32:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c67bd740 1 -- start start 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4639d40 0x55bff4638a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4638fd0 0x55bff4639450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff4644940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff459d5f0 con 0x55bff4642550 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff4546930 con 0x55bff4639d40 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff4599690 con 0x55bff4638fd0 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6baffd640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc6a40a2a10 0x7fc6a40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c10a0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff4644940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c10a0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff4644940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35620/0 (socket says 172.21.15.37:35620) 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c10a0640 1 -- 172.21.15.37:0/2627949445 learned_addr learned my addr 172.21.15.37:0/2627949445 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c20a2640 1 --2- 172.21.15.37:0/2627949445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4639d40 0x55bff4638a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/2627949445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4638fd0 0x55bff4639450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c20a2640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc6a40a2a10 0x7fc6a40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6baffd640 1 -- 172.21.15.37:0/2950169667 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc6ac0f0ab0 con 0x7fc6bc101510 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c10a0640 1 -- 172.21.15.37:0/2627949445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4638fd0 msgr2=0x55bff4639450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c10a0640 1 --2- 172.21.15.37:0/2627949445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4638fd0 0x55bff4639450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c10a0640 1 -- 172.21.15.37:0/2627949445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4639d40 msgr2=0x55bff4638a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c10a0640 1 --2- 172.21.15.37:0/2627949445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4639d40 0x55bff4638a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c10a0640 1 -- 172.21.15.37:0/2627949445 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bff42b2210 con 0x55bff4642550 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.634+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/2627949445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4638fd0 0x55bff4639450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.635+0000 7fc6c10a0640 1 --2- 172.21.15.37:0/2627949445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff4644940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009c60 tx=0x7fc6b0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc7b09dd71a97e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.635+0000 7fc6c20a2640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc6a40a2a10 0x7fc6a40a4ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009fd0 tx=0x7fc6b4005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.635+0000 7fc6a3fff640 1 -- 172.21.15.37:0/2627949445 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0004230 con 0x55bff4642550 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.635+0000 7fc6a3fff640 1 -- 172.21.15.37:0/2627949445 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6b00043d0 con 0x55bff4642550 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.635+0000 7fc6a3fff640 1 -- 172.21.15.37:0/2627949445 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00197e0 con 0x55bff4642550 2024-03-26T06:32:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.636+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2627949445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 msgr2=0x55bff4644940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.636+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2627949445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff4644940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009c60 tx=0x7fc6b0011810 comp rx=0 tx=0).stop 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.636+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2627949445 shutdown_connections 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.636+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2627949445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff4644940 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.636+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2627949445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4638fd0 0x55bff4639450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.636+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2627949445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4639d40 0x55bff4638a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.636+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2627949445 >> 172.21.15.37:0/2627949445 conn(0x55bff46005b0 msgr2=0x55bff4633cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.636+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2627949445 shutdown_connections 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.636+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2627949445 wait complete. 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.636+0000 7fc6c67bd740 1 -- start start 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.636+0000 7fc6c67bd740 1 Processor -- start 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff463f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff463fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff463ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff4645280 con 0x55bff4642550 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff4559d90 con 0x55bff4638fd0 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff4645400 con 0x55bff4639d40 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c67bd740 10 client.0 _ll_register_callbacks cb 0x55bff42b1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c20a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff463ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c20a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff463ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35624/0 (socket says 172.21.15.37:35624) 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c20a2640 1 -- 172.21.15.37:0/3891769721 learned_addr learned my addr 172.21.15.37:0/3891769721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c10a0640 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff463fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff463f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c20a2640 1 -- 172.21.15.37:0/3891769721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 msgr2=0x55bff463fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c20a2640 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff463fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c20a2640 1 -- 172.21.15.37:0/3891769721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 msgr2=0x55bff463f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c20a2640 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff463f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c20a2640 1 -- 172.21.15.37:0/3891769721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bff437e620 con 0x55bff4642550 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c20a2640 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff463ff60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400df70 tx=0x7fc6b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295f05e6157d1438 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6a17fa640 1 -- 172.21.15.37:0/3891769721 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4005020 con 0x55bff4642550 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6a17fa640 1 -- 172.21.15.37:0/3891769721 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc6b40051c0 con 0x55bff4642550 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.637+0000 7fc6c67bd740 1 -- 172.21.15.37:0/3891769721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6b00043d0 con 0x55bff4642550 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6a17fa640 1 -- 172.21.15.37:0/3891769721 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6b40053c0 con 0x55bff4642550 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6c67bd740 10 client.16227 fetch_fsmap learned FSMap version 42 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6c67bd740 1 -- 172.21.15.37:0/3891769721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bff46514f0 con 0x55bff4642550 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6a17fa640 1 -- 172.21.15.37:0/3891769721 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6b00043d0 con 0x55bff4642550 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6a17fa640 1 -- 172.21.15.37:0/3891769721 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc6b4027070 con 0x55bff4642550 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6c67bd740 10 client.16227 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6c67bd740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6c67bd740 20 client.16227 populate_metadata read hostname 'smithi037' 2024-03-26T06:32:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bff46529b0 0x55bff4654da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6c67bd740 4 client.16227 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6c67bd740 1 -- 172.21.15.37:0/3891769721 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bff45aab30 con 0x55bff46529b0 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.638+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bff46529b0 0x55bff4654da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.639+0000 7fc6a17fa640 10 client.16227 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.639+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bff46529b0 0x55bff4654da0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4e1a3ec4039e0a server_cookie=4a1df21dbf8e67a6 in_seq=0 out_seq=0 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.639+0000 7fc6a17fa640 1 -- 172.21.15.37:0/3891769721 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bff45aab30 con 0x55bff46529b0 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.639+0000 7fc6a17fa640 10 client.16227 handle_command_reply: tid=0 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.639+0000 7fc6c67bd740 1 client.16227 shutdown 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6c67bd740 1 -- 172.21.15.37:0/3891769721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 msgr2=0x55bff463ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff463ff60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400df70 tx=0x7fc6b40046c0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6c67bd740 1 -- 172.21.15.37:0/3891769721 shutdown_connections 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bff46529b0 0x55bff4654da0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff4642550 0x55bff463ff60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff463fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/3891769721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff463f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6a17fa640 0 client.16227 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6c67bd740 1 -- 172.21.15.37:0/3891769721 >> 172.21.15.37:0/3891769721 conn(0x55bff46005b0 msgr2=0x55bff4635a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6c67bd740 1 -- 172.21.15.37:0/3891769721 shutdown_connections 2024-03-26T06:32:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6c67bd740 1 -- 172.21.15.37:0/3891769721 wait complete. 2024-03-26T06:32:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.640+0000 7fc6c67bd740 20 client.16227 trim_cache size 0 max 16384 2024-03-26T06:32:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.653+0000 7fc6c67bd740 1 Processor -- start 2024-03-26T06:32:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.653+0000 7fc6c67bd740 1 -- start start 2024-03-26T06:32:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.653+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff4641c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.653+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff463c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.653+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff463cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.653+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1aee90 con 0x55bff46005b0 2024-03-26T06:32:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.653+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1b0810 con 0x55bff4638fd0 2024-03-26T06:32:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.653+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1af4f0 con 0x55bff4639d40 2024-03-26T06:32:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c18a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff4641c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c18a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff4641c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35632/0 (socket says 172.21.15.37:35632) 2024-03-26T06:32:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2517125525 learned_addr learned my addr 172.21.15.37:0/2517125525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c20a2640 1 --2- 172.21.15.37:0/2517125525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff463cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c10a0640 1 --2- 172.21.15.37:0/2517125525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff463c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2517125525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 msgr2=0x55bff463cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/2517125525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff463cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2517125525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 msgr2=0x55bff463c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/2517125525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff463c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2517125525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bff46382a0 con 0x55bff46005b0 2024-03-26T06:32:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c10a0640 1 --2- 172.21.15.37:0/2517125525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff463c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:32:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/2517125525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff4641c10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0bd1b0 tx=0x7fc6ac005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac93124b251949b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6a37fe640 1 -- 172.21.15.37:0/2517125525 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0bd220 con 0x55bff46005b0 2024-03-26T06:32:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6a37fe640 1 -- 172.21.15.37:0/2517125525 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac0c1370 con 0x55bff46005b0 2024-03-26T06:32:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.654+0000 7fc6a37fe640 1 -- 172.21.15.37:0/2517125525 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0ee480 con 0x55bff46005b0 2024-03-26T06:32:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.655+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2517125525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 msgr2=0x55bff4641c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.655+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2517125525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff4641c10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0bd1b0 tx=0x7fc6ac005c30 comp rx=0 tx=0).stop 2024-03-26T06:32:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.655+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2517125525 shutdown_connections 2024-03-26T06:32:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.655+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2517125525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff463cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.655+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2517125525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff463c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.655+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2517125525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff4641c10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.655+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2517125525 >> 172.21.15.37:0/2517125525 conn(0x55bff4638af0 msgr2=0x55bff4659740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.655+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2517125525 shutdown_connections 2024-03-26T06:32:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.655+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2517125525 wait complete. 2024-03-26T06:32:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.655+0000 7fc6c67bd740 1 -- start start 2024-03-26T06:32:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.655+0000 7fc6c67bd740 1 Processor -- start 2024-03-26T06:32:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff46487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff4648d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c67bd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff46a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1b01b0 con 0x55bff46005b0 2024-03-26T06:32:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1afb50 con 0x55bff4638fd0 2024-03-26T06:32:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c67bd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1ad150 con 0x55bff4639d40 2024-03-26T06:32:00.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c67bd740 10 client.0 _ll_register_callbacks cb 0x7fc6b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:32:00.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c18a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff46487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c18a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff46487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35634/0 (socket says 172.21.15.37:35634) 2024-03-26T06:32:00.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2171768739 learned_addr learned my addr 172.21.15.37:0/2171768739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:00.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c20a2640 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff46a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c10a0640 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff4648d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2171768739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 msgr2=0x55bff46a8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff46a8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2171768739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 msgr2=0x55bff4648d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff4648d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c18a1640 1 -- 172.21.15.37:0/2171768739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bff46a9490 con 0x55bff46005b0 2024-03-26T06:32:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c10a0640 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff4648d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:32:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6c18a1640 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff46487f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac006010 tx=0x7fc6ac003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efefe6ed1e87cfc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.656+0000 7fc6a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:32:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6a17fa640 1 -- 172.21.15.37:0/2171768739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0c1160 con 0x55bff46005b0 2024-03-26T06:32:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6a17fa640 1 -- 172.21.15.37:0/2171768739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc6ac0ee480 con 0x55bff46005b0 2024-03-26T06:32:00.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2171768739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6ac0c1370 con 0x55bff46005b0 2024-03-26T06:32:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6a17fa640 1 -- 172.21.15.37:0/2171768739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6ac0c59c0 con 0x55bff46005b0 2024-03-26T06:32:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6c67bd740 10 client.16233 fetch_fsmap learned FSMap version 42 2024-03-26T06:32:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2171768739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bff46aa6f0 con 0x55bff46005b0 2024-03-26T06:32:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6a17fa640 1 -- 172.21.15.37:0/2171768739 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac0c1370 con 0x55bff46005b0 2024-03-26T06:32:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6a17fa640 1 -- 172.21.15.37:0/2171768739 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc6ac03c030 con 0x55bff46005b0 2024-03-26T06:32:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6c67bd740 10 client.16233 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:32:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6c67bd740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:32:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6c67bd740 20 client.16233 populate_metadata read hostname 'smithi037' 2024-03-26T06:32:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bff4642550 0x55bff46acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6c67bd740 4 client.16233 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:32:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2171768739 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bff45a4e80 con 0x55bff4642550 2024-03-26T06:32:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.657+0000 7fc6c10a0640 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bff4642550 0x55bff46acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.658+0000 7fc6c10a0640 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bff4642550 0x55bff46acd10 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ad21d8e45f25e6 server_cookie=28cdaad789e055f2 in_seq=0 out_seq=0 2024-03-26T06:32:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.658+0000 7fc6a17fa640 10 client.16233 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6a17fa640 1 -- 172.21.15.37:0/2171768739 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bff45a4e80 con 0x55bff4642550 2024-03-26T06:32:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6a17fa640 10 client.16233 handle_command_reply: tid=0 2024-03-26T06:32:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 1 client.16233 shutdown 2024-03-26T06:32:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2171768739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 msgr2=0x55bff46487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff46487f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac006010 tx=0x7fc6ac003a90 comp rx=0 tx=0).stop 2024-03-26T06:32:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2171768739 shutdown_connections 2024-03-26T06:32:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bff4642550 0x55bff46acd10 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bff4639d40 0x55bff46a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bff4638fd0 0x55bff4648d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6a17fa640 0 client.16233 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 1 --2- 172.21.15.37:0/2171768739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bff46005b0 0x55bff46487f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2171768739 >> 172.21.15.37:0/2171768739 conn(0x55bff4638af0 msgr2=0x55bff46325b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2171768739 shutdown_connections 2024-03-26T06:32:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 1 -- 172.21.15.37:0/2171768739 wait complete. 2024-03-26T06:32:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c67bd740 20 client.16233 trim_cache size 0 max 16384 2024-03-26T06:32:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:32:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:32:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:32:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:32:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 -- 172.21.15.37:0/2950169667 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc6a40a2a10 msgr2=0x7fc6a40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc6a40a2a10 0x7fc6a40a4ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009fd0 tx=0x7fc6b4005f90 comp rx=0 tx=0).stop 2024-03-26T06:32:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 -- 172.21.15.37:0/2950169667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6bc101510 msgr2=0x7fc6bc199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6bc101510 0x7fc6bc199d30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac005e30 tx=0x7fc6ac032f20 comp rx=0 tx=0).stop 2024-03-26T06:32:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 -- 172.21.15.37:0/2950169667 shutdown_connections 2024-03-26T06:32:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc6a40a2a10 0x7fc6a40a4ed0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc19e5f0 0x7fc6bc19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6bc1065e0 0x7fc6bc19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 --2- 172.21.15.37:0/2950169667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6bc101510 0x7fc6bc199d30 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 -- 172.21.15.37:0/2950169667 >> 172.21.15.37:0/2950169667 conn(0x7fc6bc0fcdb0 msgr2=0x7fc6bc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 -- 172.21.15.37:0/2950169667 shutdown_connections 2024-03-26T06:32:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:00.695+0000 7fc6c3b05640 1 -- 172.21.15.37:0/2950169667 wait complete. 2024-03-26T06:32:01.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 334570 4.16 MB/sec execute 336 sec latency 2588.440 ms 2024-03-26T06:32:02.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:01 smithi066 ceph-mon[36504]: pgmap v278: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-03-26T06:32:02.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:01 smithi149 ceph-mon[36681]: pgmap v278: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-03-26T06:32:02.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:01 smithi037 ceph-mon[31974]: pgmap v278: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-03-26T06:32:02.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 336634 4.18 MB/sec execute 337 sec latency 59.349 ms 2024-03-26T06:32:03.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 337767 4.19 MB/sec execute 338 sec latency 87.950 ms 2024-03-26T06:32:04.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:03 smithi066 ceph-mon[36504]: pgmap v279: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1000 KiB/s wr, 123 op/s 2024-03-26T06:32:04.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:03 smithi149 ceph-mon[36681]: pgmap v279: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1000 KiB/s wr, 123 op/s 2024-03-26T06:32:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:03 smithi037 ceph-mon[31974]: pgmap v279: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1000 KiB/s wr, 123 op/s 2024-03-26T06:32:04.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 337767 4.17 MB/sec execute 339 sec latency 1088.056 ms 2024-03-26T06:32:05.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 337767 4.16 MB/sec execute 340 sec latency 2088.154 ms 2024-03-26T06:32:06.252 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:05 smithi037 ceph-mon[31974]: pgmap v280: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-03-26T06:32:06.252 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:05 smithi037 ceph-mon[31974]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T06:32:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:05 smithi066 ceph-mon[36504]: pgmap v280: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-03-26T06:32:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:05 smithi066 ceph-mon[36504]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T06:32:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:05 smithi149 ceph-mon[36681]: pgmap v280: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-03-26T06:32:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:05 smithi149 ceph-mon[36681]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T06:32:06.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338143 4.15 MB/sec execute 341 sec latency 2881.467 ms 2024-03-26T06:32:07.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 340351 4.18 MB/sec execute 342 sec latency 22.164 ms 2024-03-26T06:32:08.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:07 smithi066 ceph-mon[36504]: pgmap v282: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-03-26T06:32:08.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:07 smithi149 ceph-mon[36681]: pgmap v282: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-03-26T06:32:08.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:07 smithi037 ceph-mon[31974]: pgmap v282: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-03-26T06:32:08.517 INFO:tasks.workunit.client.0.smithi037.stdout: 1 341241 4.18 MB/sec execute 343 sec latency 266.676 ms 2024-03-26T06:32:09.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 341241 4.17 MB/sec execute 344 sec latency 1266.743 ms 2024-03-26T06:32:10.350 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:09 smithi066 ceph-mon[36504]: pgmap v283: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-03-26T06:32:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:09 smithi149 ceph-mon[36681]: pgmap v283: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-03-26T06:32:10.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:09 smithi037 ceph-mon[31974]: pgmap v283: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-03-26T06:32:10.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 341241 4.16 MB/sec execute 345 sec latency 2266.812 ms 2024-03-26T06:32:11.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 341557 4.15 MB/sec execute 346 sec latency 3059.870 ms 2024-03-26T06:32:12.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:11 smithi066 ceph-mon[36504]: pgmap v284: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-03-26T06:32:12.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:11 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:32:12.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:11 smithi149 ceph-mon[36681]: pgmap v284: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-03-26T06:32:12.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:11 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:32:12.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:11 smithi037 ceph-mon[31974]: pgmap v284: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-03-26T06:32:12.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:11 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:32:12.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 343694 4.17 MB/sec execute 347 sec latency 33.829 ms 2024-03-26T06:32:13.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:12 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:32:13.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:12 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:32:13.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:12 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:32:13.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:12 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:32:13.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:12 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:32:13.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:12 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:32:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:12 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:32:13.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:12 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:32:13.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:12 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:32:13.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 344790 4.17 MB/sec execute 348 sec latency 113.734 ms 2024-03-26T06:32:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.784+0000 7f2709160640 1 -- 172.21.15.37:0/2294900767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27041027a0 msgr2=0x7f270410d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.784+0000 7f2709160640 1 --2- 172.21.15.37:0/2294900767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27041027a0 0x7f270410d1a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009930 tx=0x7f26f402bc30 comp rx=0 tx=0).stop 2024-03-26T06:32:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.784+0000 7f2709160640 1 -- 172.21.15.37:0/2294900767 shutdown_connections 2024-03-26T06:32:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.784+0000 7f2709160640 1 --2- 172.21.15.37:0/2294900767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27041027a0 0x7f270410d1a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.784+0000 7f2709160640 1 --2- 172.21.15.37:0/2294900767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2704101e00 0x7f2704102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.784+0000 7f2709160640 1 --2- 172.21.15.37:0/2294900767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2704100520 0x7f2704100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.784+0000 7f2709160640 1 -- 172.21.15.37:0/2294900767 >> 172.21.15.37:0/2294900767 conn(0x7f2704077310 msgr2=0x7f2704077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.784+0000 7f2709160640 1 -- 172.21.15.37:0/2294900767 shutdown_connections 2024-03-26T06:32:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.784+0000 7f2709160640 1 -- 172.21.15.37:0/2294900767 wait complete. 2024-03-26T06:32:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f2709160640 1 Processor -- start 2024-03-26T06:32:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f2709160640 1 -- start start 2024-03-26T06:32:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f2709160640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2704100520 0x7f27041989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f2709160640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2704101e00 0x7f2704198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f2709160640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27041027a0 0x7f27041a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f2709160640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2704110670 con 0x7f27041027a0 2024-03-26T06:32:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f2709160640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27041104f0 con 0x7f2704100520 2024-03-26T06:32:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f2709160640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27041107f0 con 0x7f2704101e00 2024-03-26T06:32:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f270359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27041027a0 0x7f27041a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f270359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27041027a0 0x7f27041a3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52366/0 (socket says 172.21.15.37:52366) 2024-03-26T06:32:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f270359e640 1 -- 172.21.15.37:0/3736096740 learned_addr learned my addr 172.21.15.37:0/3736096740 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f270259c640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2704101e00 0x7f2704198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.785+0000 7f2702d9d640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2704100520 0x7f27041989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f270259c640 1 -- 172.21.15.37:0/3736096740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2704100520 msgr2=0x7f27041989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f270259c640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2704100520 0x7f27041989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f270259c640 1 -- 172.21.15.37:0/3736096740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27041027a0 msgr2=0x7f27041a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f270259c640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27041027a0 0x7f27041a3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f270259c640 1 -- 172.21.15.37:0/3736096740 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27041a3f70 con 0x7f2704101e00 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f270359e640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27041027a0 0x7f27041a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f270259c640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2704101e00 0x7f2704198ef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f26f800d6d0 tx=0x7f26f800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f26ebfff640 1 -- 172.21.15.37:0/3736096740 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8017070 con 0x7f2704101e00 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f2709160640 1 -- 172.21.15.37:0/3736096740 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27041a4140 con 0x7f2704101e00 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f26ebfff640 1 -- 172.21.15.37:0/3736096740 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26f8009820 con 0x7f2704101e00 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f26ebfff640 1 -- 172.21.15.37:0/3736096740 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8004cd0 con 0x7f2704101e00 2024-03-26T06:32:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.786+0000 7f2709160640 1 -- 172.21.15.37:0/3736096740 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27041a4660 con 0x7f2704101e00 2024-03-26T06:32:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.788+0000 7f26ebfff640 1 -- 172.21.15.37:0/3736096740 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f26f80099d0 con 0x7f2704101e00 2024-03-26T06:32:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f270be17740 1 Processor -- start 2024-03-26T06:32:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f270be17740 1 -- start start 2024-03-26T06:32:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f270be17740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796acd40 0x55c1796aba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f270be17740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796ac450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f270be17740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 0x55c1796b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f270be17740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1796105f0 con 0x55c1796abfd0 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f270be17740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1795b9930 con 0x55c1796b5550 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f270be17740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17960c690 con 0x55c1796acd40 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f2702d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f2702d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796ac450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52372/0 (socket says 172.21.15.37:52372) 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f2702d9d640 1 -- 172.21.15.37:0/313234929 learned_addr learned my addr 172.21.15.37:0/313234929 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f270359e640 1 --2- 172.21.15.37:0/313234929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796acd40 0x55c1796aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f26ebfff640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f26ec0a2a10 0x7f26ec0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f270259c640 1 --2- 172.21.15.37:0/313234929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 0x55c1796b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f270359e640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f26ec0a2a10 0x7f26ec0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f2702d9d640 1 -- 172.21.15.37:0/313234929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796acd40 msgr2=0x55c1796aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f2702d9d640 1 --2- 172.21.15.37:0/313234929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796acd40 0x55c1796aba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f2702d9d640 1 -- 172.21.15.37:0/313234929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 msgr2=0x55c1796b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f2702d9d640 1 --2- 172.21.15.37:0/313234929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 0x55c1796b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f2702d9d640 1 -- 172.21.15.37:0/313234929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c179325210 con 0x55c1796abfd0 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.789+0000 7f26ebfff640 1 -- 172.21.15.37:0/3736096740 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f26f80ce930 con 0x7f2704101e00 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f2702d9d640 1 --2- 172.21.15.37:0/313234929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796ac450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0039f0 tx=0x7f26fc003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b7d69d63fa713c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f270359e640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f26ec0a2a10 0x7f26ec0a4ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005c40 tx=0x7f26f402bc30 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f26e97fa640 1 -- 172.21.15.37:0/313234929 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc004b20 con 0x55c1796abfd0 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f26e97fa640 1 -- 172.21.15.37:0/313234929 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26fc004cc0 con 0x55c1796abfd0 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f26e97fa640 1 -- 172.21.15.37:0/313234929 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0127d0 con 0x55c1796abfd0 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f270be17740 1 -- 172.21.15.37:0/313234929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 msgr2=0x55c1796ac450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f270be17740 1 --2- 172.21.15.37:0/313234929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796ac450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0039f0 tx=0x7f26fc003eb0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f270be17740 1 -- 172.21.15.37:0/313234929 shutdown_connections 2024-03-26T06:32:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f270be17740 1 --2- 172.21.15.37:0/313234929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 0x55c1796b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f270be17740 1 --2- 172.21.15.37:0/313234929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796ac450 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f270be17740 1 --2- 172.21.15.37:0/313234929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796acd40 0x55c1796aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.790+0000 7f270be17740 1 -- 172.21.15.37:0/313234929 >> 172.21.15.37:0/313234929 conn(0x55c1796735b0 msgr2=0x55c1796a6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270be17740 1 -- 172.21.15.37:0/313234929 shutdown_connections 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270be17740 1 -- 172.21.15.37:0/313234929 wait complete. 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270be17740 1 -- start start 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270be17740 1 Processor -- start 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270be17740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270be17740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796acd40 0x55c1796b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270be17740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 0x55c1796b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270be17740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1796b8280 con 0x55c1796abfd0 2024-03-26T06:32:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270be17740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1795ccd90 con 0x55c1796b5550 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270be17740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1796b8400 con 0x55c1796acd40 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270be17740 10 client.0 _ll_register_callbacks cb 0x55c179324a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f2702d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f2702d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52388/0 (socket says 172.21.15.37:52388) 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f2702d9d640 1 -- 172.21.15.37:0/4066754513 learned_addr learned my addr 172.21.15.37:0/4066754513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f270259c640 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796acd40 0x55c1796b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f2702d9d640 1 -- 172.21.15.37:0/4066754513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796acd40 msgr2=0x55c1796b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f2702d9d640 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796acd40 0x55c1796b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f2702d9d640 1 -- 172.21.15.37:0/4066754513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 msgr2=0x55c1796b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.791+0000 7f2702d9d640 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 0x55c1796b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f2702d9d640 1 -- 172.21.15.37:0/4066754513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1793f1620 con 0x55c1796abfd0 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f2702d9d640 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796b2570 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009d00 tx=0x7f26fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3dea99ecf405c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f26e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f26e6ff5640 1 -- 172.21.15.37:0/4066754513 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc012960 con 0x55c1796abfd0 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f26e6ff5640 1 -- 172.21.15.37:0/4066754513 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f26fc012b00 con 0x55c1796abfd0 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f270be17740 1 -- 172.21.15.37:0/4066754513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26fc004cc0 con 0x55c1796abfd0 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f26e6ff5640 1 -- 172.21.15.37:0/4066754513 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26fc012d00 con 0x55c1796abfd0 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f270be17740 10 client.16245 fetch_fsmap learned FSMap version 42 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f270be17740 1 -- 172.21.15.37:0/4066754513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c1796c4540 con 0x55c1796abfd0 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f26e6ff5640 1 -- 172.21.15.37:0/4066754513 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26fc004cc0 con 0x55c1796abfd0 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f26e6ff5640 1 -- 172.21.15.37:0/4066754513 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f26fc00f040 con 0x55c1796abfd0 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f270be17740 10 client.16245 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f270be17740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f270be17740 20 client.16245 populate_metadata read hostname 'smithi037' 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f270be17740 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c1796c5950 0x55c1796c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f270be17740 4 client.16245 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:32:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.792+0000 7f270be17740 1 -- 172.21.15.37:0/4066754513 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c17961db30 con 0x55c1796c5950 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.793+0000 7f270259c640 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c1796c5950 0x55c1796c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.793+0000 7f270259c640 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c1796c5950 0x55c1796c7d40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152ebee08b3065c6 server_cookie=b4b794c52e34d827 in_seq=0 out_seq=0 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.793+0000 7f26e6ff5640 10 client.16245 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f26e6ff5640 1 -- 172.21.15.37:0/4066754513 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c17961db30 con 0x55c1796c5950 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f26e6ff5640 10 client.16245 handle_command_reply: tid=0 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 1 client.16245 shutdown 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 1 -- 172.21.15.37:0/4066754513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 msgr2=0x55c1796b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796b2570 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009d00 tx=0x7f26fc005fd0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 1 -- 172.21.15.37:0/4066754513 shutdown_connections 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c1796c5950 0x55c1796c7d40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 0x55c1796b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f26e6ff5640 0 client.16245 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796acd40 0x55c1796b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 1 --2- 172.21.15.37:0/4066754513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796abfd0 0x55c1796b2570 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 1 -- 172.21.15.37:0/4066754513 >> 172.21.15.37:0/4066754513 conn(0x55c1796735b0 msgr2=0x55c1796a8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 1 -- 172.21.15.37:0/4066754513 shutdown_connections 2024-03-26T06:32:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 1 -- 172.21.15.37:0/4066754513 wait complete. 2024-03-26T06:32:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.794+0000 7f270be17740 20 client.16245 trim_cache size 0 max 16384 2024-03-26T06:32:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270be17740 1 Processor -- start 2024-03-26T06:32:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270be17740 1 -- start start 2024-03-26T06:32:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270be17740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796abfd0 0x55c1796b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270be17740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796acd40 0x55c1796bca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270be17740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796b5550 0x55c1796b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270be17740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27041accb0 con 0x55c1796b5550 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270be17740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27041ae630 con 0x55c1796acd40 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270be17740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27041ad310 con 0x55c1796abfd0 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796b5550 0x55c1796b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796b5550 0x55c1796b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52392/0 (socket says 172.21.15.37:52392) 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270359e640 1 -- 172.21.15.37:0/3436415845 learned_addr learned my addr 172.21.15.37:0/3436415845 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f270259c640 1 --2- 172.21.15.37:0/3436415845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796acd40 0x55c1796bca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.807+0000 7f2702d9d640 1 --2- 172.21.15.37:0/3436415845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796abfd0 0x55c1796b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270359e640 1 -- 172.21.15.37:0/3436415845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796abfd0 msgr2=0x55c1796b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270359e640 1 --2- 172.21.15.37:0/3436415845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796abfd0 0x55c1796b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270359e640 1 -- 172.21.15.37:0/3436415845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796acd40 msgr2=0x55c1796bca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270359e640 1 --2- 172.21.15.37:0/3436415845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796acd40 0x55c1796bca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270359e640 1 -- 172.21.15.37:0/3436415845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1796ab2a0 con 0x55c1796b5550 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270359e640 1 --2- 172.21.15.37:0/3436415845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796b5550 0x55c1796b0e40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f26f4036040 tx=0x7f26f40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed8756c2eaaa3c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f26e8ff9640 1 -- 172.21.15.37:0/3436415845 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4052070 con 0x55c1796b5550 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f26e8ff9640 1 -- 172.21.15.37:0/3436415845 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26f4004310 con 0x55c1796b5550 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f26e8ff9640 1 -- 172.21.15.37:0/3436415845 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4003d60 con 0x55c1796b5550 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270be17740 1 -- 172.21.15.37:0/3436415845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796b5550 msgr2=0x55c1796b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270be17740 1 --2- 172.21.15.37:0/3436415845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796b5550 0x55c1796b0e40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f26f4036040 tx=0x7f26f40058d0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270be17740 1 -- 172.21.15.37:0/3436415845 shutdown_connections 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270be17740 1 --2- 172.21.15.37:0/3436415845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796b5550 0x55c1796b0e40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270be17740 1 --2- 172.21.15.37:0/3436415845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796acd40 0x55c1796bca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270be17740 1 --2- 172.21.15.37:0/3436415845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796abfd0 0x55c1796b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270be17740 1 -- 172.21.15.37:0/3436415845 >> 172.21.15.37:0/3436415845 conn(0x55c1796735b0 msgr2=0x55c1796a67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270be17740 1 -- 172.21.15.37:0/3436415845 shutdown_connections 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.808+0000 7f270be17740 1 -- 172.21.15.37:0/3436415845 wait complete. 2024-03-26T06:32:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270be17740 1 -- start start 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270be17740 1 Processor -- start 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270be17740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796abfd0 0x55c179719920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270be17740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796acd40 0x55c179719e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270be17740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 0x55c179713d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270be17740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27041adfd0 con 0x55c1796acd40 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270be17740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27041ad970 con 0x55c1796b5550 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270be17740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27041aaf70 con 0x55c1796abfd0 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270be17740 10 client.0 _ll_register_callbacks cb 0x7f26fc009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796acd40 0x55c179719e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796acd40 0x55c179719e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52402/0 (socket says 172.21.15.37:52402) 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270259c640 1 -- 172.21.15.37:0/384115791 learned_addr learned my addr 172.21.15.37:0/384115791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f2702d9d640 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796abfd0 0x55c179719920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270359e640 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 0x55c179713d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270259c640 1 -- 172.21.15.37:0/384115791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796abfd0 msgr2=0x55c179719920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270259c640 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796abfd0 0x55c179719920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270259c640 1 -- 172.21.15.37:0/384115791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 msgr2=0x55c179713d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270259c640 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 0x55c179713d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.809+0000 7f270259c640 1 -- 172.21.15.37:0/384115791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c179714420 con 0x55c1796acd40 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f270259c640 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796acd40 0x55c179719e60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f26f80cc060 tx=0x7f26f80d0d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343deab148ff64b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f26e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f26e6ff5640 1 -- 172.21.15.37:0/384115791 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8020040 con 0x55c1796acd40 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f26e6ff5640 1 -- 172.21.15.37:0/384115791 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f26f80cc280 con 0x55c1796acd40 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f270be17740 1 -- 172.21.15.37:0/384115791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26f4004310 con 0x55c1796acd40 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f26e6ff5640 1 -- 172.21.15.37:0/384115791 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26f80cc480 con 0x55c1796acd40 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f270be17740 10 client.16251 fetch_fsmap learned FSMap version 42 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f270be17740 1 -- 172.21.15.37:0/384115791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c179721130 con 0x55c1796acd40 2024-03-26T06:32:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f26e6ff5640 1 -- 172.21.15.37:0/384115791 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26f4004310 con 0x55c1796acd40 2024-03-26T06:32:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f26e6ff5640 1 -- 172.21.15.37:0/384115791 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f26f80b0070 con 0x55c1796acd40 2024-03-26T06:32:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f270be17740 10 client.16251 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:32:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f270be17740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:32:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f270be17740 20 client.16251 populate_metadata read hostname 'smithi037' 2024-03-26T06:32:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f270be17740 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c1797217e0 0x55c179723c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f270be17740 4 client.16251 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:32:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f270be17740 1 -- 172.21.15.37:0/384115791 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c179617e80 con 0x55c1797217e0 2024-03-26T06:32:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.810+0000 7f2702d9d640 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c1797217e0 0x55c179723c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.811+0000 7f26e6ff5640 10 client.16251 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.811+0000 7f2702d9d640 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c1797217e0 0x55c179723c00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d659f21e00e8fb server_cookie=3c2c02e3ea3442ed in_seq=0 out_seq=0 2024-03-26T06:32:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f26e6ff5640 1 -- 172.21.15.37:0/384115791 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c179617e80 con 0x55c1797217e0 2024-03-26T06:32:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f26e6ff5640 10 client.16251 handle_command_reply: tid=0 2024-03-26T06:32:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 1 client.16251 shutdown 2024-03-26T06:32:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 1 -- 172.21.15.37:0/384115791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796acd40 msgr2=0x55c179719e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796acd40 0x55c179719e60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f26f80cc060 tx=0x7f26f80d0d80 comp rx=0 tx=0).stop 2024-03-26T06:32:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 1 -- 172.21.15.37:0/384115791 shutdown_connections 2024-03-26T06:32:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c1797217e0 0x55c179723c00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1796b5550 0x55c179713d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f26e6ff5640 0 client.16251 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1796acd40 0x55c179719e60 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 1 --2- 172.21.15.37:0/384115791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1796abfd0 0x55c179719920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 1 -- 172.21.15.37:0/384115791 >> 172.21.15.37:0/384115791 conn(0x55c1796735b0 msgr2=0x55c1796b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 1 -- 172.21.15.37:0/384115791 shutdown_connections 2024-03-26T06:32:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 1 -- 172.21.15.37:0/384115791 wait complete. 2024-03-26T06:32:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.845+0000 7f270be17740 20 client.16251 trim_cache size 0 max 16384 2024-03-26T06:32:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:32:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:32:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:32:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:32:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 -- 172.21.15.37:0/3736096740 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f26ec0a2a10 msgr2=0x7f26ec0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f26ec0a2a10 0x7f26ec0a4ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005c40 tx=0x7f26f402bc30 comp rx=0 tx=0).stop 2024-03-26T06:32:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 -- 172.21.15.37:0/3736096740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2704101e00 msgr2=0x7f2704198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2704101e00 0x7f2704198ef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f26f800d6d0 tx=0x7f26f800db90 comp rx=0 tx=0).stop 2024-03-26T06:32:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 -- 172.21.15.37:0/3736096740 shutdown_connections 2024-03-26T06:32:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f26ec0a2a10 0x7f26ec0a4ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27041027a0 0x7f27041a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2704101e00 0x7f2704198ef0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 --2- 172.21.15.37:0/3736096740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2704100520 0x7f27041989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 -- 172.21.15.37:0/3736096740 >> 172.21.15.37:0/3736096740 conn(0x7f2704077310 msgr2=0x7f2704103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 -- 172.21.15.37:0/3736096740 shutdown_connections 2024-03-26T06:32:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:13.846+0000 7f2709160640 1 -- 172.21.15.37:0/3736096740 wait complete. 2024-03-26T06:32:14.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 344790 4.16 MB/sec execute 349 sec latency 1113.832 ms 2024-03-26T06:32:14.598 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:14 smithi066 ceph-mon[36504]: pgmap v285: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-03-26T06:32:14.662 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:14 smithi149 ceph-mon[36681]: pgmap v285: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-03-26T06:32:14.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:14 smithi037 ceph-mon[31974]: pgmap v285: 97 pgs: 97 active+clean; 120 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-03-26T06:32:15.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 344790 4.15 MB/sec execute 350 sec latency 2113.948 ms 2024-03-26T06:32:16.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:15 smithi066 ceph-mon[36504]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T06:32:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:15 smithi149 ceph-mon[36681]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T06:32:16.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:15 smithi037 ceph-mon[31974]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T06:32:16.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 345136 4.14 MB/sec execute 351 sec latency 2908.304 ms 2024-03-26T06:32:17.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:16 smithi066 ceph-mon[36504]: pgmap v287: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-03-26T06:32:17.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:16 smithi149 ceph-mon[36681]: pgmap v287: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-03-26T06:32:17.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:16 smithi037 ceph-mon[31974]: pgmap v287: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-03-26T06:32:17.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 347235 4.16 MB/sec execute 352 sec latency 31.358 ms 2024-03-26T06:32:18.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 348277 4.17 MB/sec execute 353 sec latency 163.738 ms 2024-03-26T06:32:19.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:18 smithi066 ceph-mon[36504]: pgmap v288: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:32:19.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:18 smithi149 ceph-mon[36681]: pgmap v288: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:32:19.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:18 smithi037 ceph-mon[31974]: pgmap v288: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:32:19.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 348277 4.16 MB/sec execute 354 sec latency 1163.866 ms 2024-03-26T06:32:20.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 348277 4.15 MB/sec execute 355 sec latency 2163.975 ms 2024-03-26T06:32:21.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:20 smithi066 ceph-mon[36504]: pgmap v289: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:32:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:20 smithi149 ceph-mon[36681]: pgmap v289: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:32:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:20 smithi037 ceph-mon[31974]: pgmap v289: 97 pgs: 97 active+clean; 119 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-26T06:32:21.518 INFO:tasks.workunit.client.0.smithi037.stdout: 1 348589 4.14 MB/sec execute 356 sec latency 2965.665 ms 2024-03-26T06:32:22.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 350619 4.16 MB/sec execute 357 sec latency 21.191 ms 2024-03-26T06:32:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:22 smithi066 ceph-mon[36504]: pgmap v290: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-03-26T06:32:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:22 smithi149 ceph-mon[36681]: pgmap v290: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-03-26T06:32:23.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:22 smithi037 ceph-mon[31974]: pgmap v290: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-03-26T06:32:23.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 351897 4.16 MB/sec execute 358 sec latency 30.903 ms 2024-03-26T06:32:24.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 351948 4.15 MB/sec execute 359 sec latency 971.838 ms 2024-03-26T06:32:25.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:24 smithi066 ceph-mon[36504]: pgmap v291: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-03-26T06:32:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:24 smithi149 ceph-mon[36681]: pgmap v291: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-03-26T06:32:25.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:24 smithi037 ceph-mon[31974]: pgmap v291: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-03-26T06:32:25.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 351948 4.14 MB/sec execute 360 sec latency 1971.898 ms 2024-03-26T06:32:26.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 352312 4.13 MB/sec execute 361 sec latency 2765.937 ms 2024-03-26T06:32:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.936+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3248556353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb544100440 msgr2=0x7fb5441008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.936+0000 7fb54c0c0640 1 --2- 172.21.15.37:0/3248556353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb544100440 0x7fb5441008c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb534009900 tx=0x7fb53402bc30 comp rx=0 tx=0).stop 2024-03-26T06:32:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.936+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3248556353 shutdown_connections 2024-03-26T06:32:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.936+0000 7fb54c0c0640 1 --2- 172.21.15.37:0/3248556353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb544100e00 0x7fb54410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.936+0000 7fb54c0c0640 1 --2- 172.21.15.37:0/3248556353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb544100440 0x7fb5441008c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.936+0000 7fb54c0c0640 1 --2- 172.21.15.37:0/3248556353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb544105190 0x7fb544105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.936+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3248556353 >> 172.21.15.37:0/3248556353 conn(0x7fb544077040 msgr2=0x7fb544077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.936+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3248556353 shutdown_connections 2024-03-26T06:32:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.936+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3248556353 wait complete. 2024-03-26T06:32:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.937+0000 7fb54c0c0640 1 Processor -- start 2024-03-26T06:32:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.937+0000 7fb54c0c0640 1 -- start start 2024-03-26T06:32:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.937+0000 7fb54c0c0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb544100440 0x7fb54419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.937+0000 7fb54c0c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb544100e00 0x7fb54419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.937+0000 7fb54c0c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb544105190 0x7fb544197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.937+0000 7fb54c0c0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb544110650 con 0x7fb544105190 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.937+0000 7fb54c0c0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441104d0 con 0x7fb544100e00 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.937+0000 7fb54c0c0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441107d0 con 0x7fb544100440 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb54965b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb544100e00 0x7fb54419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb549e5c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb544100440 0x7fb54419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb54a65d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb544105190 0x7fb544197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb54965b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb544100e00 0x7fb54419dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:35676/0 (socket says 172.21.15.37:35676) 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb549e5c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb544100440 0x7fb54419d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:34184/0 (socket says 172.21.15.37:34184) 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb54965b640 1 -- 172.21.15.37:0/3099264001 learned_addr learned my addr 172.21.15.37:0/3099264001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb54965b640 1 -- 172.21.15.37:0/3099264001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb544100440 msgr2=0x7fb54419d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb54965b640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb544100440 0x7fb54419d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb54965b640 1 -- 172.21.15.37:0/3099264001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb544105190 msgr2=0x7fb544197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb54965b640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb544105190 0x7fb544197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb54965b640 1 -- 172.21.15.37:0/3099264001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5441981d0 con 0x7fb544100e00 2024-03-26T06:32:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb54965b640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb544100e00 0x7fb54419dd90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb534005940 tx=0x7fb5340042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb53affd640 1 -- 172.21.15.37:0/3099264001 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534041070 con 0x7fb544100e00 2024-03-26T06:32:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb53affd640 1 -- 172.21.15.37:0/3099264001 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb53402ccc0 con 0x7fb544100e00 2024-03-26T06:32:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb53affd640 1 -- 172.21.15.37:0/3099264001 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534003d80 con 0x7fb544100e00 2024-03-26T06:32:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.938+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3099264001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb544198460 con 0x7fb544100e00 2024-03-26T06:32:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.939+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3099264001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5441a4710 con 0x7fb544100e00 2024-03-26T06:32:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.941+0000 7fb53affd640 1 -- 172.21.15.37:0/3099264001 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb534044070 con 0x7fb544100e00 2024-03-26T06:32:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.941+0000 7fb54ed7c740 1 Processor -- start 2024-03-26T06:32:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.941+0000 7fb54ed7c740 1 -- start start 2024-03-26T06:32:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.941+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110befd40 0x562110beea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.941+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110beefd0 0x562110bef450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.941+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110bfc960 0x562110bfed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.941+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562110b535f0 con 0x562110befd40 2024-03-26T06:32:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.941+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562110afc930 con 0x562110bfc960 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.941+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562110b4f690 con 0x562110beefd0 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54a65d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110befd40 0x562110beea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54a65d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110befd40 0x562110beea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35492/0 (socket says 172.21.15.37:35492) 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54a65d640 1 -- 172.21.15.37:0/3492695823 learned_addr learned my addr 172.21.15.37:0/3492695823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54965b640 1 --2- 172.21.15.37:0/3492695823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110bfc960 0x562110bfed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb549e5c640 1 --2- 172.21.15.37:0/3492695823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110beefd0 0x562110bef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb53affd640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5300a2a10 0x7fb5300a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54a65d640 1 -- 172.21.15.37:0/3492695823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110beefd0 msgr2=0x562110bef450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54a65d640 1 --2- 172.21.15.37:0/3492695823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110beefd0 0x562110bef450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54a65d640 1 -- 172.21.15.37:0/3492695823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110bfc960 msgr2=0x562110bfed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54a65d640 1 --2- 172.21.15.37:0/3492695823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110bfc960 0x562110bfed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54a65d640 1 -- 172.21.15.37:0/3492695823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562110868210 con 0x562110befd40 2024-03-26T06:32:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54965b640 1 --2- 172.21.15.37:0/3492695823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110bfc960 0x562110bfed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb53affd640 1 -- 172.21.15.37:0/3099264001 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb5340f0b80 con 0x7fb544100e00 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54a65d640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5300a2a10 0x7fb5300a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb54a65d640 1 --2- 172.21.15.37:0/3492695823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110befd40 0x562110beea90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb53c010370 tx=0x7fb53c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf3a14af3f9c6e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb52ffff640 1 -- 172.21.15.37:0/3492695823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c016070 con 0x562110befd40 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb52ffff640 1 -- 172.21.15.37:0/3492695823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb53c012b90 con 0x562110befd40 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.942+0000 7fb52ffff640 1 -- 172.21.15.37:0/3492695823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c01b7b0 con 0x562110befd40 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.943+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3492695823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110befd40 msgr2=0x562110beea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.943+0000 7fb54a65d640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5300a2a10 0x7fb5300a4ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb53c01c000 tx=0x7fb53c01be00 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.943+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/3492695823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110befd40 0x562110beea90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb53c010370 tx=0x7fb53c010830 comp rx=0 tx=0).stop 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.943+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3492695823 shutdown_connections 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.943+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/3492695823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110bfc960 0x562110bfed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.943+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/3492695823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110beefd0 0x562110bef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.943+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/3492695823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110befd40 0x562110beea90 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.943+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3492695823 >> 172.21.15.37:0/3492695823 conn(0x562110bb65b0 msgr2=0x562110be9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.943+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3492695823 shutdown_connections 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.943+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3492695823 wait complete. 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb54ed7c740 1 -- start start 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb54ed7c740 1 Processor -- start 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bf99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110befd40 0x562110bf9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110bfc960 0x562110bfa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562110bff690 con 0x562110beefd0 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562110b0fd90 con 0x562110bfc960 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562110bff810 con 0x562110befd40 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb54ed7c740 10 client.0 _ll_register_callbacks cb 0x562110867a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb549e5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bf99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb54965b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110befd40 0x562110bf9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb549e5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bf99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35498/0 (socket says 172.21.15.37:35498) 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb549e5c640 1 -- 172.21.15.37:0/3435754540 learned_addr learned my addr 172.21.15.37:0/3435754540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.944+0000 7fb54a65d640 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110bfc960 0x562110bfa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb549e5c640 1 -- 172.21.15.37:0/3435754540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110befd40 msgr2=0x562110bf9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb549e5c640 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110befd40 0x562110bf9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb549e5c640 1 -- 172.21.15.37:0/3435754540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110bfc960 msgr2=0x562110bfa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb549e5c640 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110bfc960 0x562110bfa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb549e5c640 1 -- 172.21.15.37:0/3435754540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562110934620 con 0x562110beefd0 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb54965b640 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110befd40 0x562110bf9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb549e5c640 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bf99a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb540012730 tx=0x7fb540012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28063b32c218a073 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb52d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb52d7fa640 1 -- 172.21.15.37:0/3435754540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000f8c0 con 0x562110beefd0 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb52d7fa640 1 -- 172.21.15.37:0/3435754540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb54000fa60 con 0x562110beefd0 2024-03-26T06:32:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3435754540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb53c012b90 con 0x562110beefd0 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb52d7fa640 1 -- 172.21.15.37:0/3435754540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb54000fc60 con 0x562110beefd0 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb54ed7c740 10 client.16263 fetch_fsmap learned FSMap version 42 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3435754540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x562110c0b9f0 con 0x562110beefd0 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb52d7fa640 1 -- 172.21.15.37:0/3435754540 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb53c012b90 con 0x562110beefd0 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.945+0000 7fb52d7fa640 1 -- 172.21.15.37:0/3435754540 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb54001c590 con 0x562110beefd0 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.946+0000 7fb54ed7c740 10 client.16263 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.946+0000 7fb54ed7c740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.946+0000 7fb54ed7c740 20 client.16263 populate_metadata read hostname 'smithi037' 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.946+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562110c0ce60 0x562110c0f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.946+0000 7fb54ed7c740 4 client.16263 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.946+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3435754540 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562110b60b30 con 0x562110c0ce60 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.946+0000 7fb54965b640 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562110c0ce60 0x562110c0f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.946+0000 7fb52d7fa640 10 client.16263 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.946+0000 7fb54965b640 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562110c0ce60 0x562110c0f290 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9986dcd8643a7a6 server_cookie=b26aaa71280fdbdd in_seq=0 out_seq=0 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.947+0000 7fb52d7fa640 1 -- 172.21.15.37:0/3435754540 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562110b60b30 con 0x562110c0ce60 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.947+0000 7fb52d7fa640 10 client.16263 handle_command_reply: tid=0 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.947+0000 7fb54ed7c740 1 client.16263 shutdown 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.947+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3435754540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 msgr2=0x562110bf99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.947+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bf99a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb540012730 tx=0x7fb540012b30 comp rx=0 tx=0).stop 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.947+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3435754540 shutdown_connections 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.947+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562110c0ce60 0x562110c0f290 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.947+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110bfc960 0x562110bfa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.947+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110befd40 0x562110bf9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.947+0000 7fb52d7fa640 0 client.16263 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.947+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/3435754540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bf99a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.948+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3435754540 >> 172.21.15.37:0/3435754540 conn(0x562110bb65b0 msgr2=0x562110beba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.948+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3435754540 shutdown_connections 2024-03-26T06:32:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.948+0000 7fb54ed7c740 1 -- 172.21.15.37:0/3435754540 wait complete. 2024-03-26T06:32:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.948+0000 7fb54ed7c740 20 client.16263 trim_cache size 0 max 16384 2024-03-26T06:32:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb54ed7c740 1 Processor -- start 2024-03-26T06:32:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb54ed7c740 1 -- start start 2024-03-26T06:32:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bfc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 0x562110bf8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110bfc960 0x562110bf87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441aaaa0 con 0x562110beefd0 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441ac420 con 0x562110befd40 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441ab100 con 0x562110bfc960 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb549e5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bfc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb549e5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bfc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35508/0 (socket says 172.21.15.37:35508) 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb54965b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 0x562110bf8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb549e5c640 1 -- 172.21.15.37:0/1864208303 learned_addr learned my addr 172.21.15.37:0/1864208303 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.963+0000 7fb54a65d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110bfc960 0x562110bf87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.964+0000 7fb549e5c640 1 -- 172.21.15.37:0/1864208303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110bfc960 msgr2=0x562110bf87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.964+0000 7fb549e5c640 1 --2- 172.21.15.37:0/1864208303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110bfc960 0x562110bf87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.964+0000 7fb549e5c640 1 -- 172.21.15.37:0/1864208303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 msgr2=0x562110bf8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.964+0000 7fb549e5c640 1 --2- 172.21.15.37:0/1864208303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 0x562110bf8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.964+0000 7fb549e5c640 1 -- 172.21.15.37:0/1864208303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562110bee2a0 con 0x562110beefd0 2024-03-26T06:32:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.964+0000 7fb54965b640 1 --2- 172.21.15.37:0/1864208303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 0x562110bf8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:32:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.964+0000 7fb549e5c640 1 --2- 172.21.15.37:0/1864208303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bfc0d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb540012c90 tx=0x7fb54000ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1409cde2cf5a4c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.964+0000 7fb52f7fe640 1 -- 172.21.15.37:0/1864208303 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54001cb20 con 0x562110beefd0 2024-03-26T06:32:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.964+0000 7fb52f7fe640 1 -- 172.21.15.37:0/1864208303 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb54002a070 con 0x562110beefd0 2024-03-26T06:32:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.964+0000 7fb52f7fe640 1 -- 172.21.15.37:0/1864208303 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000f790 con 0x562110beefd0 2024-03-26T06:32:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.965+0000 7fb54ed7c740 1 -- 172.21.15.37:0/1864208303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 msgr2=0x562110bfc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.965+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/1864208303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bfc0d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb540012c90 tx=0x7fb54000ff90 comp rx=0 tx=0).stop 2024-03-26T06:32:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.965+0000 7fb54ed7c740 1 -- 172.21.15.37:0/1864208303 shutdown_connections 2024-03-26T06:32:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.965+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/1864208303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110bfc960 0x562110bf87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.965+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/1864208303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 0x562110bf8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.965+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/1864208303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110bfc0d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.965+0000 7fb54ed7c740 1 -- 172.21.15.37:0/1864208303 >> 172.21.15.37:0/1864208303 conn(0x562110bb65b0 msgr2=0x562110bf4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.965+0000 7fb54ed7c740 1 -- 172.21.15.37:0/1864208303 shutdown_connections 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.965+0000 7fb54ed7c740 1 -- 172.21.15.37:0/1864208303 wait complete. 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.965+0000 7fb54ed7c740 1 -- start start 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.965+0000 7fb54ed7c740 1 Processor -- start 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110c02580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 0x562110c02ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb54ed7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110bfc960 0x562110c03000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441abdc0 con 0x562110beefd0 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441ab760 con 0x562110befd40 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb54ed7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441a8d80 con 0x562110bfc960 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb54ed7c740 10 client.0 _ll_register_callbacks cb 0x7fb53c009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:32:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb549e5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110c02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb549e5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110c02580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35516/0 (socket says 172.21.15.37:35516) 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb549e5c640 1 -- 172.21.15.37:0/199482537 learned_addr learned my addr 172.21.15.37:0/199482537 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb54965b640 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 0x562110c02ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb54a65d640 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110bfc960 0x562110c03000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb549e5c640 1 -- 172.21.15.37:0/199482537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110bfc960 msgr2=0x562110c03000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb549e5c640 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110bfc960 0x562110c03000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb549e5c640 1 -- 172.21.15.37:0/199482537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 msgr2=0x562110c02ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb549e5c640 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 0x562110c02ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb549e5c640 1 -- 172.21.15.37:0/199482537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562110c63470 con 0x562110beefd0 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb54965b640 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 0x562110c02ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.966+0000 7fb54a65d640 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110bfc960 0x562110c03000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb549e5c640 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110c02580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb540012b70 tx=0x7fb540009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33418dd25c866e39 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb52d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:32:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb52d7fa640 1 -- 172.21.15.37:0/199482537 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54001cdb0 con 0x562110beefd0 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb54ed7c740 1 -- 172.21.15.37:0/199482537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb54002a070 con 0x562110beefd0 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb52d7fa640 1 -- 172.21.15.37:0/199482537 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb54000f9a0 con 0x562110beefd0 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb52d7fa640 1 -- 172.21.15.37:0/199482537 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb54003d050 con 0x562110beefd0 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb54ed7c740 10 client.16269 fetch_fsmap learned FSMap version 42 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb54ed7c740 1 -- 172.21.15.37:0/199482537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x562110c64780 con 0x562110beefd0 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb52d7fa640 1 -- 172.21.15.37:0/199482537 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb54002a070 con 0x562110beefd0 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb52d7fa640 1 -- 172.21.15.37:0/199482537 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb540004ca0 con 0x562110beefd0 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb54ed7c740 10 client.16269 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb54ed7c740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb54ed7c740 20 client.16269 populate_metadata read hostname 'smithi037' 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562110c64dc0 0x562110c671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb54ed7c740 4 client.16269 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:32:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.967+0000 7fb54ed7c740 1 -- 172.21.15.37:0/199482537 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562110beed60 con 0x562110c64dc0 2024-03-26T06:32:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.968+0000 7fb54965b640 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562110c64dc0 0x562110c671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.968+0000 7fb54965b640 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562110c64dc0 0x562110c671e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f05b29fe20ff0e9 server_cookie=27641e721e85cea8 in_seq=0 out_seq=0 2024-03-26T06:32:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:26.968+0000 7fb52d7fa640 10 client.16269 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.000+0000 7fb52d7fa640 1 -- 172.21.15.37:0/199482537 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562110beed60 con 0x562110c64dc0 2024-03-26T06:32:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.000+0000 7fb52d7fa640 10 client.16269 handle_command_reply: tid=0 2024-03-26T06:32:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.000+0000 7fb54ed7c740 1 client.16269 shutdown 2024-03-26T06:32:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54ed7c740 1 -- 172.21.15.37:0/199482537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 msgr2=0x562110c02580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110c02580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb540012b70 tx=0x7fb540009eb0 comp rx=0 tx=0).stop 2024-03-26T06:32:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54ed7c740 1 -- 172.21.15.37:0/199482537 shutdown_connections 2024-03-26T06:32:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562110c64dc0 0x562110c671e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562110bfc960 0x562110c03000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562110befd40 0x562110c02ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb52d7fa640 0 client.16269 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54ed7c740 1 --2- 172.21.15.37:0/199482537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562110beefd0 0x562110c02580 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54ed7c740 1 -- 172.21.15.37:0/199482537 >> 172.21.15.37:0/199482537 conn(0x562110bb65b0 msgr2=0x562110bf8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54ed7c740 1 -- 172.21.15.37:0/199482537 shutdown_connections 2024-03-26T06:32:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54ed7c740 1 -- 172.21.15.37:0/199482537 wait complete. 2024-03-26T06:32:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54ed7c740 20 client.16269 trim_cache size 0 max 16384 2024-03-26T06:32:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:32:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:32:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:32:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:32:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3099264001 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5300a2a10 msgr2=0x7fb5300a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54c0c0640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5300a2a10 0x7fb5300a4ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb53c01c000 tx=0x7fb53c01be00 comp rx=0 tx=0).stop 2024-03-26T06:32:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.001+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3099264001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb544100e00 msgr2=0x7fb54419dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.002+0000 7fb54c0c0640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb544100e00 0x7fb54419dd90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb534005940 tx=0x7fb5340042f0 comp rx=0 tx=0).stop 2024-03-26T06:32:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.002+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3099264001 shutdown_connections 2024-03-26T06:32:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.002+0000 7fb54c0c0640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5300a2a10 0x7fb5300a4ed0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.002+0000 7fb54c0c0640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb544105190 0x7fb544197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.002+0000 7fb54c0c0640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb544100e00 0x7fb54419dd90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.002+0000 7fb54c0c0640 1 --2- 172.21.15.37:0/3099264001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb544100440 0x7fb54419d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.002+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3099264001 >> 172.21.15.37:0/3099264001 conn(0x7fb544077040 msgr2=0x7fb544068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.002+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3099264001 shutdown_connections 2024-03-26T06:32:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:27.002+0000 7fb54c0c0640 1 -- 172.21.15.37:0/3099264001 wait complete. 2024-03-26T06:32:27.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:26 smithi066 ceph-mon[36504]: pgmap v292: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 228 op/s 2024-03-26T06:32:27.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:26 smithi149 ceph-mon[36681]: pgmap v292: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 228 op/s 2024-03-26T06:32:27.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:26 smithi037 ceph-mon[31974]: pgmap v292: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 228 op/s 2024-03-26T06:32:27.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 354545 4.16 MB/sec execute 362 sec latency 22.810 ms 2024-03-26T06:32:28.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 355367 4.16 MB/sec execute 363 sec latency 293.923 ms 2024-03-26T06:32:29.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:28 smithi066 ceph-mon[36504]: pgmap v293: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-03-26T06:32:29.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:28 smithi149 ceph-mon[36681]: pgmap v293: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-03-26T06:32:29.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:28 smithi037 ceph-mon[31974]: pgmap v293: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-03-26T06:32:29.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 355367 4.15 MB/sec execute 364 sec latency 1294.037 ms 2024-03-26T06:32:30.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 355367 4.14 MB/sec execute 365 sec latency 2294.134 ms 2024-03-26T06:32:31.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:30 smithi066 ceph-mon[36504]: pgmap v294: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-03-26T06:32:31.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:30 smithi149 ceph-mon[36681]: pgmap v294: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-03-26T06:32:31.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:30 smithi037 ceph-mon[31974]: pgmap v294: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-03-26T06:32:31.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 355491 4.13 MB/sec execute 366 sec latency 3087.182 ms 2024-03-26T06:32:32.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 355491 4.11 MB/sec execute 367 sec latency 1117.025 ms 2024-03-26T06:32:33.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:32 smithi066 ceph-mon[36504]: pgmap v295: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-03-26T06:32:33.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:32 smithi149 ceph-mon[36681]: pgmap v295: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-03-26T06:32:33.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:32 smithi037 ceph-mon[31974]: pgmap v295: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-03-26T06:32:33.520 INFO:tasks.workunit.client.0.smithi037.stdout: 1 355491 4.10 MB/sec execute 368 sec latency 2117.142 ms 2024-03-26T06:32:34.519 INFO:tasks.workunit.client.0.smithi037.stdout: 1 355491 4.09 MB/sec execute 369 sec latency 3117.224 ms 2024-03-26T06:32:35.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:34 smithi066 ceph-mon[36504]: pgmap v296: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-03-26T06:32:35.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:34 smithi149 ceph-mon[36681]: pgmap v296: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-03-26T06:32:35.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:34 smithi037 ceph-mon[31974]: pgmap v296: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-03-26T06:32:35.520 INFO:tasks.workunit.client.0.smithi037.stdout: 1 355491 4.08 MB/sec execute 370 sec latency 4117.350 ms 2024-03-26T06:32:36.520 INFO:tasks.workunit.client.0.smithi037.stdout: 1 355855 4.08 MB/sec execute 371 sec latency 4911.560 ms 2024-03-26T06:32:37.001 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:36 smithi037 ceph-mon[31974]: pgmap v297: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-03-26T06:32:37.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:36 smithi066 ceph-mon[36504]: pgmap v297: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-03-26T06:32:37.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:36 smithi149 ceph-mon[36681]: pgmap v297: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-03-26T06:32:37.520 INFO:tasks.workunit.client.0.smithi037.stdout: 1 358243 4.11 MB/sec execute 372 sec latency 34.486 ms 2024-03-26T06:32:38.520 INFO:tasks.workunit.client.0.smithi037.stdout: 1 359500 4.10 MB/sec execute 373 sec latency 11.756 ms 2024-03-26T06:32:39.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:38 smithi066 ceph-mon[36504]: pgmap v298: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 851 KiB/s wr, 85 op/s 2024-03-26T06:32:39.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:38 smithi149 ceph-mon[36681]: pgmap v298: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 851 KiB/s wr, 85 op/s 2024-03-26T06:32:39.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:38 smithi037 ceph-mon[31974]: pgmap v298: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 851 KiB/s wr, 85 op/s 2024-03-26T06:32:39.520 INFO:tasks.workunit.client.0.smithi037.stdout: 1 361900 4.14 MB/sec execute 374 sec latency 16.345 ms 2024-03-26T06:32:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.094+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/1305739665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3b0100440 msgr2=0x7fa3b01008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.094+0000 7fa3b4d8b640 1 --2- 172.21.15.37:0/1305739665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3b0100440 0x7fa3b01008c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa398009900 tx=0x7fa39802bc30 comp rx=0 tx=0).stop 2024-03-26T06:32:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.094+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/1305739665 shutdown_connections 2024-03-26T06:32:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.094+0000 7fa3b4d8b640 1 --2- 172.21.15.37:0/1305739665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b0100e00 0x7fa3b010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.094+0000 7fa3b4d8b640 1 --2- 172.21.15.37:0/1305739665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3b0100440 0x7fa3b01008c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.094+0000 7fa3b4d8b640 1 --2- 172.21.15.37:0/1305739665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3b0105190 0x7fa3b0105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.094+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/1305739665 >> 172.21.15.37:0/1305739665 conn(0x7fa3b0077040 msgr2=0x7fa3b0077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.094+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/1305739665 shutdown_connections 2024-03-26T06:32:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.094+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/1305739665 wait complete. 2024-03-26T06:32:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.095+0000 7fa3b4d8b640 1 Processor -- start 2024-03-26T06:32:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.095+0000 7fa3b4d8b640 1 -- start start 2024-03-26T06:32:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.095+0000 7fa3b4d8b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b0100440 0x7fa3b0101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.095+0000 7fa3b4d8b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3b0100e00 0x7fa3b01023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.095+0000 7fa3b4d8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3b0105190 0x7fa3b01028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.095+0000 7fa3b4d8b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b0110650 con 0x7fa3b0105190 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.095+0000 7fa3b4d8b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b01104d0 con 0x7fa3b0100440 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.095+0000 7fa3b4d8b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b01107d0 con 0x7fa3b0100e00 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3b0105190 0x7fa3b01028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3b0105190 0x7fa3b01028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48476/0 (socket says 172.21.15.37:48476) 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3ae59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b0100440 0x7fa3b0101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3aed9d640 1 -- 172.21.15.37:0/3215068949 learned_addr learned my addr 172.21.15.37:0/3215068949 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3a5d9b640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3b0100e00 0x7fa3b01023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3ae59c640 1 -- 172.21.15.37:0/3215068949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3b0100e00 msgr2=0x7fa3b01023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3ae59c640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3b0100e00 0x7fa3b01023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3ae59c640 1 -- 172.21.15.37:0/3215068949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3b0105190 msgr2=0x7fa3b01028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3ae59c640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3b0105190 0x7fa3b01028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3ae59c640 1 -- 172.21.15.37:0/3215068949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b01a6110 con 0x7fa3b0100440 2024-03-26T06:32:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3ae59c640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b0100440 0x7fa3b0101e60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000e9f0 tx=0x7fa3a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3a7fff640 1 -- 172.21.15.37:0/3215068949 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a000ccc0 con 0x7fa3b0100440 2024-03-26T06:32:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/3215068949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b01a6400 con 0x7fa3b0100440 2024-03-26T06:32:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3a7fff640 1 -- 172.21.15.37:0/3215068949 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3a0005020 con 0x7fa3b0100440 2024-03-26T06:32:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3a7fff640 1 -- 172.21.15.37:0/3215068949 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00049e0 con 0x7fa3b0100440 2024-03-26T06:32:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.096+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/3215068949 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b01a68d0 con 0x7fa3b0100440 2024-03-26T06:32:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.099+0000 7fa3a7fff640 1 -- 172.21.15.37:0/3215068949 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa3a0005560 con 0x7fa3b0100440 2024-03-26T06:32:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.099+0000 7fa3b7968740 1 Processor -- start 2024-03-26T06:32:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3b7968740 1 -- start start 2024-03-26T06:32:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4adaad40 0x562c4ada9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ada9fd0 0x562c4adaa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adb3550 0x562c4adb5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3b7968740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4ad0e5f0 con 0x562c4ada9fd0 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3b7968740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4acb7930 con 0x562c4adb3550 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3b7968740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4ad0a690 con 0x562c4adaad40 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3ae59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ada9fd0 0x562c4adaa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3ae59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ada9fd0 0x562c4adaa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48480/0 (socket says 172.21.15.37:48480) 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3ae59c640 1 -- 172.21.15.37:0/3792528694 learned_addr learned my addr 172.21.15.37:0/3792528694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3a5d9b640 1 --2- 172.21.15.37:0/3792528694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adb3550 0x562c4adb5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3a7fff640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa39c0a2a10 0x7fa39c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3aed9d640 1 --2- 172.21.15.37:0/3792528694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4adaad40 0x562c4ada9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3aed9d640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa39c0a2a10 0x7fa39c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3a5d9b640 1 -- 172.21.15.37:0/3792528694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4adaad40 msgr2=0x562c4ada9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3a5d9b640 1 --2- 172.21.15.37:0/3792528694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4adaad40 0x562c4ada9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3a5d9b640 1 -- 172.21.15.37:0/3792528694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ada9fd0 msgr2=0x562c4adaa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3a5d9b640 1 --2- 172.21.15.37:0/3792528694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ada9fd0 0x562c4adaa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3a5d9b640 1 -- 172.21.15.37:0/3792528694 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c4aa23210 con 0x562c4adb3550 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3ae59c640 1 --2- 172.21.15.37:0/3792528694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ada9fd0 0x562c4adaa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3aed9d640 1 --2- 172.21.15.37:0/3792528694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4adaad40 0x562c4ada9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.100+0000 7fa3a7fff640 1 -- 172.21.15.37:0/3215068949 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa3a00ced30 con 0x7fa3b0100440 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3a5d9b640 1 --2- 172.21.15.37:0/3792528694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adb3550 0x562c4adb5940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa398000c00 tx=0x7fa398005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54e7e4e33529f1f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3aed9d640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa39c0a2a10 0x7fa39c0a4ed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80099a0 tx=0x7fa3a8004ab0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3a4d99640 1 -- 172.21.15.37:0/3792528694 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa398042070 con 0x562c4adb3550 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3a4d99640 1 -- 172.21.15.37:0/3792528694 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa39802c8b0 con 0x562c4adb3550 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3a4d99640 1 -- 172.21.15.37:0/3792528694 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa398004ec0 con 0x562c4adb3550 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3b7968740 1 -- 172.21.15.37:0/3792528694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adb3550 msgr2=0x562c4adb5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3b7968740 1 --2- 172.21.15.37:0/3792528694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adb3550 0x562c4adb5940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa398000c00 tx=0x7fa398005be0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3b7968740 1 -- 172.21.15.37:0/3792528694 shutdown_connections 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3b7968740 1 --2- 172.21.15.37:0/3792528694 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adb3550 0x562c4adb5940 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3b7968740 1 --2- 172.21.15.37:0/3792528694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ada9fd0 0x562c4adaa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3b7968740 1 --2- 172.21.15.37:0/3792528694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4adaad40 0x562c4ada9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.101+0000 7fa3b7968740 1 -- 172.21.15.37:0/3792528694 >> 172.21.15.37:0/3792528694 conn(0x562c4ad715b0 msgr2=0x562c4ada4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3b7968740 1 -- 172.21.15.37:0/3792528694 shutdown_connections 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3b7968740 1 -- 172.21.15.37:0/3792528694 wait complete. 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3b7968740 1 -- start start 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3b7968740 1 Processor -- start 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ada9fd0 0x562c4adb0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adaad40 0x562c4adb0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adb3550 0x562c4adb1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3b7968740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4adb6220 con 0x562c4adb3550 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3b7968740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4accad90 con 0x562c4adaad40 2024-03-26T06:32:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3b7968740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4adb63a0 con 0x562c4ada9fd0 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3b7968740 10 client.0 _ll_register_callbacks cb 0x562c4aa22a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adb3550 0x562c4adb1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adb3550 0x562c4adb1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48482/0 (socket says 172.21.15.37:48482) 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3a5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adaad40 0x562c4adb0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3aed9d640 1 -- 172.21.15.37:0/3118404043 learned_addr learned my addr 172.21.15.37:0/3118404043 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.102+0000 7fa3ae59c640 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ada9fd0 0x562c4adb0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa3aed9d640 1 -- 172.21.15.37:0/3118404043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ada9fd0 msgr2=0x562c4adb0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa3aed9d640 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ada9fd0 0x562c4adb0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa3aed9d640 1 -- 172.21.15.37:0/3118404043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adaad40 msgr2=0x562c4adb0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa3aed9d640 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adaad40 0x562c4adb0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa3aed9d640 1 -- 172.21.15.37:0/3118404043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c4aaef620 con 0x562c4adb3550 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa3a5d9b640 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adaad40 0x562c4adb0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa3aed9d640 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adb3550 0x562c4adb1080 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800fee0 tx=0x7fa3a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce71fac60073c52 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa395ffb640 1 -- 172.21.15.37:0/3118404043 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8019070 con 0x562c4adb3550 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa395ffb640 1 -- 172.21.15.37:0/3118404043 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa3a8004d20 con 0x562c4adb3550 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa3b7968740 1 -- 172.21.15.37:0/3118404043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa39802c8b0 con 0x562c4adb3550 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa395ffb640 1 -- 172.21.15.37:0/3118404043 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3a801f640 con 0x562c4adb3550 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa3b7968740 10 client.16281 fetch_fsmap learned FSMap version 42 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa3b7968740 1 -- 172.21.15.37:0/3118404043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x562c4adc25f0 con 0x562c4adb3550 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.103+0000 7fa395ffb640 1 -- 172.21.15.37:0/3118404043 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa39802c8b0 con 0x562c4adb3550 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.104+0000 7fa395ffb640 1 -- 172.21.15.37:0/3118404043 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa3a80055b0 con 0x562c4adb3550 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.104+0000 7fa3b7968740 10 client.16281 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.104+0000 7fa3b7968740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.104+0000 7fa3b7968740 20 client.16281 populate_metadata read hostname 'smithi037' 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.104+0000 7fa3b7968740 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562c4adc3a60 0x562c4adc5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.104+0000 7fa3b7968740 4 client.16281 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.104+0000 7fa3b7968740 1 -- 172.21.15.37:0/3118404043 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c4ad1bb30 con 0x562c4adc3a60 2024-03-26T06:32:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.104+0000 7fa3ae59c640 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562c4adc3a60 0x562c4adc5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.104+0000 7fa3ae59c640 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562c4adc3a60 0x562c4adc5e50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1eb3010a7afa7d server_cookie=ea4bb09ef5462d6d in_seq=0 out_seq=0 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.104+0000 7fa395ffb640 10 client.16281 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa395ffb640 1 -- 172.21.15.37:0/3118404043 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c4ad1bb30 con 0x562c4adc3a60 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa395ffb640 10 client.16281 handle_command_reply: tid=0 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 1 client.16281 shutdown 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 1 -- 172.21.15.37:0/3118404043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adb3550 msgr2=0x562c4adb1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adb3550 0x562c4adb1080 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800fee0 tx=0x7fa3a8004a40 comp rx=0 tx=0).stop 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 1 -- 172.21.15.37:0/3118404043 shutdown_connections 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562c4adc3a60 0x562c4adc5e50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adb3550 0x562c4adb1080 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adaad40 0x562c4adb0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa395ffb640 0 client.16281 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 1 --2- 172.21.15.37:0/3118404043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ada9fd0 0x562c4adb0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 1 -- 172.21.15.37:0/3118404043 >> 172.21.15.37:0/3118404043 conn(0x562c4ad715b0 msgr2=0x562c4ada6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 1 -- 172.21.15.37:0/3118404043 shutdown_connections 2024-03-26T06:32:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 1 -- 172.21.15.37:0/3118404043 wait complete. 2024-03-26T06:32:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.105+0000 7fa3b7968740 20 client.16281 trim_cache size 0 max 16384 2024-03-26T06:32:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3b7968740 1 Processor -- start 2024-03-26T06:32:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3b7968740 1 -- start start 2024-03-26T06:32:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ad715b0 0x562c4adb2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4ada9fd0 0x562c4adba610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adaad40 0x562c4adbab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3b7968740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b01aee40 con 0x562c4adaad40 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3b7968740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b01b07c0 con 0x562c4ada9fd0 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3b7968740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b01af4a0 con 0x562c4ad715b0 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adaad40 0x562c4adbab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adaad40 0x562c4adbab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48486/0 (socket says 172.21.15.37:48486) 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3aed9d640 1 -- 172.21.15.37:0/1193193665 learned_addr learned my addr 172.21.15.37:0/1193193665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3a5d9b640 1 --2- 172.21.15.37:0/1193193665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4ada9fd0 0x562c4adba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3ae59c640 1 --2- 172.21.15.37:0/1193193665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ad715b0 0x562c4adb2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3aed9d640 1 -- 172.21.15.37:0/1193193665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ad715b0 msgr2=0x562c4adb2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3aed9d640 1 --2- 172.21.15.37:0/1193193665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ad715b0 0x562c4adb2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.122+0000 7fa3aed9d640 1 -- 172.21.15.37:0/1193193665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4ada9fd0 msgr2=0x562c4adba610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.123+0000 7fa3aed9d640 1 --2- 172.21.15.37:0/1193193665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4ada9fd0 0x562c4adba610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.123+0000 7fa3aed9d640 1 -- 172.21.15.37:0/1193193665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c4adab4d0 con 0x562c4adaad40 2024-03-26T06:32:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.123+0000 7fa3a5d9b640 1 --2- 172.21.15.37:0/1193193665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4ada9fd0 0x562c4adba610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:32:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.123+0000 7fa3aed9d640 1 --2- 172.21.15.37:0/1193193665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adaad40 0x562c4adbab50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800feb0 tx=0x7fa3a800f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18e7323381af46f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.123+0000 7fa397fff640 1 -- 172.21.15.37:0/1193193665 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8019070 con 0x562c4adaad40 2024-03-26T06:32:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.123+0000 7fa397fff640 1 -- 172.21.15.37:0/1193193665 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3a8024070 con 0x562c4adaad40 2024-03-26T06:32:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.123+0000 7fa397fff640 1 -- 172.21.15.37:0/1193193665 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a801fca0 con 0x562c4adaad40 2024-03-26T06:32:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 -- 172.21.15.37:0/1193193665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adaad40 msgr2=0x562c4adbab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 --2- 172.21.15.37:0/1193193665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adaad40 0x562c4adbab50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800feb0 tx=0x7fa3a800f930 comp rx=0 tx=0).stop 2024-03-26T06:32:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 -- 172.21.15.37:0/1193193665 shutdown_connections 2024-03-26T06:32:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 --2- 172.21.15.37:0/1193193665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4adaad40 0x562c4adbab50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 --2- 172.21.15.37:0/1193193665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4ada9fd0 0x562c4adba610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 --2- 172.21.15.37:0/1193193665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ad715b0 0x562c4adb2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 -- 172.21.15.37:0/1193193665 >> 172.21.15.37:0/1193193665 conn(0x562c4adb5b20 msgr2=0x562c4ada47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 -- 172.21.15.37:0/1193193665 shutdown_connections 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 -- 172.21.15.37:0/1193193665 wait complete. 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 -- start start 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 Processor -- start 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ad715b0 0x562c4adad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ada9fd0 0x562c4adada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adaad40 0x562c4adadf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b01b0160 con 0x562c4ad715b0 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b01afb00 con 0x562c4adaad40 2024-03-26T06:32:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.124+0000 7fa3b7968740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b01ad100 con 0x562c4ada9fd0 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3b7968740 10 client.0 _ll_register_callbacks cb 0x7fa3a0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3ae59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ad715b0 0x562c4adad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3ae59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ad715b0 0x562c4adad4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48502/0 (socket says 172.21.15.37:48502) 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3ae59c640 1 -- 172.21.15.37:0/2993936426 learned_addr learned my addr 172.21.15.37:0/2993936426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3a5d9b640 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ada9fd0 0x562c4adada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3ae59c640 1 -- 172.21.15.37:0/2993936426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ada9fd0 msgr2=0x562c4adada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3ae59c640 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ada9fd0 0x562c4adada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3ae59c640 1 -- 172.21.15.37:0/2993936426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adaad40 msgr2=0x562c4adadf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3ae59c640 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adaad40 0x562c4adadf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3ae59c640 1 -- 172.21.15.37:0/2993936426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c4ae1a180 con 0x562c4ad715b0 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3ae59c640 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ad715b0 0x562c4adad4c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0005510 tx=0x7fa3a00d1240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d906d59b3b4bc08e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa395ffb640 1 -- 172.21.15.37:0/2993936426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a001c040 con 0x562c4ad715b0 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa395ffb640 1 -- 172.21.15.37:0/2993936426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa3a009f170 con 0x562c4ad715b0 2024-03-26T06:32:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3b7968740 1 -- 172.21.15.37:0/2993936426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3a8024070 con 0x562c4ad715b0 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa395ffb640 1 -- 172.21.15.37:0/2993936426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3a009b390 con 0x562c4ad715b0 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3b7968740 10 client.16287 fetch_fsmap learned FSMap version 42 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.125+0000 7fa3b7968740 1 -- 172.21.15.37:0/2993936426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x562c4ae1b490 con 0x562c4ad715b0 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.126+0000 7fa395ffb640 1 -- 172.21.15.37:0/2993936426 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3a8024070 con 0x562c4ad715b0 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.126+0000 7fa395ffb640 1 -- 172.21.15.37:0/2993936426 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa3a0097a00 con 0x562c4ad715b0 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.126+0000 7fa3b7968740 10 client.16287 fetch_fsmap finished waiting for FSMap version 42 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.126+0000 7fa3b7968740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.126+0000 7fa3b7968740 20 client.16287 populate_metadata read hostname 'smithi037' 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.126+0000 7fa3b7968740 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562c4adb3550 0x562c4ae1db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.126+0000 7fa3b7968740 4 client.16287 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.126+0000 7fa3b7968740 1 -- 172.21.15.37:0/2993936426 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562c4ad15e80 con 0x562c4adb3550 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.126+0000 7fa3a5d9b640 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562c4adb3550 0x562c4ae1db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.126+0000 7fa3a5d9b640 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562c4adb3550 0x562c4ae1db80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff49dbbe16cfd891 server_cookie=1b77ba85b50eda90 in_seq=0 out_seq=0 2024-03-26T06:32:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.126+0000 7fa395ffb640 10 client.16287 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa395ffb640 1 -- 172.21.15.37:0/2993936426 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562c4ad15e80 con 0x562c4adb3550 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa395ffb640 10 client.16287 handle_command_reply: tid=0 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa3b7968740 1 client.16287 shutdown 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa3b7968740 1 -- 172.21.15.37:0/2993936426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ad715b0 msgr2=0x562c4adad4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa3b7968740 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ad715b0 0x562c4adad4c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0005510 tx=0x7fa3a00d1240 comp rx=0 tx=0).stop 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa3b7968740 1 -- 172.21.15.37:0/2993936426 shutdown_connections 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa3b7968740 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x562c4adb3550 0x562c4ae1db80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa3b7968740 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c4adaad40 0x562c4adadf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa3b7968740 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c4ada9fd0 0x562c4adada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa3b7968740 1 --2- 172.21.15.37:0/2993936426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c4ad715b0 0x562c4adad4c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa3b7968740 1 -- 172.21.15.37:0/2993936426 >> 172.21.15.37:0/2993936426 conn(0x562c4adb5b20 msgr2=0x562c4adaeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa395ffb640 0 client.16287 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.156+0000 7fa3b7968740 1 -- 172.21.15.37:0/2993936426 shutdown_connections 2024-03-26T06:32:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.157+0000 7fa3b7968740 1 -- 172.21.15.37:0/2993936426 wait complete. 2024-03-26T06:32:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.157+0000 7fa3b7968740 20 client.16287 trim_cache size 0 max 16384 2024-03-26T06:32:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:32:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:32:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:32:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:32:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.157+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/3215068949 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa39c0a2a10 msgr2=0x7fa39c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.157+0000 7fa3b4d8b640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa39c0a2a10 0x7fa39c0a4ed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80099a0 tx=0x7fa3a8004ab0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.158+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/3215068949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b0100440 msgr2=0x7fa3b0101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.158+0000 7fa3b4d8b640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b0100440 0x7fa3b0101e60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000e9f0 tx=0x7fa3a000eeb0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.158+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/3215068949 shutdown_connections 2024-03-26T06:32:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.158+0000 7fa3b4d8b640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa39c0a2a10 0x7fa39c0a4ed0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.158+0000 7fa3b4d8b640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3b0105190 0x7fa3b01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.158+0000 7fa3b4d8b640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3b0100e00 0x7fa3b01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.158+0000 7fa3b4d8b640 1 --2- 172.21.15.37:0/3215068949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3b0100440 0x7fa3b0101e60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.158+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/3215068949 >> 172.21.15.37:0/3215068949 conn(0x7fa3b0077040 msgr2=0x7fa3b0068cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.159+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/3215068949 shutdown_connections 2024-03-26T06:32:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:40.159+0000 7fa3b4d8b640 1 -- 172.21.15.37:0/3215068949 wait complete. 2024-03-26T06:32:40.520 INFO:tasks.workunit.client.0.smithi037.stdout: 1 362514 4.13 MB/sec execute 375 sec latency 319.596 ms 2024-03-26T06:32:41.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:40 smithi066 ceph-mon[36504]: pgmap v299: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 851 KiB/s wr, 85 op/s 2024-03-26T06:32:41.102 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:40 smithi149 ceph-mon[36681]: pgmap v299: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 851 KiB/s wr, 85 op/s 2024-03-26T06:32:41.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:40 smithi037 ceph-mon[31974]: pgmap v299: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 851 KiB/s wr, 85 op/s 2024-03-26T06:32:41.411 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:32:41 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: ignoring --setuser ceph since I am not root 2024-03-26T06:32:41.411 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:32:41 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: ignoring --setgroup ceph since I am not root 2024-03-26T06:32:41.411 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:32:41 smithi149 ceph-mds[67761]: -- 172.21.15.149:0/58901321 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d9e755fa40 con 0x55d9e7532800 2024-03-26T06:32:41.411 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:32:41 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: starting mds.c at 2024-03-26T06:32:41.520 INFO:tasks.workunit.client.0.smithi037.stdout: 1 362582 4.12 MB/sec execute 376 sec latency 1110.879 ms 2024-03-26T06:32:42.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:41 smithi066 ceph-mon[36504]: daemon mds.c restarted 2024-03-26T06:32:42.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:41 smithi149 ceph-mon[36681]: daemon mds.c restarted 2024-03-26T06:32:42.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:41 smithi037 ceph-mon[31974]: daemon mds.c restarted 2024-03-26T06:32:42.520 INFO:tasks.workunit.client.0.smithi037.stdout: 1 362582 4.11 MB/sec execute 377 sec latency 1162.429 ms 2024-03-26T06:32:43.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:42 smithi066 ceph-mon[36504]: pgmap v300: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-03-26T06:32:43.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:42 smithi066 ceph-mon[36504]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T06:32:43.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:42 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6836/3884902906,v1:172.21.15.149:6837/3884902906] up:boot 2024-03-26T06:32:43.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:42 smithi066 ceph-mon[36504]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:32:43.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:42 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:32:43.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:42 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:32:43.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:42 smithi149 ceph-mon[36681]: pgmap v300: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-03-26T06:32:43.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:42 smithi149 ceph-mon[36681]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T06:32:43.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:42 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6836/3884902906,v1:172.21.15.149:6837/3884902906] up:boot 2024-03-26T06:32:43.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:42 smithi149 ceph-mon[36681]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:32:43.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:42 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:32:43.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:42 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:32:43.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:42 smithi037 ceph-mon[31974]: pgmap v300: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-03-26T06:32:43.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:42 smithi037 ceph-mon[31974]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T06:32:43.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:42 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6836/3884902906,v1:172.21.15.149:6837/3884902906] up:boot 2024-03-26T06:32:43.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:42 smithi037 ceph-mon[31974]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:32:43.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:42 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:32:43.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:42 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:32:43.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 362582 4.10 MB/sec execute 378 sec latency 2162.482 ms 2024-03-26T06:32:44.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 362582 4.09 MB/sec execute 379 sec latency 3162.584 ms 2024-03-26T06:32:45.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:44 smithi066 ceph-mon[36504]: pgmap v302: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-03-26T06:32:45.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:44 smithi149 ceph-mon[36681]: pgmap v302: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-03-26T06:32:45.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:44 smithi037 ceph-mon[31974]: pgmap v302: 97 pgs: 97 active+clean; 129 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-03-26T06:32:45.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 362582 4.07 MB/sec execute 380 sec latency 4162.708 ms 2024-03-26T06:32:46.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 362961 4.07 MB/sec execute 381 sec latency 4955.019 ms 2024-03-26T06:32:47.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:46 smithi066 ceph-mon[36504]: pgmap v303: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:32:47.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:46 smithi149 ceph-mon[36681]: pgmap v303: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:32:47.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:46 smithi037 ceph-mon[31974]: pgmap v303: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:32:47.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 365420 4.10 MB/sec execute 382 sec latency 13.542 ms 2024-03-26T06:32:48.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 366725 4.10 MB/sec execute 383 sec latency 13.642 ms 2024-03-26T06:32:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:48 smithi066 ceph-mon[36504]: pgmap v304: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:32:49.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:48 smithi149 ceph-mon[36681]: pgmap v304: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:32:49.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:48 smithi037 ceph-mon[31974]: pgmap v304: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:32:49.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 369094 4.13 MB/sec execute 384 sec latency 14.661 ms 2024-03-26T06:32:50.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 369674 4.12 MB/sec execute 385 sec latency 344.523 ms 2024-03-26T06:32:51.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:50 smithi066 ceph-mon[36504]: pgmap v305: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:32:51.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:50 smithi149 ceph-mon[36681]: pgmap v305: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:32:51.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:50 smithi037 ceph-mon[31974]: pgmap v305: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-03-26T06:32:51.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370053 4.11 MB/sec execute 386 sec latency 1134.351 ms 2024-03-26T06:32:52.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 371602 4.13 MB/sec execute 387 sec latency 15.730 ms 2024-03-26T06:32:53.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:52 smithi066 ceph-mon[36504]: pgmap v306: 97 pgs: 97 active+clean; 127 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-26T06:32:53.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:52 smithi149 ceph-mon[36681]: pgmap v306: 97 pgs: 97 active+clean; 127 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-26T06:32:53.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:52 smithi037 ceph-mon[31974]: pgmap v306: 97 pgs: 97 active+clean; 127 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-26T06:32:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 -- 172.21.15.37:0/4031089860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef8068970 msgr2=0x7f6ef8068dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 --2- 172.21.15.37:0/4031089860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef8068970 0x7f6ef8068dd0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009900 tx=0x7f6ee802bcd0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 -- 172.21.15.37:0/4031089860 shutdown_connections 2024-03-26T06:32:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 --2- 172.21.15.37:0/4031089860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8069310 0x7f6ef810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 --2- 172.21.15.37:0/4031089860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef8068970 0x7f6ef8068dd0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 --2- 172.21.15.37:0/4031089860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ef81073e0 0x7f6ef81077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 -- 172.21.15.37:0/4031089860 >> 172.21.15.37:0/4031089860 conn(0x7f6ef8077040 msgr2=0x7f6ef8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 -- 172.21.15.37:0/4031089860 shutdown_connections 2024-03-26T06:32:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 -- 172.21.15.37:0/4031089860 wait complete. 2024-03-26T06:32:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 Processor -- start 2024-03-26T06:32:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 -- start start 2024-03-26T06:32:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.250+0000 7f6f00729640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ef8068970 0x7f6ef819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6f00729640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8069310 0x7f6ef819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6f00729640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef81073e0 0x7f6ef81979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6f00729640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef8111670 con 0x7f6ef81073e0 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6f00729640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81114f0 con 0x7f6ef8068970 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6f00729640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81117f0 con 0x7f6ef8069310 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6efecc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef81073e0 0x7f6ef81979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6efe4c5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ef8068970 0x7f6ef819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6efecc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef81073e0 0x7f6ef81979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58316/0 (socket says 172.21.15.37:58316) 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6efecc6640 1 -- 172.21.15.37:0/323782219 learned_addr learned my addr 172.21.15.37:0/323782219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6efdcc4640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8069310 0x7f6ef819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6efecc6640 1 -- 172.21.15.37:0/323782219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8069310 msgr2=0x7f6ef819dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6efecc6640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8069310 0x7f6ef819dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6efecc6640 1 -- 172.21.15.37:0/323782219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ef8068970 msgr2=0x7f6ef819d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6efecc6640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ef8068970 0x7f6ef819d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6efecc6640 1 -- 172.21.15.37:0/323782219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef8198130 con 0x7f6ef81073e0 2024-03-26T06:32:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6efecc6640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef81073e0 0x7f6ef81979b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400e970 tx=0x7f6ef400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6eef7fe640 1 -- 172.21.15.37:0/323782219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400ccc0 con 0x7f6ef81073e0 2024-03-26T06:32:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6eef7fe640 1 -- 172.21.15.37:0/323782219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ef400ce60 con 0x7f6ef81073e0 2024-03-26T06:32:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6f00729640 1 -- 172.21.15.37:0/323782219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef8198420 con 0x7f6ef81073e0 2024-03-26T06:32:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6eef7fe640 1 -- 172.21.15.37:0/323782219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4010430 con 0x7f6ef81073e0 2024-03-26T06:32:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.251+0000 7f6f00729640 1 -- 172.21.15.37:0/323782219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef81a46c0 con 0x7f6ef81073e0 2024-03-26T06:32:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6eef7fe640 1 -- 172.21.15.37:0/323782219 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6ef4005020 con 0x7f6ef81073e0 2024-03-26T06:32:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6f033e5740 1 Processor -- start 2024-03-26T06:32:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6f033e5740 1 -- start start 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6eef7fe640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6ee40a2a10 0x7f6ee40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a5fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a60450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a69550 0x55cbc6a6b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6f033e5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc69c45f0 con 0x55cbc6a60d40 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6f033e5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc696d930 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6f033e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc69c0690 con 0x55cbc6a69550 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6eef7fe640 1 -- 172.21.15.37:0/323782219 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f6ef40cf190 con 0x7f6ef81073e0 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6efdcc4640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6ee40a2a10 0x7f6ee40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.254+0000 7f6efe4c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a5fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efe4c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a5fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58324/0 (socket says 172.21.15.37:58324) 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efe4c5640 1 -- 172.21.15.37:0/3619937557 learned_addr learned my addr 172.21.15.37:0/3619937557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efecc6640 1 --2- 172.21.15.37:0/3619937557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a69550 0x55cbc6a6b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efdcc4640 1 --2- 172.21.15.37:0/3619937557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a60450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efecc6640 1 -- 172.21.15.37:0/3619937557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a5ffd0 msgr2=0x55cbc6a60450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efecc6640 1 --2- 172.21.15.37:0/3619937557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a60450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efecc6640 1 -- 172.21.15.37:0/3619937557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 msgr2=0x55cbc6a5fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efecc6640 1 --2- 172.21.15.37:0/3619937557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a5fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efecc6640 1 -- 172.21.15.37:0/3619937557 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbc66d9210 con 0x55cbc6a69550 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efe4c5640 1 --2- 172.21.15.37:0/3619937557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a5fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efdcc4640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6ee40a2a10 0x7f6ee40a4ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ee802efd0 tx=0x7f6ee8005de0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6efecc6640 1 --2- 172.21.15.37:0/3619937557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a69550 0x55cbc6a6b940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40cc8b0 tx=0x7f6ef409b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52fe377113dc3564 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6eecff9640 1 -- 172.21.15.37:0/3619937557 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400fca0 con 0x55cbc6a69550 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6eecff9640 1 -- 172.21.15.37:0/3619937557 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ef401fdb0 con 0x55cbc6a69550 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.255+0000 7f6eecff9640 1 -- 172.21.15.37:0/3619937557 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4086070 con 0x55cbc6a69550 2024-03-26T06:32:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.256+0000 7f6f033e5740 1 -- 172.21.15.37:0/3619937557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a69550 msgr2=0x55cbc6a6b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.256+0000 7f6f033e5740 1 --2- 172.21.15.37:0/3619937557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a69550 0x55cbc6a6b940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40cc8b0 tx=0x7f6ef409b650 comp rx=0 tx=0).stop 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.256+0000 7f6f033e5740 1 -- 172.21.15.37:0/3619937557 shutdown_connections 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.256+0000 7f6f033e5740 1 --2- 172.21.15.37:0/3619937557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a69550 0x55cbc6a6b940 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.256+0000 7f6f033e5740 1 --2- 172.21.15.37:0/3619937557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a60450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.256+0000 7f6f033e5740 1 --2- 172.21.15.37:0/3619937557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a5fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.256+0000 7f6f033e5740 1 -- 172.21.15.37:0/3619937557 >> 172.21.15.37:0/3619937557 conn(0x55cbc6a275b0 msgr2=0x55cbc6a5acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.256+0000 7f6f033e5740 1 -- 172.21.15.37:0/3619937557 shutdown_connections 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.256+0000 7f6f033e5740 1 -- 172.21.15.37:0/3619937557 wait complete. 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.256+0000 7f6f033e5740 1 -- start start 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6f033e5740 1 Processor -- start 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a66540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a60d40 0x55cbc6a66a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a69550 0x55cbc6a66fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6f033e5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc6a6c220 con 0x55cbc6a69550 2024-03-26T06:32:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6f033e5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc6980d90 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6f033e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc6a6c3a0 con 0x55cbc6a60d40 2024-03-26T06:32:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6f033e5740 10 client.0 _ll_register_callbacks cb 0x55cbc66d8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:32:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6efecc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a69550 0x55cbc6a66fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6efe4c5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a66540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6efecc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a69550 0x55cbc6a66fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58330/0 (socket says 172.21.15.37:58330) 2024-03-26T06:32:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6efecc6640 1 -- 172.21.15.37:0/2229182261 learned_addr learned my addr 172.21.15.37:0/2229182261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6efdcc4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a60d40 0x55cbc6a66a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6efecc6640 1 -- 172.21.15.37:0/2229182261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a60d40 msgr2=0x55cbc6a66a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6efecc6640 1 --2- 172.21.15.37:0/2229182261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a60d40 0x55cbc6a66a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6efecc6640 1 -- 172.21.15.37:0/2229182261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a5ffd0 msgr2=0x55cbc6a66540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6efecc6640 1 --2- 172.21.15.37:0/2229182261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a66540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6efecc6640 1 -- 172.21.15.37:0/2229182261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbc67a5620 con 0x55cbc6a69550 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6efecc6640 1 --2- 172.21.15.37:0/2229182261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a69550 0x55cbc6a66fc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4000c00 tx=0x7f6ef409b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c645da1d0eebeab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6ee1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.257+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2229182261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef409bec0 con 0x55cbc6a69550 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2229182261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f6ef40860b0 con 0x55cbc6a69550 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6f033e5740 1 -- 172.21.15.37:0/2229182261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ef401fdb0 con 0x55cbc6a69550 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2229182261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ef409b7c0 con 0x55cbc6a69550 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6f033e5740 10 client.16302 fetch_fsmap learned FSMap version 44 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6f033e5740 1 -- 172.21.15.37:0/2229182261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cbc6a784b0 con 0x55cbc6a69550 2024-03-26T06:32:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2229182261 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ef401fdb0 con 0x55cbc6a69550 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2229182261 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6ef40a3ad0 con 0x55cbc6a69550 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6f033e5740 10 client.16302 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6f033e5740 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6f033e5740 20 client.16302 populate_metadata read hostname 'smithi037' 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2229182261 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbc6a79800 0x55cbc6a7bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6f033e5740 4 client.16302 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6f033e5740 1 -- 172.21.15.37:0/2229182261 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbc69d1b30 con 0x55cbc6a79800 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.258+0000 7f6efe4c5640 1 --2- 172.21.15.37:0/2229182261 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbc6a79800 0x55cbc6a7bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.259+0000 7f6efe4c5640 1 --2- 172.21.15.37:0/2229182261 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbc6a79800 0x55cbc6a7bbf0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd9eb1ff44f8a62c server_cookie=54b63028b7504458 in_seq=0 out_seq=0 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.259+0000 7f6ee1ffb640 10 client.16302 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.259+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2229182261 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbc69d1b30 con 0x55cbc6a79800 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.259+0000 7f6ee1ffb640 10 client.16302 handle_command_reply: tid=0 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.259+0000 7f6f033e5740 1 client.16302 shutdown 2024-03-26T06:32:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6f033e5740 1 -- 172.21.15.37:0/2229182261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a69550 msgr2=0x55cbc6a66fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2229182261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a69550 0x55cbc6a66fc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4000c00 tx=0x7f6ef409b6c0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6f033e5740 1 -- 172.21.15.37:0/2229182261 shutdown_connections 2024-03-26T06:32:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2229182261 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbc6a79800 0x55cbc6a7bbf0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2229182261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a69550 0x55cbc6a66fc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6ee1ffb640 0 client.16302 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2229182261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a60d40 0x55cbc6a66a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2229182261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a66540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6f033e5740 1 -- 172.21.15.37:0/2229182261 >> 172.21.15.37:0/2229182261 conn(0x55cbc6a275b0 msgr2=0x55cbc6a5ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6f033e5740 1 -- 172.21.15.37:0/2229182261 shutdown_connections 2024-03-26T06:32:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6f033e5740 1 -- 172.21.15.37:0/2229182261 wait complete. 2024-03-26T06:32:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.260+0000 7f6f033e5740 20 client.16302 trim_cache size 0 max 16384 2024-03-26T06:32:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.273+0000 7f6f033e5740 1 Processor -- start 2024-03-26T06:32:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.273+0000 7f6f033e5740 1 -- start start 2024-03-26T06:32:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.273+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a68c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.273+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a7dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.273+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a69550 0x55cbc6a74d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.273+0000 7f6f033e5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81acbd0 con 0x55cbc6a60d40 2024-03-26T06:32:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6f033e5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81ae550 con 0x55cbc6a69550 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6f033e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81ad230 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efdcc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a7dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efdcc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a7dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58344/0 (socket says 172.21.15.37:58344) 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efdcc4640 1 -- 172.21.15.37:0/617491078 learned_addr learned my addr 172.21.15.37:0/617491078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efe4c5640 1 --2- 172.21.15.37:0/617491078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a68c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efecc6640 1 --2- 172.21.15.37:0/617491078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a69550 0x55cbc6a74d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efdcc4640 1 -- 172.21.15.37:0/617491078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a5ffd0 msgr2=0x55cbc6a68c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efdcc4640 1 --2- 172.21.15.37:0/617491078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a68c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efdcc4640 1 -- 172.21.15.37:0/617491078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a69550 msgr2=0x55cbc6a74d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efdcc4640 1 --2- 172.21.15.37:0/617491078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a69550 0x55cbc6a74d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efdcc4640 1 -- 172.21.15.37:0/617491078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbc6a5f2a0 con 0x55cbc6a60d40 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efecc6640 1 --2- 172.21.15.37:0/617491078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a69550 0x55cbc6a74d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6efdcc4640 1 --2- 172.21.15.37:0/617491078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a7dff0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8006010 tx=0x7f6ee8048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afea551f434e7b3e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6ee3fff640 1 -- 172.21.15.37:0/617491078 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8004780 con 0x55cbc6a60d40 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6ee3fff640 1 -- 172.21.15.37:0/617491078 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ee8053070 con 0x55cbc6a60d40 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.274+0000 7f6ee3fff640 1 -- 172.21.15.37:0/617491078 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8003850 con 0x55cbc6a60d40 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 -- 172.21.15.37:0/617491078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 msgr2=0x55cbc6a7dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 --2- 172.21.15.37:0/617491078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a7dff0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8006010 tx=0x7f6ee8048000 comp rx=0 tx=0).stop 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 -- 172.21.15.37:0/617491078 shutdown_connections 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 --2- 172.21.15.37:0/617491078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a69550 0x55cbc6a74d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 --2- 172.21.15.37:0/617491078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a60d40 0x55cbc6a7dff0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 --2- 172.21.15.37:0/617491078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6a68c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 -- 172.21.15.37:0/617491078 >> 172.21.15.37:0/617491078 conn(0x55cbc6a275b0 msgr2=0x55cbc6a74630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 -- 172.21.15.37:0/617491078 shutdown_connections 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 -- 172.21.15.37:0/617491078 wait complete. 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 -- start start 2024-03-26T06:32:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 Processor -- start 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6acd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a60d40 0x55cbc6acdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a69550 0x55cbc6ac7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81adef0 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81ad890 con 0x55cbc6a69550 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81aae70 con 0x55cbc6a60d40 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.275+0000 7f6f033e5740 10 client.0 _ll_register_callbacks cb 0x7f6ef40973a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6efe4c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6acd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6efe4c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6acd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58356/0 (socket says 172.21.15.37:58356) 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6efe4c5640 1 -- 172.21.15.37:0/2982200179 learned_addr learned my addr 172.21.15.37:0/2982200179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6efdcc4640 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a60d40 0x55cbc6acdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6efecc6640 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a69550 0x55cbc6ac7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6efe4c5640 1 -- 172.21.15.37:0/2982200179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a60d40 msgr2=0x55cbc6acdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6efe4c5640 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a60d40 0x55cbc6acdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6efe4c5640 1 -- 172.21.15.37:0/2982200179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a69550 msgr2=0x55cbc6ac7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6efe4c5640 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a69550 0x55cbc6ac7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6efe4c5640 1 -- 172.21.15.37:0/2982200179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbc6ac8240 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6efe4c5640 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6acd740 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x55cbc6a6c430 tx=0x7f6ef001fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcd3e4a043535cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6ee1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2982200179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0004d20 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2982200179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f6ef0011820 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6f033e5740 1 -- 172.21.15.37:0/2982200179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ee8053070 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2982200179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ef002f050 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6f033e5740 10 client.16308 fetch_fsmap learned FSMap version 44 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6f033e5740 1 -- 172.21.15.37:0/2982200179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cbc6ad4fb0 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.276+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2982200179 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ee8053070 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.277+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2982200179 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6ef001d3f0 con 0x55cbc6a5ffd0 2024-03-26T06:32:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.277+0000 7f6f033e5740 10 client.16308 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:32:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.277+0000 7f6f033e5740 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:32:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.277+0000 7f6f033e5740 20 client.16308 populate_metadata read hostname 'smithi037' 2024-03-26T06:32:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.277+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbc6ad5660 0x55cbc6ad7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:32:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.277+0000 7f6f033e5740 4 client.16308 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:32:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.277+0000 7f6f033e5740 1 -- 172.21.15.37:0/2982200179 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cbc69cbe80 con 0x55cbc6ad5660 2024-03-26T06:32:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.277+0000 7f6efdcc4640 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbc6ad5660 0x55cbc6ad7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:32:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.277+0000 7f6ee1ffb640 10 client.16308 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.277+0000 7f6efdcc4640 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbc6ad5660 0x55cbc6ad7a80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ec6cffd42198c9 server_cookie=58aeb70b0a2e75b in_seq=0 out_seq=0 2024-03-26T06:32:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.312+0000 7f6ee1ffb640 1 -- 172.21.15.37:0/2982200179 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cbc69cbe80 con 0x55cbc6ad5660 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.312+0000 7f6ee1ffb640 10 client.16308 handle_command_reply: tid=0 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.312+0000 7f6f033e5740 1 client.16308 shutdown 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f033e5740 1 -- 172.21.15.37:0/2982200179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a5ffd0 msgr2=0x55cbc6acd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6acd740 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x55cbc6a6c430 tx=0x7f6ef001fa00 comp rx=0 tx=0).stop 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f033e5740 1 -- 172.21.15.37:0/2982200179 shutdown_connections 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55cbc6ad5660 0x55cbc6ad7a80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cbc6a69550 0x55cbc6ac7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6ee1ffb640 0 client.16308 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbc6a60d40 0x55cbc6acdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f033e5740 1 --2- 172.21.15.37:0/2982200179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbc6a5ffd0 0x55cbc6acd740 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f033e5740 1 -- 172.21.15.37:0/2982200179 >> 172.21.15.37:0/2982200179 conn(0x55cbc6a275b0 msgr2=0x55cbc6a59e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f033e5740 1 -- 172.21.15.37:0/2982200179 shutdown_connections 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f033e5740 1 -- 172.21.15.37:0/2982200179 wait complete. 2024-03-26T06:32:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f033e5740 20 client.16308 trim_cache size 0 max 16384 2024-03-26T06:32:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:32:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:32:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:32:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:32:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f00729640 1 -- 172.21.15.37:0/323782219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6ee40a2a10 msgr2=0x7f6ee40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f00729640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6ee40a2a10 0x7f6ee40a4ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ee802efd0 tx=0x7f6ee8005de0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f00729640 1 -- 172.21.15.37:0/323782219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef81073e0 msgr2=0x7f6ef81979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:32:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f00729640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef81073e0 0x7f6ef81979b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400e970 tx=0x7f6ef400ee30 comp rx=0 tx=0).stop 2024-03-26T06:32:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f00729640 1 -- 172.21.15.37:0/323782219 shutdown_connections 2024-03-26T06:32:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f00729640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6ee40a2a10 0x7f6ee40a4ed0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f00729640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef81073e0 0x7f6ef81979b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f00729640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8069310 0x7f6ef819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f00729640 1 --2- 172.21.15.37:0/323782219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ef8068970 0x7f6ef819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:32:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.313+0000 7f6f00729640 1 -- 172.21.15.37:0/323782219 >> 172.21.15.37:0/323782219 conn(0x7f6ef8077040 msgr2=0x7f6ef80ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:32:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.314+0000 7f6f00729640 1 -- 172.21.15.37:0/323782219 shutdown_connections 2024-03-26T06:32:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:32:53.314+0000 7f6f00729640 1 -- 172.21.15.37:0/323782219 wait complete. 2024-03-26T06:32:53.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 372989 4.14 MB/sec execute 388 sec latency 24.086 ms 2024-03-26T06:32:54.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 373222 4.13 MB/sec execute 389 sec latency 832.685 ms 2024-03-26T06:32:55.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:54 smithi066 ceph-mon[36504]: pgmap v307: 97 pgs: 97 active+clean; 127 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-26T06:32:55.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:54 smithi149 ceph-mon[36681]: pgmap v307: 97 pgs: 97 active+clean; 127 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-26T06:32:55.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:54 smithi037 ceph-mon[31974]: pgmap v307: 97 pgs: 97 active+clean; 127 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-26T06:32:55.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 373222 4.12 MB/sec execute 390 sec latency 1832.798 ms 2024-03-26T06:32:56.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 373557 4.11 MB/sec execute 391 sec latency 2625.440 ms 2024-03-26T06:32:57.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:56 smithi066 ceph-mon[36504]: pgmap v308: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-03-26T06:32:57.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:56 smithi149 ceph-mon[36681]: pgmap v308: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-03-26T06:32:57.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:56 smithi037 ceph-mon[31974]: pgmap v308: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-03-26T06:32:57.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 375781 4.13 MB/sec execute 392 sec latency 20.640 ms 2024-03-26T06:32:58.521 INFO:tasks.workunit.client.0.smithi037.stdout: 1 376953 4.14 MB/sec execute 393 sec latency 13.815 ms 2024-03-26T06:32:59.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:32:58 smithi066 ceph-mon[36504]: pgmap v309: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-03-26T06:32:59.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:32:58 smithi149 ceph-mon[36681]: pgmap v309: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-03-26T06:32:59.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:32:58 smithi037 ceph-mon[31974]: pgmap v309: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-03-26T06:32:59.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 379026 4.16 MB/sec execute 394 sec latency 23.811 ms 2024-03-26T06:33:00.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 380304 4.16 MB/sec execute 395 sec latency 45.184 ms 2024-03-26T06:33:01.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:00 smithi066 ceph-mon[36504]: pgmap v310: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-03-26T06:33:01.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:00 smithi149 ceph-mon[36681]: pgmap v310: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-03-26T06:33:01.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:00 smithi037 ceph-mon[31974]: pgmap v310: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-03-26T06:33:01.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 381543 4.16 MB/sec execute 396 sec latency 26.241 ms 2024-03-26T06:33:02.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:01 smithi066 ceph-mon[36504]: pgmap v311: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 240 op/s 2024-03-26T06:33:02.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:01 smithi149 ceph-mon[36681]: pgmap v311: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 240 op/s 2024-03-26T06:33:02.202 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:01 smithi037 ceph-mon[31974]: pgmap v311: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 240 op/s 2024-03-26T06:33:02.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 383468 4.19 MB/sec execute 397 sec latency 13.974 ms 2024-03-26T06:33:03.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 383895 4.18 MB/sec execute 398 sec latency 572.966 ms 2024-03-26T06:33:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:03 smithi066 ceph-mon[36504]: pgmap v312: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-03-26T06:33:04.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:03 smithi149 ceph-mon[36681]: pgmap v312: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-03-26T06:33:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:03 smithi037 ceph-mon[31974]: pgmap v312: 97 pgs: 97 active+clean; 131 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-03-26T06:33:04.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 383895 4.17 MB/sec execute 399 sec latency 1573.061 ms 2024-03-26T06:33:05.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 383895 4.16 MB/sec execute 400 sec latency 2573.117 ms 2024-03-26T06:33:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:05 smithi066 ceph-mon[36504]: pgmap v313: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-03-26T06:33:06.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:05 smithi066 ceph-mon[36504]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T06:33:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:05 smithi149 ceph-mon[36681]: pgmap v313: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-03-26T06:33:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:05 smithi149 ceph-mon[36681]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T06:33:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.429+0000 7ff62a189640 1 -- 172.21.15.37:0/3174916917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff624100df0 msgr2=0x7ff624104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.429+0000 7ff62a189640 1 --2- 172.21.15.37:0/3174916917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff624100df0 0x7ff624104e30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff618009900 tx=0x7ff61802bc30 comp rx=0 tx=0).stop 2024-03-26T06:33:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.429+0000 7ff62a189640 1 -- 172.21.15.37:0/3174916917 shutdown_connections 2024-03-26T06:33:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.429+0000 7ff62a189640 1 --2- 172.21.15.37:0/3174916917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff624105510 0x7ff62410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.429+0000 7ff62a189640 1 --2- 172.21.15.37:0/3174916917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff624100df0 0x7ff624104e30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.429+0000 7ff62a189640 1 --2- 172.21.15.37:0/3174916917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624100440 0x7ff624100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.429+0000 7ff62a189640 1 -- 172.21.15.37:0/3174916917 >> 172.21.15.37:0/3174916917 conn(0x7ff624077350 msgr2=0x7ff624077760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.429+0000 7ff62a189640 1 -- 172.21.15.37:0/3174916917 shutdown_connections 2024-03-26T06:33:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.429+0000 7ff62a189640 1 -- 172.21.15.37:0/3174916917 wait complete. 2024-03-26T06:33:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff62a189640 1 Processor -- start 2024-03-26T06:33:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff62a189640 1 -- start start 2024-03-26T06:33:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff62a189640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff624100440 0x7ff62419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff62a189640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff624105510 0x7ff62419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff62a189640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624197910 0x7ff624197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff62a189640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624110580 con 0x7ff624105510 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff62a189640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624110400 con 0x7ff624197910 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff62a189640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624110700 con 0x7ff624100440 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff622ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff624105510 0x7ff62419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff622ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff624105510 0x7ff62419dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47844/0 (socket says 172.21.15.37:47844) 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff622ffd640 1 -- 172.21.15.37:0/3127151398 learned_addr learned my addr 172.21.15.37:0/3127151398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff6237fe640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff624100440 0x7ff62419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.430+0000 7ff623fff640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624197910 0x7ff624197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff6237fe640 1 -- 172.21.15.37:0/3127151398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624197910 msgr2=0x7ff624197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff6237fe640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624197910 0x7ff624197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff6237fe640 1 -- 172.21.15.37:0/3127151398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff624105510 msgr2=0x7ff62419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff6237fe640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff624105510 0x7ff62419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff6237fe640 1 -- 172.21.15.37:0/3127151398 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff624198430 con 0x7ff624100440 2024-03-26T06:33:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff622ffd640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff624105510 0x7ff62419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:33:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff6237fe640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff624100440 0x7ff62419d760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff61400e9f0 tx=0x7ff61400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff620ff9640 1 -- 172.21.15.37:0/3127151398 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61400ccc0 con 0x7ff624100440 2024-03-26T06:33:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff62a189640 1 -- 172.21.15.37:0/3127151398 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff624198720 con 0x7ff624100440 2024-03-26T06:33:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff620ff9640 1 -- 172.21.15.37:0/3127151398 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff614010910 con 0x7ff624100440 2024-03-26T06:33:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff620ff9640 1 -- 172.21.15.37:0/3127151398 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614018960 con 0x7ff624100440 2024-03-26T06:33:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.431+0000 7ff62a189640 1 -- 172.21.15.37:0/3127151398 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6241a4970 con 0x7ff624100440 2024-03-26T06:33:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.434+0000 7ff620ff9640 1 -- 172.21.15.37:0/3127151398 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff614010ac0 con 0x7ff624100440 2024-03-26T06:33:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.434+0000 7ff62cd6f740 1 Processor -- start 2024-03-26T06:33:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.434+0000 7ff62cd6f740 1 -- start start 2024-03-26T06:33:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.434+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 0x55697e2f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.434+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f6fd0 0x55697e2f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.434+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 0x55697e302940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.434+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697e25b5f0 con 0x55697e300550 2024-03-26T06:33:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.434+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697e204930 con 0x55697e2f6fd0 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.434+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697e257690 con 0x55697e2f7d40 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff623fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 0x55697e2f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff623fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 0x55697e2f6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:47704/0 (socket says 172.21.15.37:47704) 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff622ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 0x55697e302940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff623fff640 1 -- 172.21.15.37:0/2659672854 learned_addr learned my addr 172.21.15.37:0/2659672854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff6237fe640 1 --2- 172.21.15.37:0/2659672854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f6fd0 0x55697e2f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff620ff9640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff60c0a2a10 0x7ff60c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff623fff640 1 -- 172.21.15.37:0/2659672854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f6fd0 msgr2=0x55697e2f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff623fff640 1 --2- 172.21.15.37:0/2659672854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f6fd0 0x55697e2f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff623fff640 1 -- 172.21.15.37:0/2659672854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 msgr2=0x55697e302940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff623fff640 1 --2- 172.21.15.37:0/2659672854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 0x55697e302940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff623fff640 1 -- 172.21.15.37:0/2659672854 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55697df70210 con 0x55697e2f7d40 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff622ffd640 1 --2- 172.21.15.37:0/2659672854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 0x55697e302940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff620ff9640 1 -- 172.21.15.37:0/3127151398 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff6140cf240 con 0x7ff624100440 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff623fff640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff60c0a2a10 0x7ff60c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.435+0000 7ff623fff640 1 --2- 172.21.15.37:0/2659672854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 0x55697e2f6a90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff61c011590 tx=0x7ff61c011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b0fb394bb8cb10 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.436+0000 7ff609ffb640 1 -- 172.21.15.37:0/2659672854 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c017050 con 0x55697e2f7d40 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.436+0000 7ff609ffb640 1 -- 172.21.15.37:0/2659672854 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff61c01bb90 con 0x55697e2f7d40 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.436+0000 7ff609ffb640 1 -- 172.21.15.37:0/2659672854 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c00e3b0 con 0x55697e2f7d40 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.436+0000 7ff62cd6f740 1 -- 172.21.15.37:0/2659672854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 msgr2=0x55697e2f6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.436+0000 7ff623fff640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff60c0a2a10 0x7ff60c0a4ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff61c022000 tx=0x7ff61c023000 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.436+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/2659672854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 0x55697e2f6a90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff61c011590 tx=0x7ff61c011a50 comp rx=0 tx=0).stop 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.436+0000 7ff62cd6f740 1 -- 172.21.15.37:0/2659672854 shutdown_connections 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.436+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/2659672854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 0x55697e302940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.436+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/2659672854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f6fd0 0x55697e2f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.437+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/2659672854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 0x55697e2f6a90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.437+0000 7ff62cd6f740 1 -- 172.21.15.37:0/2659672854 >> 172.21.15.37:0/2659672854 conn(0x55697e2be5b0 msgr2=0x55697e2f1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.437+0000 7ff62cd6f740 1 -- 172.21.15.37:0/2659672854 shutdown_connections 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.437+0000 7ff62cd6f740 1 -- 172.21.15.37:0/2659672854 wait complete. 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.437+0000 7ff62cd6f740 1 -- start start 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.437+0000 7ff62cd6f740 1 Processor -- start 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.437+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f6fd0 0x55697e2fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.437+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f7d40 0x55697e2fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.437+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e300550 0x55697e2fe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.437+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697e307690 con 0x55697e2f7d40 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697e217d90 con 0x55697e2f6fd0 2024-03-26T06:33:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697e307810 con 0x55697e300550 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff62cd6f740 10 client.0 _ll_register_callbacks cb 0x55697df6fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff622ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f7d40 0x55697e2fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff622ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f7d40 0x55697e2fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47852/0 (socket says 172.21.15.37:47852) 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff622ffd640 1 -- 172.21.15.37:0/529186441 learned_addr learned my addr 172.21.15.37:0/529186441 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff6237fe640 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f6fd0 0x55697e2fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff623fff640 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e300550 0x55697e2fe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff622ffd640 1 -- 172.21.15.37:0/529186441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e300550 msgr2=0x55697e2fe0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff622ffd640 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e300550 0x55697e2fe0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff622ffd640 1 -- 172.21.15.37:0/529186441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f6fd0 msgr2=0x55697e2fd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff622ffd640 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f6fd0 0x55697e2fd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff622ffd640 1 -- 172.21.15.37:0/529186441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55697e03c620 con 0x55697e2f7d40 2024-03-26T06:33:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff623fff640 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e300550 0x55697e2fe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:33:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff622ffd640 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f7d40 0x55697e2fdb60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff618009230 tx=0x7ff61802cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bb819883873ae0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff602ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:33:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff602ffd640 1 -- 172.21.15.37:0/529186441 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618004040 con 0x55697e2f7d40 2024-03-26T06:33:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff602ffd640 1 -- 172.21.15.37:0/529186441 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff6180041e0 con 0x55697e2f7d40 2024-03-26T06:33:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.438+0000 7ff62cd6f740 1 -- 172.21.15.37:0/529186441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff61c01bb90 con 0x55697e2f7d40 2024-03-26T06:33:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff602ffd640 1 -- 172.21.15.37:0/529186441 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6180043e0 con 0x55697e2f7d40 2024-03-26T06:33:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff62cd6f740 10 client.16320 fetch_fsmap learned FSMap version 44 2024-03-26T06:33:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff62cd6f740 1 -- 172.21.15.37:0/529186441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55697e313a00 con 0x55697e2f7d40 2024-03-26T06:33:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff602ffd640 1 -- 172.21.15.37:0/529186441 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff61c01bb90 con 0x55697e2f7d40 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff602ffd640 1 -- 172.21.15.37:0/529186441 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff618003950 con 0x55697e2f7d40 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff62cd6f740 10 client.16320 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff62cd6f740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff62cd6f740 20 client.16320 populate_metadata read hostname 'smithi037' 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55697e314ec0 0x55697e3172b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff62cd6f740 4 client.16320 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff62cd6f740 1 -- 172.21.15.37:0/529186441 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55697e268b30 con 0x55697e314ec0 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.439+0000 7ff6237fe640 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55697e314ec0 0x55697e3172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.440+0000 7ff6237fe640 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55697e314ec0 0x55697e3172b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d965c07f68ee52d7 server_cookie=7ae8a37312e4de8f in_seq=0 out_seq=0 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.440+0000 7ff602ffd640 10 client.16320 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.440+0000 7ff602ffd640 1 -- 172.21.15.37:0/529186441 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55697e268b30 con 0x55697e314ec0 2024-03-26T06:33:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.440+0000 7ff602ffd640 10 client.16320 handle_command_reply: tid=0 2024-03-26T06:33:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.440+0000 7ff62cd6f740 1 client.16320 shutdown 2024-03-26T06:33:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.440+0000 7ff62cd6f740 1 -- 172.21.15.37:0/529186441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f7d40 msgr2=0x55697e2fdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.440+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f7d40 0x55697e2fdb60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff618009230 tx=0x7ff61802cab0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.440+0000 7ff62cd6f740 1 -- 172.21.15.37:0/529186441 shutdown_connections 2024-03-26T06:33:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.441+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55697e314ec0 0x55697e3172b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.441+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e300550 0x55697e2fe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.441+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f7d40 0x55697e2fdb60 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.441+0000 7ff602ffd640 0 client.16320 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.441+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/529186441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f6fd0 0x55697e2fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.441+0000 7ff62cd6f740 1 -- 172.21.15.37:0/529186441 >> 172.21.15.37:0/529186441 conn(0x55697e2be5b0 msgr2=0x55697e2f3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.441+0000 7ff62cd6f740 1 -- 172.21.15.37:0/529186441 shutdown_connections 2024-03-26T06:33:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.441+0000 7ff62cd6f740 1 -- 172.21.15.37:0/529186441 wait complete. 2024-03-26T06:33:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.441+0000 7ff62cd6f740 20 client.16320 trim_cache size 0 max 16384 2024-03-26T06:33:06.453 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:05 smithi037 ceph-mon[31974]: pgmap v313: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-03-26T06:33:06.453 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:05 smithi037 ceph-mon[31974]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T06:33:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff62cd6f740 1 Processor -- start 2024-03-26T06:33:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff62cd6f740 1 -- start start 2024-03-26T06:33:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f6fd0 0x55697e30ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 0x55697e30bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e300550 0x55697e2fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241aabc0 con 0x55697e2f6fd0 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241ac540 con 0x55697e300550 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241ab220 con 0x55697e2f7d40 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff6237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f6fd0 0x55697e30ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff622ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 0x55697e30bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff6237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f6fd0 0x55697e30ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47860/0 (socket says 172.21.15.37:47860) 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff6237fe640 1 -- 172.21.15.37:0/1975417617 learned_addr learned my addr 172.21.15.37:0/1975417617 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff623fff640 1 --2- 172.21.15.37:0/1975417617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e300550 0x55697e2fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff6237fe640 1 -- 172.21.15.37:0/1975417617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 msgr2=0x55697e30bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff6237fe640 1 --2- 172.21.15.37:0/1975417617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 0x55697e30bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff6237fe640 1 -- 172.21.15.37:0/1975417617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e300550 msgr2=0x55697e2fbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff6237fe640 1 --2- 172.21.15.37:0/1975417617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e300550 0x55697e2fbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff6237fe640 1 -- 172.21.15.37:0/1975417617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55697e2f6590 con 0x55697e2f6fd0 2024-03-26T06:33:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.456+0000 7ff622ffd640 1 --2- 172.21.15.37:0/1975417617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 0x55697e30bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.457+0000 7ff6237fe640 1 --2- 172.21.15.37:0/1975417617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f6fd0 0x55697e30ba80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff61409b8c0 tx=0x7ff6140d17c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2422f5d458bced server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.457+0000 7ff6097fa640 1 -- 172.21.15.37:0/1975417617 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61409b930 con 0x55697e2f6fd0 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.457+0000 7ff6097fa640 1 -- 172.21.15.37:0/1975417617 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff61409f9d0 con 0x55697e2f6fd0 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.457+0000 7ff6097fa640 1 -- 172.21.15.37:0/1975417617 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6140a3b90 con 0x55697e2f6fd0 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 -- 172.21.15.37:0/1975417617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f6fd0 msgr2=0x55697e30ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/1975417617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f6fd0 0x55697e30ba80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff61409b8c0 tx=0x7ff6140d17c0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 -- 172.21.15.37:0/1975417617 shutdown_connections 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/1975417617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e300550 0x55697e2fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/1975417617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f7d40 0x55697e30bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/1975417617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e2f6fd0 0x55697e30ba80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 -- 172.21.15.37:0/1975417617 >> 172.21.15.37:0/1975417617 conn(0x55697e2be5b0 msgr2=0x55697e2f17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 -- 172.21.15.37:0/1975417617 shutdown_connections 2024-03-26T06:33:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 -- 172.21.15.37:0/1975417617 wait complete. 2024-03-26T06:33:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 -- start start 2024-03-26T06:33:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 Processor -- start 2024-03-26T06:33:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f6fd0 0x55697e2faa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f7d40 0x55697e2faf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 0x55697e36b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.458+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241abee0 con 0x55697e300550 2024-03-26T06:33:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241ab880 con 0x55697e2f7d40 2024-03-26T06:33:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff62cd6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6241a8e60 con 0x55697e2f6fd0 2024-03-26T06:33:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff62cd6f740 10 client.0 _ll_register_callbacks cb 0x7ff61c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:33:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff623fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 0x55697e36b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff622ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f7d40 0x55697e2faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff622ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f7d40 0x55697e2faf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:53632/0 (socket says 172.21.15.37:53632) 2024-03-26T06:33:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff622ffd640 1 -- 172.21.15.37:0/3546292664 learned_addr learned my addr 172.21.15.37:0/3546292664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff6237fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f6fd0 0x55697e2faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff623fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 0x55697e36b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47876/0 (socket says 172.21.15.37:47876) 2024-03-26T06:33:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff622ffd640 1 -- 172.21.15.37:0/3546292664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f6fd0 msgr2=0x55697e2faa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff622ffd640 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f6fd0 0x55697e2faa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff622ffd640 1 -- 172.21.15.37:0/3546292664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 msgr2=0x55697e36b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff622ffd640 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 0x55697e36b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff622ffd640 1 -- 172.21.15.37:0/3546292664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55697e2f3fc0 con 0x55697e2f7d40 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff623fff640 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 0x55697e36b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff622ffd640 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f7d40 0x55697e2faf40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff61802cfd0 tx=0x7ff6180097e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5549f1737e1ca7c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff602ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff602ffd640 1 -- 172.21.15.37:0/3546292664 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618000e60 con 0x55697e2f7d40 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff602ffd640 1 -- 172.21.15.37:0/3546292664 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff618003f10 con 0x55697e2f7d40 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.459+0000 7ff62cd6f740 1 -- 172.21.15.37:0/3546292664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff61409f9d0 con 0x55697e2f7d40 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff602ffd640 1 -- 172.21.15.37:0/3546292664 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff618034440 con 0x55697e2f7d40 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff62cd6f740 10 client.25862 fetch_fsmap learned FSMap version 44 2024-03-26T06:33:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff62cd6f740 1 -- 172.21.15.37:0/3546292664 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55697e2f62a0 con 0x55697e2f7d40 2024-03-26T06:33:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff602ffd640 1 -- 172.21.15.37:0/3546292664 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff61409f9d0 con 0x55697e2f7d40 2024-03-26T06:33:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff602ffd640 1 -- 172.21.15.37:0/3546292664 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff6180052f0 con 0x55697e2f7d40 2024-03-26T06:33:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff62cd6f740 10 client.25862 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:33:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff62cd6f740 10 client.25862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:33:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff62cd6f740 20 client.25862 populate_metadata read hostname 'smithi037' 2024-03-26T06:33:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55697e36cef0 0x55697e36f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff62cd6f740 4 client.25862 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:33:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff62cd6f740 1 -- 172.21.15.37:0/3546292664 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55697e262e80 con 0x55697e36cef0 2024-03-26T06:33:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.460+0000 7ff6237fe640 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55697e36cef0 0x55697e36f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.461+0000 7ff602ffd640 10 client.25862 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.461+0000 7ff6237fe640 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55697e36cef0 0x55697e36f310 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ef5fc91336dc83 server_cookie=f708d605ee16bf44 in_seq=0 out_seq=0 2024-03-26T06:33:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.496+0000 7ff602ffd640 1 -- 172.21.15.37:0/3546292664 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55697e262e80 con 0x55697e36cef0 2024-03-26T06:33:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.496+0000 7ff602ffd640 10 client.25862 handle_command_reply: tid=0 2024-03-26T06:33:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.496+0000 7ff62cd6f740 1 client.25862 shutdown 2024-03-26T06:33:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62cd6f740 1 -- 172.21.15.37:0/3546292664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f7d40 msgr2=0x55697e2faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f7d40 0x55697e2faf40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff61802cfd0 tx=0x7ff6180097e0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62cd6f740 1 -- 172.21.15.37:0/3546292664 shutdown_connections 2024-03-26T06:33:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55697e36cef0 0x55697e36f310 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55697e300550 0x55697e36b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55697e2f7d40 0x55697e2faf40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff602ffd640 0 client.25862 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62cd6f740 1 --2- 172.21.15.37:0/3546292664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55697e2f6fd0 0x55697e2faa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62cd6f740 1 -- 172.21.15.37:0/3546292664 >> 172.21.15.37:0/3546292664 conn(0x55697e2be5b0 msgr2=0x55697e2fbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62cd6f740 1 -- 172.21.15.37:0/3546292664 shutdown_connections 2024-03-26T06:33:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62cd6f740 1 -- 172.21.15.37:0/3546292664 wait complete. 2024-03-26T06:33:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62cd6f740 20 client.25862 trim_cache size 0 max 16384 2024-03-26T06:33:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:33:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:33:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:33:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:33:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62a189640 1 -- 172.21.15.37:0/3127151398 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff60c0a2a10 msgr2=0x7ff60c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62a189640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff60c0a2a10 0x7ff60c0a4ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff61c022000 tx=0x7ff61c023000 comp rx=0 tx=0).stop 2024-03-26T06:33:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62a189640 1 -- 172.21.15.37:0/3127151398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff624100440 msgr2=0x7ff62419d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62a189640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff624100440 0x7ff62419d760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff61400e9f0 tx=0x7ff61400eeb0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62a189640 1 -- 172.21.15.37:0/3127151398 shutdown_connections 2024-03-26T06:33:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62a189640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff60c0a2a10 0x7ff60c0a4ed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62a189640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624197910 0x7ff624197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62a189640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff624105510 0x7ff62419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62a189640 1 --2- 172.21.15.37:0/3127151398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff624100440 0x7ff62419d760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.497+0000 7ff62a189640 1 -- 172.21.15.37:0/3127151398 >> 172.21.15.37:0/3127151398 conn(0x7ff624077350 msgr2=0x7ff624069760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.498+0000 7ff62a189640 1 -- 172.21.15.37:0/3127151398 shutdown_connections 2024-03-26T06:33:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:06.498+0000 7ff62a189640 1 -- 172.21.15.37:0/3127151398 wait complete. 2024-03-26T06:33:06.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 384176 4.15 MB/sec execute 401 sec latency 3365.933 ms 2024-03-26T06:33:07.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 386295 4.17 MB/sec execute 402 sec latency 15.507 ms 2024-03-26T06:33:08.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:07 smithi037 ceph-mon[31974]: pgmap v315: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2024-03-26T06:33:08.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:07 smithi066 ceph-mon[36504]: pgmap v315: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2024-03-26T06:33:08.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:07 smithi149 ceph-mon[36681]: pgmap v315: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2024-03-26T06:33:08.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 387343 4.17 MB/sec execute 403 sec latency 149.290 ms 2024-03-26T06:33:09.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 387343 4.16 MB/sec execute 404 sec latency 1149.390 ms 2024-03-26T06:33:10.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 387343 4.15 MB/sec execute 405 sec latency 2149.497 ms 2024-03-26T06:33:11.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:10 smithi066 ceph-mon[36504]: pgmap v316: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2024-03-26T06:33:11.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:10 smithi149 ceph-mon[36681]: pgmap v316: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2024-03-26T06:33:11.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:10 smithi037 ceph-mon[31974]: pgmap v316: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2024-03-26T06:33:11.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 387399 4.14 MB/sec execute 406 sec latency 2945.158 ms 2024-03-26T06:33:12.522 INFO:tasks.workunit.client.0.smithi037.stdout: 1 387399 4.13 MB/sec execute 407 sec latency 1162.557 ms 2024-03-26T06:33:13.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:12 smithi066 ceph-mon[36504]: pgmap v317: 97 pgs: 97 active+clean; 137 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-03-26T06:33:13.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:12 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:33:13.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:12 smithi149 ceph-mon[36681]: pgmap v317: 97 pgs: 97 active+clean; 137 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-03-26T06:33:13.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:12 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:33:13.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:12 smithi037 ceph-mon[31974]: pgmap v317: 97 pgs: 97 active+clean; 137 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-03-26T06:33:13.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:12 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:33:13.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 387399 4.12 MB/sec execute 408 sec latency 2162.633 ms 2024-03-26T06:33:14.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 387399 4.11 MB/sec execute 409 sec latency 3162.677 ms 2024-03-26T06:33:15.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:14 smithi066 ceph-mon[36504]: pgmap v318: 97 pgs: 97 active+clean; 137 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-03-26T06:33:15.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:14 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:14 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:14 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:14 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:14 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:15.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:14 smithi066 ceph-mon[36504]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T06:33:15.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:14 smithi149 ceph-mon[36681]: pgmap v318: 97 pgs: 97 active+clean; 137 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-03-26T06:33:15.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:14 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:14 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:14 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:14 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:14 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:15.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:14 smithi149 ceph-mon[36681]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T06:33:15.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:14 smithi037 ceph-mon[31974]: pgmap v318: 97 pgs: 97 active+clean; 137 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-03-26T06:33:15.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:14 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:14 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:14 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:14 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:15.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:14 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:15.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:14 smithi037 ceph-mon[31974]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T06:33:15.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 387399 4.10 MB/sec execute 410 sec latency 4162.712 ms 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:33:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:33:16.100 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:15 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:33:16.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:33:16.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:15 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T06:33:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:33:16.201 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:33:16.201 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:15 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:33:16.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 387745 4.10 MB/sec execute 411 sec latency 4954.218 ms 2024-03-26T06:33:17.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:16 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-03-26T06:33:17.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:16 smithi066 ceph-mon[36504]: Unable to set osd_memory_target on smithi037 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-03-26T06:33:17.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:16 smithi066 ceph-mon[36504]: pgmap v320: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-03-26T06:33:17.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:16 smithi066 ceph-mon[36504]: Adjusting osd_memory_target on smithi066 to 174.1M 2024-03-26T06:33:17.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:16 smithi066 ceph-mon[36504]: Unable to set osd_memory_target on smithi066 to 182611251: error parsing value: Value '182611251' is below minimum 939524096 2024-03-26T06:33:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:16 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-03-26T06:33:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:16 smithi149 ceph-mon[36681]: Unable to set osd_memory_target on smithi037 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-03-26T06:33:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:16 smithi149 ceph-mon[36681]: pgmap v320: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-03-26T06:33:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:16 smithi149 ceph-mon[36681]: Adjusting osd_memory_target on smithi066 to 174.1M 2024-03-26T06:33:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:16 smithi149 ceph-mon[36681]: Unable to set osd_memory_target on smithi066 to 182611251: error parsing value: Value '182611251' is below minimum 939524096 2024-03-26T06:33:17.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:16 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-03-26T06:33:17.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:16 smithi037 ceph-mon[31974]: Unable to set osd_memory_target on smithi037 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-03-26T06:33:17.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:16 smithi037 ceph-mon[31974]: pgmap v320: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-03-26T06:33:17.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:16 smithi037 ceph-mon[31974]: Adjusting osd_memory_target on smithi066 to 174.1M 2024-03-26T06:33:17.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:16 smithi037 ceph-mon[31974]: Unable to set osd_memory_target on smithi066 to 182611251: error parsing value: Value '182611251' is below minimum 939524096 2024-03-26T06:33:17.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 390051 4.12 MB/sec execute 412 sec latency 20.470 ms 2024-03-26T06:33:18.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 391407 4.12 MB/sec execute 413 sec latency 11.922 ms 2024-03-26T06:33:19.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:18 smithi066 ceph-mon[36504]: pgmap v321: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-03-26T06:33:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:18 smithi149 ceph-mon[36681]: pgmap v321: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-03-26T06:33:19.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:18 smithi037 ceph-mon[31974]: pgmap v321: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-03-26T06:33:19.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 393746 4.15 MB/sec execute 414 sec latency 27.089 ms 2024-03-26T06:33:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.587+0000 7fac252e8640 1 -- 172.21.15.37:0/740767644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac20107400 msgr2=0x7fac201077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.587+0000 7fac252e8640 1 --2- 172.21.15.37:0/740767644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac20107400 0x7fac201077e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fac14009900 tx=0x7fac1402bc30 comp rx=0 tx=0).stop 2024-03-26T06:33:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.587+0000 7fac252e8640 1 -- 172.21.15.37:0/740767644 shutdown_connections 2024-03-26T06:33:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.587+0000 7fac252e8640 1 --2- 172.21.15.37:0/740767644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac20069310 0x7fac2010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.587+0000 7fac252e8640 1 --2- 172.21.15.37:0/740767644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac20068970 0x7fac20068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.587+0000 7fac252e8640 1 --2- 172.21.15.37:0/740767644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac20107400 0x7fac201077e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.587+0000 7fac252e8640 1 -- 172.21.15.37:0/740767644 >> 172.21.15.37:0/740767644 conn(0x7fac20077040 msgr2=0x7fac20077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.587+0000 7fac252e8640 1 -- 172.21.15.37:0/740767644 shutdown_connections 2024-03-26T06:33:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.587+0000 7fac252e8640 1 -- 172.21.15.37:0/740767644 wait complete. 2024-03-26T06:33:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac252e8640 1 Processor -- start 2024-03-26T06:33:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac252e8640 1 -- start start 2024-03-26T06:33:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac252e8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac20068970 0x7fac2019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac252e8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac20069310 0x7fac2019dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac252e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac20107400 0x7fac20197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac252e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac20111650 con 0x7fac20107400 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac252e8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201114d0 con 0x7fac20068970 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac252e8640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201117d0 con 0x7fac20069310 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac20107400 0x7fac20197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac20068970 0x7fac2019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac20068970 0x7fac2019d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:44272/0 (socket says 172.21.15.37:44272) 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac20107400 0x7fac20197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38486/0 (socket says 172.21.15.37:38486) 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1effd640 1 -- 172.21.15.37:0/1204030410 learned_addr learned my addr 172.21.15.37:0/1204030410 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac20069310 0x7fac2019dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1f7fe640 1 -- 172.21.15.37:0/1204030410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac20069310 msgr2=0x7fac2019dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1f7fe640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac20069310 0x7fac2019dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1f7fe640 1 -- 172.21.15.37:0/1204030410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac20068970 msgr2=0x7fac2019d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1f7fe640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac20068970 0x7fac2019d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1f7fe640 1 -- 172.21.15.37:0/1204030410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac20198010 con 0x7fac20107400 2024-03-26T06:33:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.588+0000 7fac1effd640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac20068970 0x7fac2019d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:33:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.589+0000 7fac1f7fe640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac20107400 0x7fac20197890 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fac18005fd0 tx=0x7fac1800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.589+0000 7fac07fff640 1 -- 172.21.15.37:0/1204030410 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1800cd80 con 0x7fac20107400 2024-03-26T06:33:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.589+0000 7fac252e8640 1 -- 172.21.15.37:0/1204030410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac20198300 con 0x7fac20107400 2024-03-26T06:33:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.589+0000 7fac07fff640 1 -- 172.21.15.37:0/1204030410 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18005020 con 0x7fac20107400 2024-03-26T06:33:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.589+0000 7fac07fff640 1 -- 172.21.15.37:0/1204030410 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18010720 con 0x7fac20107400 2024-03-26T06:33:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.589+0000 7fac252e8640 1 -- 172.21.15.37:0/1204030410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac201a45a0 con 0x7fac20107400 2024-03-26T06:33:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.591+0000 7fac07fff640 1 -- 172.21.15.37:0/1204030410 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fac180051d0 con 0x7fac20107400 2024-03-26T06:33:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac27f9e740 1 Processor -- start 2024-03-26T06:33:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac27f9e740 1 -- start start 2024-03-26T06:33:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac07fff640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac080a2a10 0x7fac080a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b67d40 0x560df3b66a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3b67450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac07fff640 1 -- 172.21.15.37:0/1204030410 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fac180cf480 con 0x7fac20107400 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac080a2a10 0x7fac080a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b72940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac27f9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df3acb5f0 con 0x560df3b70550 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac27f9e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df3a74930 con 0x560df3b66fd0 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac27f9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df3ac7690 con 0x560df3b67d40 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b72940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b72940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38492/0 (socket says 172.21.15.37:38492) 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1f7fe640 1 -- 172.21.15.37:0/880746976 learned_addr learned my addr 172.21.15.37:0/880746976 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1effd640 1 --2- 172.21.15.37:0/880746976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b67d40 0x560df3b66a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/880746976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3b67450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1f7fe640 1 -- 172.21.15.37:0/880746976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b67d40 msgr2=0x560df3b66a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1f7fe640 1 --2- 172.21.15.37:0/880746976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b67d40 0x560df3b66a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1f7fe640 1 -- 172.21.15.37:0/880746976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 msgr2=0x560df3b67450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1f7fe640 1 --2- 172.21.15.37:0/880746976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3b67450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1f7fe640 1 -- 172.21.15.37:0/880746976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560df37e0210 con 0x560df3b70550 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac080a2a10 0x7fac080a4ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac200775d0 tx=0x7fac10011920 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.592+0000 7fac1effd640 1 --2- 172.21.15.37:0/880746976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b67d40 0x560df3b66a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.593+0000 7fac1f7fe640 1 --2- 172.21.15.37:0/880746976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b72940 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fac180a3cf0 tx=0x7fac1809b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b9dac0d6ab0d50 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.593+0000 7fac057fa640 1 -- 172.21.15.37:0/880746976 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180afe60 con 0x560df3b70550 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.593+0000 7fac057fa640 1 -- 172.21.15.37:0/880746976 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18005540 con 0x560df3b70550 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.593+0000 7fac057fa640 1 -- 172.21.15.37:0/880746976 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180a3ff0 con 0x560df3b70550 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.593+0000 7fac27f9e740 1 -- 172.21.15.37:0/880746976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 msgr2=0x560df3b72940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.593+0000 7fac27f9e740 1 --2- 172.21.15.37:0/880746976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b72940 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fac180a3cf0 tx=0x7fac1809b6f0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 -- 172.21.15.37:0/880746976 shutdown_connections 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 --2- 172.21.15.37:0/880746976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b72940 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 --2- 172.21.15.37:0/880746976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3b67450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 --2- 172.21.15.37:0/880746976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b67d40 0x560df3b66a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 -- 172.21.15.37:0/880746976 >> 172.21.15.37:0/880746976 conn(0x560df3b2e5b0 msgr2=0x560df3b61cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 -- 172.21.15.37:0/880746976 shutdown_connections 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 -- 172.21.15.37:0/880746976 wait complete. 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 -- start start 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 Processor -- start 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b66fd0 0x560df3b6d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b67d40 0x560df3b6db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b6e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df3b73280 con 0x560df3b70550 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df3a87d90 con 0x560df3b67d40 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df3b73400 con 0x560df3b66fd0 2024-03-26T06:33:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac27f9e740 10 client.0 _ll_register_callbacks cb 0x560df37dfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b6e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac1e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b67d40 0x560df3b6db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b6e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38498/0 (socket says 172.21.15.37:38498) 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac1f7fe640 1 -- 172.21.15.37:0/2279705416 learned_addr learned my addr 172.21.15.37:0/2279705416 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.594+0000 7fac1e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b67d40 0x560df3b6db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:44282/0 (socket says 172.21.15.37:44282) 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac1effd640 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b66fd0 0x560df3b6d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac1e7fc640 1 -- 172.21.15.37:0/2279705416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b66fd0 msgr2=0x560df3b6d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b66fd0 0x560df3b6d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac1e7fc640 1 -- 172.21.15.37:0/2279705416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 msgr2=0x560df3b6e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b6e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac1e7fc640 1 -- 172.21.15.37:0/2279705416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560df38ac620 con 0x560df3b67d40 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac1f7fe640 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b6e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:33:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b67d40 0x560df3b6db00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fac10004510 tx=0x7fac10004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c64cb85e94a1e2a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac02ff5640 1 -- 172.21.15.37:0/2279705416 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1001a040 con 0x560df3b67d40 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac27f9e740 1 -- 172.21.15.37:0/2279705416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac18005540 con 0x560df3b67d40 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac02ff5640 1 -- 172.21.15.37:0/2279705416 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fac10003a00 con 0x560df3b67d40 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac02ff5640 1 -- 172.21.15.37:0/2279705416 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac10003c00 con 0x560df3b67d40 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac27f9e740 10 client.25874 fetch_fsmap learned FSMap version 44 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.595+0000 7fac27f9e740 1 -- 172.21.15.37:0/2279705416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x560df3b7f5b0 con 0x560df3b67d40 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.596+0000 7fac02ff5640 1 -- 172.21.15.37:0/2279705416 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18005540 con 0x560df3b67d40 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.596+0000 7fac02ff5640 1 -- 172.21.15.37:0/2279705416 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fac1001d030 con 0x560df3b67d40 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.596+0000 7fac27f9e740 10 client.25874 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.596+0000 7fac27f9e740 10 client.25874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.596+0000 7fac27f9e740 20 client.25874 populate_metadata read hostname 'smithi037' 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.596+0000 7fac27f9e740 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560df3b80a20 0x560df3b82e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.596+0000 7fac27f9e740 4 client.25874 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.596+0000 7fac27f9e740 1 -- 172.21.15.37:0/2279705416 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560df3ad8b30 con 0x560df3b80a20 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.596+0000 7fac1effd640 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560df3b80a20 0x560df3b82e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.596+0000 7fac02ff5640 10 client.25874 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.596+0000 7fac1effd640 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560df3b80a20 0x560df3b82e10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f460381de0bbd8 server_cookie=632ec67f1b1a0bb1 in_seq=0 out_seq=0 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac02ff5640 1 -- 172.21.15.37:0/2279705416 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560df3ad8b30 con 0x560df3b80a20 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac02ff5640 10 client.25874 handle_command_reply: tid=0 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 1 client.25874 shutdown 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 1 -- 172.21.15.37:0/2279705416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b67d40 msgr2=0x560df3b6db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b67d40 0x560df3b6db00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fac10004510 tx=0x7fac10004730 comp rx=0 tx=0).stop 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 1 -- 172.21.15.37:0/2279705416 shutdown_connections 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560df3b80a20 0x560df3b82e10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b70550 0x560df3b6e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b67d40 0x560df3b6db00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac02ff5640 0 client.25874 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 1 --2- 172.21.15.37:0/2279705416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b66fd0 0x560df3b6d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 1 -- 172.21.15.37:0/2279705416 >> 172.21.15.37:0/2279705416 conn(0x560df3b2e5b0 msgr2=0x560df3b63a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 1 -- 172.21.15.37:0/2279705416 shutdown_connections 2024-03-26T06:33:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 1 -- 172.21.15.37:0/2279705416 wait complete. 2024-03-26T06:33:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.597+0000 7fac27f9e740 20 client.25874 trim_cache size 0 max 16384 2024-03-26T06:33:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.611+0000 7fac27f9e740 1 Processor -- start 2024-03-26T06:33:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac27f9e740 1 -- start start 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3b6fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3b6a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b70550 0x560df3b6ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac27f9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201acc00 con 0x560df3b67d40 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac27f9e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201ae580 con 0x560df3b66fd0 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac27f9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201ad260 con 0x560df3b70550 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac1e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3b6a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac1e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3b6a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38502/0 (socket says 172.21.15.37:38502) 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac1e7fc640 1 -- 172.21.15.37:0/3839631506 learned_addr learned my addr 172.21.15.37:0/3839631506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac1effd640 1 --2- 172.21.15.37:0/3839631506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3b6fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac1f7fe640 1 --2- 172.21.15.37:0/3839631506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b70550 0x560df3b6ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac1e7fc640 1 -- 172.21.15.37:0/3839631506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b70550 msgr2=0x560df3b6ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/3839631506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b70550 0x560df3b6ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac1e7fc640 1 -- 172.21.15.37:0/3839631506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 msgr2=0x560df3b6fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/3839631506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3b6fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac1e7fc640 1 -- 172.21.15.37:0/3839631506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560df3b6bb30 con 0x560df3b67d40 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/3839631506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3b6a860 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fac1000ff10 tx=0x7fac1000ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f001a27e36a589a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.612+0000 7fac04ff9640 1 -- 172.21.15.37:0/3839631506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1001a070 con 0x560df3b67d40 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.613+0000 7fac04ff9640 1 -- 172.21.15.37:0/3839631506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac10036030 con 0x560df3b67d40 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.613+0000 7fac04ff9640 1 -- 172.21.15.37:0/3839631506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10003a60 con 0x560df3b67d40 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.613+0000 7fac27f9e740 1 -- 172.21.15.37:0/3839631506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 msgr2=0x560df3b6a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.613+0000 7fac27f9e740 1 --2- 172.21.15.37:0/3839631506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3b6a860 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fac1000ff10 tx=0x7fac1000ff40 comp rx=0 tx=0).stop 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.613+0000 7fac27f9e740 1 -- 172.21.15.37:0/3839631506 shutdown_connections 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.613+0000 7fac27f9e740 1 --2- 172.21.15.37:0/3839631506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b70550 0x560df3b6ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.613+0000 7fac27f9e740 1 --2- 172.21.15.37:0/3839631506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3b6a860 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.613+0000 7fac27f9e740 1 --2- 172.21.15.37:0/3839631506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3b6fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.613+0000 7fac27f9e740 1 -- 172.21.15.37:0/3839631506 >> 172.21.15.37:0/3839631506 conn(0x560df3b2e5b0 msgr2=0x560df3b6beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.613+0000 7fac27f9e740 1 -- 172.21.15.37:0/3839631506 shutdown_connections 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.613+0000 7fac27f9e740 1 -- 172.21.15.37:0/3839631506 wait complete. 2024-03-26T06:33:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac27f9e740 1 -- start start 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac27f9e740 1 Processor -- start 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3bd12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3bd1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac27f9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b70550 0x560df3b76850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac27f9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201adf20 con 0x560df3b67d40 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac27f9e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201ad8c0 con 0x560df3b66fd0 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac27f9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201aaec0 con 0x560df3b70550 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac27f9e740 10 client.0 _ll_register_callbacks cb 0x7fac18097d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3bd1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3bd1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38510/0 (socket says 172.21.15.37:38510) 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1e7fc640 1 -- 172.21.15.37:0/1910556321 learned_addr learned my addr 172.21.15.37:0/1910556321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1effd640 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3bd12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1f7fe640 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b70550 0x560df3b76850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1e7fc640 1 -- 172.21.15.37:0/1910556321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b70550 msgr2=0x560df3b76850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b70550 0x560df3b76850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1e7fc640 1 -- 172.21.15.37:0/1910556321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 msgr2=0x560df3bd12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3bd12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1e7fc640 1 -- 172.21.15.37:0/1910556321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560df3b76fc0 con 0x560df3b67d40 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1f7fe640 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b70550 0x560df3b76850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac1e7fc640 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3bd1820 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fac10004510 tx=0x7fac10009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6ec9959d318c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.614+0000 7fac02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac02ff5640 1 -- 172.21.15.37:0/1910556321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10040060 con 0x560df3b67d40 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac27f9e740 1 -- 172.21.15.37:0/1910556321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac10036030 con 0x560df3b67d40 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac02ff5640 1 -- 172.21.15.37:0/1910556321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fac10020790 con 0x560df3b67d40 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac02ff5640 1 -- 172.21.15.37:0/1910556321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac1001a040 con 0x560df3b67d40 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac27f9e740 10 client.16344 fetch_fsmap learned FSMap version 44 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac27f9e740 1 -- 172.21.15.37:0/1910556321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x560df3bd8a60 con 0x560df3b67d40 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac02ff5640 1 -- 172.21.15.37:0/1910556321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac10036030 con 0x560df3b67d40 2024-03-26T06:33:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac02ff5640 1 -- 172.21.15.37:0/1910556321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fac10031ad0 con 0x560df3b67d40 2024-03-26T06:33:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac27f9e740 10 client.16344 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:33:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac27f9e740 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:33:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac27f9e740 20 client.16344 populate_metadata read hostname 'smithi037' 2024-03-26T06:33:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac27f9e740 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560df3bd8ff0 0x560df3bdb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac27f9e740 4 client.16344 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:33:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac27f9e740 1 -- 172.21.15.37:0/1910556321 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560df3ad2e80 con 0x560df3bd8ff0 2024-03-26T06:33:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.615+0000 7fac1effd640 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560df3bd8ff0 0x560df3bdb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.616+0000 7fac1effd640 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560df3bd8ff0 0x560df3bdb410 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7294f59358bb34ea server_cookie=41892d73a3ea87b5 in_seq=0 out_seq=0 2024-03-26T06:33:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.616+0000 7fac02ff5640 10 client.16344 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac02ff5640 1 -- 172.21.15.37:0/1910556321 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560df3ad2e80 con 0x560df3bd8ff0 2024-03-26T06:33:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac02ff5640 10 client.16344 handle_command_reply: tid=0 2024-03-26T06:33:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 1 client.16344 shutdown 2024-03-26T06:33:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 1 -- 172.21.15.37:0/1910556321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 msgr2=0x560df3bd1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3bd1820 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fac10004510 tx=0x7fac10009270 comp rx=0 tx=0).stop 2024-03-26T06:33:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 1 -- 172.21.15.37:0/1910556321 shutdown_connections 2024-03-26T06:33:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560df3bd8ff0 0x560df3bdb410 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560df3b70550 0x560df3b76850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac02ff5640 0 client.16344 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560df3b67d40 0x560df3bd1820 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 1 --2- 172.21.15.37:0/1910556321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560df3b66fd0 0x560df3bd12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 1 -- 172.21.15.37:0/1910556321 >> 172.21.15.37:0/1910556321 conn(0x560df3b2e5b0 msgr2=0x560df3b605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 1 -- 172.21.15.37:0/1910556321 shutdown_connections 2024-03-26T06:33:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 1 -- 172.21.15.37:0/1910556321 wait complete. 2024-03-26T06:33:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.646+0000 7fac27f9e740 20 client.16344 trim_cache size 0 max 16384 2024-03-26T06:33:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:33:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:33:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:33:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 -- 172.21.15.37:0/1204030410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac080a2a10 msgr2=0x7fac080a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac080a2a10 0x7fac080a4ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac200775d0 tx=0x7fac10011920 comp rx=0 tx=0).stop 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 -- 172.21.15.37:0/1204030410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac20107400 msgr2=0x7fac20197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac20107400 0x7fac20197890 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fac18005fd0 tx=0x7fac1800ee60 comp rx=0 tx=0).stop 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 -- 172.21.15.37:0/1204030410 shutdown_connections 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fac080a2a10 0x7fac080a4ed0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac20107400 0x7fac20197890 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac20069310 0x7fac2019dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 --2- 172.21.15.37:0/1204030410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac20068970 0x7fac2019d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 -- 172.21.15.37:0/1204030410 >> 172.21.15.37:0/1204030410 conn(0x7fac20077040 msgr2=0x7fac200fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 -- 172.21.15.37:0/1204030410 shutdown_connections 2024-03-26T06:33:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:19.647+0000 7fac252e8640 1 -- 172.21.15.37:0/1204030410 wait complete. 2024-03-26T06:33:20.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 394557 4.15 MB/sec execute 415 sec latency 224.671 ms 2024-03-26T06:33:21.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:20 smithi066 ceph-mon[36504]: pgmap v322: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-03-26T06:33:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:20 smithi149 ceph-mon[36681]: pgmap v322: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-03-26T06:33:21.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:20 smithi037 ceph-mon[31974]: pgmap v322: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-03-26T06:33:21.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 394937 4.14 MB/sec execute 416 sec latency 1018.425 ms 2024-03-26T06:33:22.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 396371 4.14 MB/sec execute 417 sec latency 19.366 ms 2024-03-26T06:33:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:22 smithi066 ceph-mon[36504]: pgmap v323: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:33:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:22 smithi149 ceph-mon[36681]: pgmap v323: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:33:23.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:22 smithi037 ceph-mon[31974]: pgmap v323: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:33:23.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 397971 4.16 MB/sec execute 418 sec latency 12.370 ms 2024-03-26T06:33:24.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 399900 4.17 MB/sec execute 419 sec latency 27.041 ms 2024-03-26T06:33:25.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:24 smithi066 ceph-mon[36504]: pgmap v324: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:33:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:24 smithi149 ceph-mon[36681]: pgmap v324: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:33:25.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:24 smithi037 ceph-mon[31974]: pgmap v324: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:33:25.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 401467 4.19 MB/sec execute 420 sec latency 17.237 ms 2024-03-26T06:33:26.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 401838 4.18 MB/sec execute 421 sec latency 781.627 ms 2024-03-26T06:33:27.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:26 smithi066 ceph-mon[36504]: pgmap v325: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-03-26T06:33:27.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:26 smithi149 ceph-mon[36681]: pgmap v325: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-03-26T06:33:27.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:26 smithi037 ceph-mon[31974]: pgmap v325: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-03-26T06:33:27.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 403289 4.18 MB/sec execute 422 sec latency 16.395 ms 2024-03-26T06:33:28.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 404858 4.20 MB/sec execute 423 sec latency 12.051 ms 2024-03-26T06:33:29.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:28 smithi066 ceph-mon[36504]: pgmap v326: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-03-26T06:33:29.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:28 smithi149 ceph-mon[36681]: pgmap v326: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-03-26T06:33:29.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:28 smithi037 ceph-mon[31974]: pgmap v326: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-03-26T06:33:29.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 405123 4.19 MB/sec execute 424 sec latency 791.923 ms 2024-03-26T06:33:30.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 405123 4.18 MB/sec execute 425 sec latency 1792.042 ms 2024-03-26T06:33:31.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:30 smithi066 ceph-mon[36504]: pgmap v327: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-03-26T06:33:31.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:30 smithi149 ceph-mon[36681]: pgmap v327: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-03-26T06:33:31.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:30 smithi037 ceph-mon[31974]: pgmap v327: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-03-26T06:33:31.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 405469 4.18 MB/sec execute 426 sec latency 2580.317 ms 2024-03-26T06:33:32.523 INFO:tasks.workunit.client.0.smithi037.stdout: 1 407776 4.20 MB/sec execute 427 sec latency 15.502 ms 2024-03-26T06:33:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.739+0000 7f6c99f5b640 1 -- 172.21.15.37:0/2600664654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c94100440 msgr2=0x7f6c941008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.739+0000 7f6c99f5b640 1 --2- 172.21.15.37:0/2600664654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c94100440 0x7f6c941008c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009900 tx=0x7f6c8802bc30 comp rx=0 tx=0).stop 2024-03-26T06:33:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.740+0000 7f6c99f5b640 1 -- 172.21.15.37:0/2600664654 shutdown_connections 2024-03-26T06:33:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.740+0000 7f6c99f5b640 1 --2- 172.21.15.37:0/2600664654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c94100e00 0x7f6c9410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.740+0000 7f6c99f5b640 1 --2- 172.21.15.37:0/2600664654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c94100440 0x7f6c941008c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.740+0000 7f6c99f5b640 1 --2- 172.21.15.37:0/2600664654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c94105190 0x7f6c94105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.740+0000 7f6c99f5b640 1 -- 172.21.15.37:0/2600664654 >> 172.21.15.37:0/2600664654 conn(0x7f6c94077040 msgr2=0x7f6c94077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.740+0000 7f6c99f5b640 1 -- 172.21.15.37:0/2600664654 shutdown_connections 2024-03-26T06:33:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.740+0000 7f6c99f5b640 1 -- 172.21.15.37:0/2600664654 wait complete. 2024-03-26T06:33:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.740+0000 7f6c99f5b640 1 Processor -- start 2024-03-26T06:33:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.740+0000 7f6c99f5b640 1 -- start start 2024-03-26T06:33:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c99f5b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c94100440 0x7f6c94198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c99f5b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c94100e00 0x7f6c94198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c99f5b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c94105190 0x7f6c941a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c99f5b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94110650 con 0x7f6c94100e00 2024-03-26T06:33:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c99f5b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941104d0 con 0x7f6c94100440 2024-03-26T06:33:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c99f5b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941107d0 con 0x7f6c94105190 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c94100e00 0x7f6c94198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c94100e00 0x7f6c94198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46312/0 (socket says 172.21.15.37:46312) 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c92ffd640 1 -- 172.21.15.37:0/1540217062 learned_addr learned my addr 172.21.15.37:0/1540217062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c937fe640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c94100440 0x7f6c94198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c93fff640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c94105190 0x7f6c941a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c92ffd640 1 -- 172.21.15.37:0/1540217062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c94105190 msgr2=0x7f6c941a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c94105190 0x7f6c941a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c92ffd640 1 -- 172.21.15.37:0/1540217062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c94100440 msgr2=0x7f6c94198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c94100440 0x7f6c94198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c92ffd640 1 -- 172.21.15.37:0/1540217062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c941a3f50 con 0x7f6c94100e00 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c937fe640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c94100440 0x7f6c94198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c94100e00 0x7f6c94198ed0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6c88005940 tx=0x7f6c88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c90ff9640 1 -- 172.21.15.37:0/1540217062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88041070 con 0x7f6c94100e00 2024-03-26T06:33:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c99f5b640 1 -- 172.21.15.37:0/1540217062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c941a4120 con 0x7f6c94100e00 2024-03-26T06:33:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.741+0000 7f6c90ff9640 1 -- 172.21.15.37:0/1540217062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c880054e0 con 0x7f6c94100e00 2024-03-26T06:33:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.742+0000 7f6c90ff9640 1 -- 172.21.15.37:0/1540217062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88003d00 con 0x7f6c94100e00 2024-03-26T06:33:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.742+0000 7f6c99f5b640 1 -- 172.21.15.37:0/1540217062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c941a45e0 con 0x7f6c94100e00 2024-03-26T06:33:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.744+0000 7f6c90ff9640 1 -- 172.21.15.37:0/1540217062 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6c88044070 con 0x7f6c94100e00 2024-03-26T06:33:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.744+0000 7f6c9cc1c740 1 Processor -- start 2024-03-26T06:33:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.744+0000 7f6c9cc1c740 1 -- start start 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c90ff9640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6c7c0a2a10 0x7f6c7c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 0x559ad1514a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 0x559ad1515450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad1520940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad14795f0 con 0x559ad1514fd0 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad1422930 con 0x559ad151e550 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad1475690 con 0x559ad1515d40 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c937fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 0x559ad1514a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c93fff640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6c7c0a2a10 0x7f6c7c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 0x559ad1515450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c937fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 0x559ad1514a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:60240/0 (socket says 172.21.15.37:60240) 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c937fe640 1 -- 172.21.15.37:0/1624013929 learned_addr learned my addr 172.21.15.37:0/1624013929 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c90ff9640 1 -- 172.21.15.37:0/1540217062 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6c880f02f0 con 0x7f6c94100e00 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c93fff640 1 --2- 172.21.15.37:0/1624013929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad1520940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c937fe640 1 -- 172.21.15.37:0/1624013929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 msgr2=0x559ad1520940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c937fe640 1 --2- 172.21.15.37:0/1624013929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad1520940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c937fe640 1 -- 172.21.15.37:0/1624013929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 msgr2=0x559ad1515450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c937fe640 1 --2- 172.21.15.37:0/1624013929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 0x559ad1515450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c937fe640 1 -- 172.21.15.37:0/1624013929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ad118e210 con 0x559ad1515d40 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c93fff640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6c7c0a2a10 0x7f6c7c0a4ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c005fd0 tx=0x7f6c8c009f00 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.745+0000 7f6c937fe640 1 --2- 172.21.15.37:0/1624013929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 0x559ad1514a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c840114c0 tx=0x7f6c840118c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=120c2a10710aee6f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c79ffb640 1 -- 172.21.15.37:0/1624013929 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84004020 con 0x559ad1515d40 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c79ffb640 1 -- 172.21.15.37:0/1624013929 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c840041c0 con 0x559ad1515d40 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c79ffb640 1 -- 172.21.15.37:0/1624013929 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400f860 con 0x559ad1515d40 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1624013929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 msgr2=0x559ad1514a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/1624013929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 0x559ad1514a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c840114c0 tx=0x7f6c840118c0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1624013929 shutdown_connections 2024-03-26T06:33:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/1624013929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad1520940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/1624013929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 0x559ad1515450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/1624013929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 0x559ad1514a90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1624013929 >> 172.21.15.37:0/1624013929 conn(0x559ad14dc5b0 msgr2=0x559ad150fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1624013929 shutdown_connections 2024-03-26T06:33:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.746+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1624013929 wait complete. 2024-03-26T06:33:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.747+0000 7f6c9cc1c740 1 -- start start 2024-03-26T06:33:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.747+0000 7f6c9cc1c740 1 Processor -- start 2024-03-26T06:33:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.747+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 0x559ad151b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.747+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 0x559ad151bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.747+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad151c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.747+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad1521280 con 0x559ad1514fd0 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.747+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad1435d90 con 0x559ad151e550 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.747+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad1521400 con 0x559ad1515d40 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.747+0000 7f6c9cc1c740 10 client.0 _ll_register_callbacks cb 0x559ad118da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.747+0000 7f6c937fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 0x559ad151b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.747+0000 7f6c937fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 0x559ad151b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46332/0 (socket says 172.21.15.37:46332) 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c937fe640 1 -- 172.21.15.37:0/1154562120 learned_addr learned my addr 172.21.15.37:0/1154562120 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 0x559ad151bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c93fff640 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad151c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c937fe640 1 -- 172.21.15.37:0/1154562120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 msgr2=0x559ad151bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c937fe640 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 0x559ad151bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c937fe640 1 -- 172.21.15.37:0/1154562120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 msgr2=0x559ad151c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c937fe640 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad151c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c937fe640 1 -- 172.21.15.37:0/1154562120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ad125a620 con 0x559ad1514fd0 2024-03-26T06:33:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c937fe640 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 0x559ad151b5c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6c84005f10 tx=0x7f6c84003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f901cd45e693f9ca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c72ffd640 1 -- 172.21.15.37:0/1154562120 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84027020 con 0x559ad1514fd0 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c72ffd640 1 -- 172.21.15.37:0/1154562120 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6c84004370 con 0x559ad1514fd0 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1154562120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c840041c0 con 0x559ad1514fd0 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c72ffd640 1 -- 172.21.15.37:0/1154562120 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c84032050 con 0x559ad1514fd0 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c9cc1c740 10 client.16356 fetch_fsmap learned FSMap version 44 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1154562120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x559ad152d5b0 con 0x559ad1514fd0 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.748+0000 7f6c72ffd640 1 -- 172.21.15.37:0/1154562120 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c840041c0 con 0x559ad1514fd0 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.749+0000 7f6c72ffd640 1 -- 172.21.15.37:0/1154562120 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6c84004cc0 con 0x559ad1514fd0 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.749+0000 7f6c9cc1c740 10 client.16356 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.749+0000 7f6c9cc1c740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.749+0000 7f6c9cc1c740 20 client.16356 populate_metadata read hostname 'smithi037' 2024-03-26T06:33:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.749+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x559ad152ea10 0x559ad1530e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.749+0000 7f6c9cc1c740 4 client.16356 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.749+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1154562120 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ad1486b30 con 0x559ad152ea10 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.749+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x559ad152ea10 0x559ad1530e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.749+0000 7f6c72ffd640 10 client.16356 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.749+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x559ad152ea10 0x559ad1530e00 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963bcf4a885f70ae server_cookie=f855e748be3306f8 in_seq=0 out_seq=0 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.750+0000 7f6c72ffd640 1 -- 172.21.15.37:0/1154562120 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559ad1486b30 con 0x559ad152ea10 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.750+0000 7f6c72ffd640 10 client.16356 handle_command_reply: tid=0 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.750+0000 7f6c9cc1c740 1 client.16356 shutdown 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1154562120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 msgr2=0x559ad151b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 0x559ad151b5c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6c84005f10 tx=0x7f6c84003d30 comp rx=0 tx=0).stop 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1154562120 shutdown_connections 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x559ad152ea10 0x559ad1530e00 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad151c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1515d40 0x559ad151bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c72ffd640 0 client.16356 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/1154562120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1514fd0 0x559ad151b5c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1154562120 >> 172.21.15.37:0/1154562120 conn(0x559ad14dc5b0 msgr2=0x559ad1511a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1154562120 shutdown_connections 2024-03-26T06:33:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/1154562120 wait complete. 2024-03-26T06:33:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.751+0000 7f6c9cc1c740 20 client.16356 trim_cache size 0 max 16384 2024-03-26T06:33:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.765+0000 7f6c9cc1c740 1 Processor -- start 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.765+0000 7f6c9cc1c740 1 -- start start 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.765+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1514fd0 0x559ad151dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.765+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1519e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.765+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad151a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.765+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941acc00 con 0x559ad1515d40 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.765+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941ae580 con 0x559ad151e550 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.765+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941ad260 con 0x559ad1514fd0 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.765+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1519e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.766+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1519e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46334/0 (socket says 172.21.15.37:46334) 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.766+0000 7f6c92ffd640 1 -- 172.21.15.37:0/2503042464 learned_addr learned my addr 172.21.15.37:0/2503042464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.766+0000 7f6c937fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1514fd0 0x559ad151dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.766+0000 7f6c93fff640 1 --2- 172.21.15.37:0/2503042464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad151a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.766+0000 7f6c92ffd640 1 -- 172.21.15.37:0/2503042464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1514fd0 msgr2=0x559ad151dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.766+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/2503042464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1514fd0 0x559ad151dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.766+0000 7f6c92ffd640 1 -- 172.21.15.37:0/2503042464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 msgr2=0x559ad151a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.766+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/2503042464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad151a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.766+0000 7f6c92ffd640 1 -- 172.21.15.37:0/2503042464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ad1520da0 con 0x559ad1515d40 2024-03-26T06:33:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.766+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/2503042464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1519e90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6c88000c00 tx=0x7f6c8802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9eaed7ef5ae6ac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.769+0000 7f6c797fa640 1 -- 172.21.15.37:0/2503042464 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8804b040 con 0x559ad1515d40 2024-03-26T06:33:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.769+0000 7f6c797fa640 1 -- 172.21.15.37:0/2503042464 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c8802ccd0 con 0x559ad1515d40 2024-03-26T06:33:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.769+0000 7f6c797fa640 1 -- 172.21.15.37:0/2503042464 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c880babb0 con 0x559ad1515d40 2024-03-26T06:33:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.770+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2503042464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 msgr2=0x559ad1519e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.770+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/2503042464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1519e90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6c88000c00 tx=0x7f6c8802bc30 comp rx=0 tx=0).stop 2024-03-26T06:33:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.770+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2503042464 shutdown_connections 2024-03-26T06:33:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.770+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/2503042464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad151e550 0x559ad151a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.770+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/2503042464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1519e90 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.770+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/2503042464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad1514fd0 0x559ad151dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.770+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2503042464 >> 172.21.15.37:0/2503042464 conn(0x559ad14dc5b0 msgr2=0x559ad150f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.770+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2503042464 shutdown_connections 2024-03-26T06:33:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.770+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2503042464 wait complete. 2024-03-26T06:33:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.770+0000 7f6c9cc1c740 1 -- start start 2024-03-26T06:33:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.770+0000 7f6c9cc1c740 1 Processor -- start 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad1514fd0 0x559ad1525960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1523d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c9cc1c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad151e550 0x559ad1524240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941adf20 con 0x559ad1515d40 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941ad8c0 con 0x559ad1514fd0 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c9cc1c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941aaec0 con 0x559ad151e550 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c9cc1c740 10 client.0 _ll_register_callbacks cb 0x7f6c84009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1523d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c937fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad1514fd0 0x559ad1525960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1523d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46344/0 (socket says 172.21.15.37:46344) 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c93fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad151e550 0x559ad1524240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c92ffd640 1 -- 172.21.15.37:0/2883977462 learned_addr learned my addr 172.21.15.37:0/2883977462 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c92ffd640 1 -- 172.21.15.37:0/2883977462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad151e550 msgr2=0x559ad1524240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/2883977462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad151e550 0x559ad1524240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c92ffd640 1 -- 172.21.15.37:0/2883977462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad1514fd0 msgr2=0x559ad1525960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/2883977462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad1514fd0 0x559ad1525960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c92ffd640 1 -- 172.21.15.37:0/2883977462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ad1524a40 con 0x559ad1515d40 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c92ffd640 1 --2- 172.21.15.37:0/2883977462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1523d00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6c880bcaa0 tx=0x7f6c880c5200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d2cfa04f3094f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c72ffd640 1 -- 172.21.15.37:0/2883977462 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8804b070 con 0x559ad1515d40 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c72ffd640 1 -- 172.21.15.37:0/2883977462 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6c880bae10 con 0x559ad1515d40 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.771+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2883977462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ad1480e80 con 0x559ad1515d40 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c72ffd640 1 -- 172.21.15.37:0/2883977462 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c880edcc0 con 0x559ad1515d40 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c9cc1c740 10 client.16362 fetch_fsmap learned FSMap version 44 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2883977462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x559ad1585d50 con 0x559ad1515d40 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c72ffd640 1 -- 172.21.15.37:0/2883977462 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559ad1480e80 con 0x559ad1515d40 2024-03-26T06:33:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c72ffd640 1 -- 172.21.15.37:0/2883977462 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6c880bcd70 con 0x559ad1515d40 2024-03-26T06:33:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c9cc1c740 10 client.16362 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:33:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c9cc1c740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:33:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c9cc1c740 20 client.16362 populate_metadata read hostname 'smithi037' 2024-03-26T06:33:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/2883977462 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x559ad1586400 0x559ad1588820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c9cc1c740 4 client.16362 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:33:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2883977462 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559ad13ff710 con 0x559ad1586400 2024-03-26T06:33:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.772+0000 7f6c937fe640 1 --2- 172.21.15.37:0/2883977462 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x559ad1586400 0x559ad1588820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.773+0000 7f6c72ffd640 10 client.16362 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.773+0000 7f6c937fe640 1 --2- 172.21.15.37:0/2883977462 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x559ad1586400 0x559ad1588820 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24fa628e50cfd046 server_cookie=bcfc172a20bc40d4 in_seq=0 out_seq=0 2024-03-26T06:33:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c72ffd640 1 -- 172.21.15.37:0/2883977462 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559ad13ff710 con 0x559ad1586400 2024-03-26T06:33:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c72ffd640 10 client.16362 handle_command_reply: tid=0 2024-03-26T06:33:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 1 client.16362 shutdown 2024-03-26T06:33:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2883977462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 msgr2=0x559ad1523d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/2883977462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1523d00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6c880bcaa0 tx=0x7f6c880c5200 comp rx=0 tx=0).stop 2024-03-26T06:33:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2883977462 shutdown_connections 2024-03-26T06:33:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/2883977462 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x559ad1586400 0x559ad1588820 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/2883977462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ad151e550 0x559ad1524240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c72ffd640 0 client.16362 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/2883977462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ad1515d40 0x559ad1523d00 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 1 --2- 172.21.15.37:0/2883977462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ad1514fd0 0x559ad1525960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2883977462 >> 172.21.15.37:0/2883977462 conn(0x559ad14dc5b0 msgr2=0x559ad1519e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2883977462 shutdown_connections 2024-03-26T06:33:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 1 -- 172.21.15.37:0/2883977462 wait complete. 2024-03-26T06:33:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.805+0000 7f6c9cc1c740 20 client.16362 trim_cache size 0 max 16384 2024-03-26T06:33:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:33:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:33:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:33:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:33:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 -- 172.21.15.37:0/1540217062 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6c7c0a2a10 msgr2=0x7f6c7c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6c7c0a2a10 0x7f6c7c0a4ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c005fd0 tx=0x7f6c8c009f00 comp rx=0 tx=0).stop 2024-03-26T06:33:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 -- 172.21.15.37:0/1540217062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c94100e00 msgr2=0x7f6c94198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c94100e00 0x7f6c94198ed0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6c88005940 tx=0x7f6c88004270 comp rx=0 tx=0).stop 2024-03-26T06:33:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 -- 172.21.15.37:0/1540217062 shutdown_connections 2024-03-26T06:33:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6c7c0a2a10 0x7f6c7c0a4ed0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c94105190 0x7f6c941a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c94100e00 0x7f6c94198ed0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 --2- 172.21.15.37:0/1540217062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c94100440 0x7f6c94198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 -- 172.21.15.37:0/1540217062 >> 172.21.15.37:0/1540217062 conn(0x7f6c94077040 msgr2=0x7f6c940fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 -- 172.21.15.37:0/1540217062 shutdown_connections 2024-03-26T06:33:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:32.806+0000 7f6c99f5b640 1 -- 172.21.15.37:0/1540217062 wait complete. 2024-03-26T06:33:33.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:32 smithi066 ceph-mon[36504]: pgmap v328: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-26T06:33:33.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:32 smithi149 ceph-mon[36681]: pgmap v328: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-26T06:33:33.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:32 smithi037 ceph-mon[31974]: pgmap v328: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-26T06:33:33.524 INFO:tasks.workunit.client.0.smithi037.stdout: 1 408615 4.20 MB/sec execute 428 sec latency 236.513 ms 2024-03-26T06:33:34.524 INFO:tasks.workunit.client.0.smithi037.stdout: 1 408615 4.19 MB/sec execute 429 sec latency 1236.632 ms 2024-03-26T06:33:35.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:34 smithi066 ceph-mon[36504]: pgmap v329: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-03-26T06:33:35.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:34 smithi149 ceph-mon[36681]: pgmap v329: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-03-26T06:33:35.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:34 smithi037 ceph-mon[31974]: pgmap v329: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-03-26T06:33:35.524 INFO:tasks.workunit.client.0.smithi037.stdout: 1 408615 4.18 MB/sec execute 430 sec latency 2236.743 ms 2024-03-26T06:33:36.524 INFO:tasks.workunit.client.0.smithi037.stdout: 1 408671 4.17 MB/sec execute 431 sec latency 3032.372 ms 2024-03-26T06:33:37.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:36 smithi066 ceph-mon[36504]: pgmap v330: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-03-26T06:33:37.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:36 smithi149 ceph-mon[36681]: pgmap v330: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-03-26T06:33:37.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:36 smithi037 ceph-mon[31974]: pgmap v330: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-03-26T06:33:37.524 INFO:tasks.workunit.client.0.smithi037.stdout: 1 408671 4.16 MB/sec execute 432 sec latency 1164.466 ms 2024-03-26T06:33:38.524 INFO:tasks.workunit.client.0.smithi037.stdout: 1 408671 4.15 MB/sec execute 433 sec latency 2164.520 ms 2024-03-26T06:33:39.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:38 smithi066 ceph-mon[36504]: pgmap v331: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-03-26T06:33:39.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:38 smithi149 ceph-mon[36681]: pgmap v331: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-03-26T06:33:39.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:38 smithi037 ceph-mon[31974]: pgmap v331: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-03-26T06:33:39.524 INFO:tasks.workunit.client.0.smithi037.stdout: 1 408671 4.14 MB/sec execute 434 sec latency 3164.586 ms 2024-03-26T06:33:40.524 INFO:tasks.workunit.client.0.smithi037.stdout: 1 408671 4.13 MB/sec execute 435 sec latency 4164.680 ms 2024-03-26T06:33:41.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:40 smithi066 ceph-mon[36504]: pgmap v332: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-03-26T06:33:41.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:40 smithi149 ceph-mon[36681]: pgmap v332: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-03-26T06:33:41.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:40 smithi037 ceph-mon[31974]: pgmap v332: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-03-26T06:33:41.524 INFO:tasks.workunit.client.0.smithi037.stdout: 1 409022 4.13 MB/sec execute 436 sec latency 4954.976 ms 2024-03-26T06:33:42.524 INFO:tasks.workunit.client.0.smithi037.stdout: 1 411474 4.15 MB/sec execute 437 sec latency 17.527 ms 2024-03-26T06:33:43.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:42 smithi066 ceph-mon[36504]: pgmap v333: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-03-26T06:33:43.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:42 smithi149 ceph-mon[36681]: pgmap v333: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-03-26T06:33:43.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:42 smithi037 ceph-mon[31974]: pgmap v333: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-03-26T06:33:43.524 INFO:tasks.workunit.client.0.smithi037.stdout: 1 412158 4.15 MB/sec execute 438 sec latency 324.256 ms 2024-03-26T06:33:44.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 412158 4.14 MB/sec execute 439 sec latency 1324.330 ms 2024-03-26T06:33:45.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:44 smithi066 ceph-mon[36504]: pgmap v334: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 891 KiB/s wr, 90 op/s 2024-03-26T06:33:45.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:44 smithi149 ceph-mon[36681]: pgmap v334: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 891 KiB/s wr, 90 op/s 2024-03-26T06:33:45.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:44 smithi037 ceph-mon[31974]: pgmap v334: 97 pgs: 97 active+clean; 144 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 891 KiB/s wr, 90 op/s 2024-03-26T06:33:45.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 412158 4.13 MB/sec execute 440 sec latency 2324.473 ms 2024-03-26T06:33:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.904+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4203929026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a600736c0 msgr2=0x7f7a60073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.904+0000 7f7a64bf1640 1 --2- 172.21.15.37:0/4203929026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a600736c0 0x7f7a60073aa0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009900 tx=0x7f7a5002bcd0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.905+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4203929026 shutdown_connections 2024-03-26T06:33:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.905+0000 7f7a64bf1640 1 --2- 172.21.15.37:0/4203929026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a6010db40 0x7f7a6010ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.905+0000 7f7a64bf1640 1 --2- 172.21.15.37:0/4203929026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a60073fe0 0x7f7a6010d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.905+0000 7f7a64bf1640 1 --2- 172.21.15.37:0/4203929026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a600736c0 0x7f7a60073aa0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.905+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4203929026 >> 172.21.15.37:0/4203929026 conn(0x7f7a600fcdb0 msgr2=0x7f7a600ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.905+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4203929026 shutdown_connections 2024-03-26T06:33:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.905+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4203929026 wait complete. 2024-03-26T06:33:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.905+0000 7f7a64bf1640 1 Processor -- start 2024-03-26T06:33:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.905+0000 7f7a64bf1640 1 -- start start 2024-03-26T06:33:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a64bf1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a600736c0 0x7f7a6019fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a64bf1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a60073fe0 0x7f7a6019ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a64bf1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a6010db40 0x7f7a60199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a64bf1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a601138c0 con 0x7f7a600736c0 2024-03-26T06:33:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a64bf1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a60113740 con 0x7f7a6010db40 2024-03-26T06:33:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a64bf1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a60113a40 con 0x7f7a60073fe0 2024-03-26T06:33:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a600736c0 0x7f7a6019fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a600736c0 0x7f7a6019fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57684/0 (socket says 172.21.15.37:57684) 2024-03-26T06:33:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5e59c640 1 -- 172.21.15.37:0/4099473813 learned_addr learned my addr 172.21.15.37:0/4099473813 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5ed9d640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a6010db40 0x7f7a60199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5dd9b640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a60073fe0 0x7f7a6019ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5e59c640 1 -- 172.21.15.37:0/4099473813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a60073fe0 msgr2=0x7f7a6019ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a60073fe0 0x7f7a6019ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5e59c640 1 -- 172.21.15.37:0/4099473813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a6010db40 msgr2=0x7f7a60199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a6010db40 0x7f7a60199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5e59c640 1 -- 172.21.15.37:0/4099473813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6019a440 con 0x7f7a600736c0 2024-03-26T06:33:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5dd9b640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a60073fe0 0x7f7a6019ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:33:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5ed9d640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a6010db40 0x7f7a60199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:33:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a600736c0 0x7f7a6019fa70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005940 tx=0x7f7a5002cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.906+0000 7f7a4f7fe640 1 -- 172.21.15.37:0/4099473813 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50041070 con 0x7f7a600736c0 2024-03-26T06:33:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.907+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4099473813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6019a6d0 con 0x7f7a600736c0 2024-03-26T06:33:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.907+0000 7f7a4f7fe640 1 -- 172.21.15.37:0/4099473813 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a50004450 con 0x7f7a600736c0 2024-03-26T06:33:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.907+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4099473813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a601a6810 con 0x7f7a600736c0 2024-03-26T06:33:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.907+0000 7f7a4f7fe640 1 -- 172.21.15.37:0/4099473813 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5004b060 con 0x7f7a600736c0 2024-03-26T06:33:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.909+0000 7f7a4f7fe640 1 -- 172.21.15.37:0/4099473813 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f7a50044070 con 0x7f7a600736c0 2024-03-26T06:33:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a677d5740 1 Processor -- start 2024-03-26T06:33:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a677d5740 1 -- start start 2024-03-26T06:33:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a091d40 0x55fd3a090a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 0x55fd3a091450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a09c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a677d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd39ff55f0 con 0x55fd3a09a550 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a677d5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd39f9e930 con 0x55fd3a090fd0 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a677d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd39ff1690 con 0x55fd3a091d40 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a4f7fe640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7a480a2a10 0x7f7a480a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a5dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a09c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a5dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a09c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57690/0 (socket says 172.21.15.37:57690) 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a5dd9b640 1 -- 172.21.15.37:0/3870187563 learned_addr learned my addr 172.21.15.37:0/3870187563 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a5ed9d640 1 --2- 172.21.15.37:0/3870187563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a091d40 0x55fd3a090a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/3870187563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 0x55fd3a091450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.910+0000 7f7a5ed9d640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7a480a2a10 0x7f7a480a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a4f7fe640 1 -- 172.21.15.37:0/4099473813 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7a500f09e0 con 0x7f7a600736c0 2024-03-26T06:33:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a5dd9b640 1 -- 172.21.15.37:0/3870187563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a091d40 msgr2=0x55fd3a090a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a5dd9b640 1 --2- 172.21.15.37:0/3870187563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a091d40 0x55fd3a090a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a5dd9b640 1 -- 172.21.15.37:0/3870187563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 msgr2=0x55fd3a091450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a5dd9b640 1 --2- 172.21.15.37:0/3870187563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 0x55fd3a091450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a5dd9b640 1 -- 172.21.15.37:0/3870187563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd39d0a210 con 0x55fd3a09a550 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/3870187563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 0x55fd3a091450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a5dd9b640 1 --2- 172.21.15.37:0/3870187563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a09c940 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009c20 tx=0x7f7a5400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c1714a3ae6fac7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a5ed9d640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7a480a2a10 0x7f7a480a4ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a58005c40 tx=0x7f7a58009280 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a4cff9640 1 -- 172.21.15.37:0/3870187563 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54016070 con 0x55fd3a09a550 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a4cff9640 1 -- 172.21.15.37:0/3870187563 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a54011950 con 0x55fd3a09a550 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.911+0000 7f7a4cff9640 1 -- 172.21.15.37:0/3870187563 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54010910 con 0x55fd3a09a550 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.912+0000 7f7a677d5740 1 -- 172.21.15.37:0/3870187563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 msgr2=0x55fd3a09c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.912+0000 7f7a677d5740 1 --2- 172.21.15.37:0/3870187563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a09c940 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009c20 tx=0x7f7a5400e6c0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.912+0000 7f7a677d5740 1 -- 172.21.15.37:0/3870187563 shutdown_connections 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.912+0000 7f7a677d5740 1 --2- 172.21.15.37:0/3870187563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a09c940 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.912+0000 7f7a677d5740 1 --2- 172.21.15.37:0/3870187563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 0x55fd3a091450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.912+0000 7f7a677d5740 1 --2- 172.21.15.37:0/3870187563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a091d40 0x55fd3a090a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.912+0000 7f7a677d5740 1 -- 172.21.15.37:0/3870187563 >> 172.21.15.37:0/3870187563 conn(0x55fd3a0585b0 msgr2=0x55fd3a08bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.912+0000 7f7a677d5740 1 -- 172.21.15.37:0/3870187563 shutdown_connections 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.912+0000 7f7a677d5740 1 -- 172.21.15.37:0/3870187563 wait complete. 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.912+0000 7f7a677d5740 1 -- start start 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.912+0000 7f7a677d5740 1 Processor -- start 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a090fd0 0x55fd3a097520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a091d40 0x55fd3a097a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a097fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a677d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3a09d280 con 0x55fd3a09a550 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a677d5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd39fb1d90 con 0x55fd3a091d40 2024-03-26T06:33:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a677d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3a09d400 con 0x55fd3a090fd0 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a677d5740 10 client.0 _ll_register_callbacks cb 0x55fd39d09a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a097fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a097fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57700/0 (socket says 172.21.15.37:57700) 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5ed9d640 1 -- 172.21.15.37:0/395495437 learned_addr learned my addr 172.21.15.37:0/395495437 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5dd9b640 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a091d40 0x55fd3a097a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a090fd0 0x55fd3a097520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5ed9d640 1 -- 172.21.15.37:0/395495437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a090fd0 msgr2=0x55fd3a097520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5ed9d640 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a090fd0 0x55fd3a097520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5ed9d640 1 -- 172.21.15.37:0/395495437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a091d40 msgr2=0x55fd3a097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5ed9d640 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a091d40 0x55fd3a097a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5ed9d640 1 -- 172.21.15.37:0/395495437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd39dd6620 con 0x55fd3a09a550 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a090fd0 0x55fd3a097520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a5ed9d640 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a097fa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009db0 tx=0x7f7a58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca80e21b7cf8cea8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.913+0000 7f7a45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a45ffb640 1 -- 172.21.15.37:0/395495437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a58025070 con 0x55fd3a09a550 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a45ffb640 1 -- 172.21.15.37:0/395495437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7a5800fdc0 con 0x55fd3a09a550 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a677d5740 1 -- 172.21.15.37:0/395495437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a54011950 con 0x55fd3a09a550 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a45ffb640 1 -- 172.21.15.37:0/395495437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a58012030 con 0x55fd3a09a550 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a677d5740 10 client.16374 fetch_fsmap learned FSMap version 44 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a677d5740 1 -- 172.21.15.37:0/395495437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55fd3a0a9560 con 0x55fd3a09a550 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a45ffb640 1 -- 172.21.15.37:0/395495437 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a54011950 con 0x55fd3a09a550 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a45ffb640 1 -- 172.21.15.37:0/395495437 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7a58005620 con 0x55fd3a09a550 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a677d5740 10 client.16374 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a677d5740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a677d5740 20 client.16374 populate_metadata read hostname 'smithi037' 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a677d5740 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55fd3a0aa9d0 0x55fd3a0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a677d5740 4 client.16374 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:33:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a677d5740 1 -- 172.21.15.37:0/395495437 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd3a002b30 con 0x55fd3a0aa9d0 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.914+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55fd3a0aa9d0 0x55fd3a0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.915+0000 7f7a45ffb640 10 client.16374 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.915+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55fd3a0aa9d0 0x55fd3a0acdc0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870dc3a6951b17b4 server_cookie=e24ca563a2cd7386 in_seq=0 out_seq=0 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a45ffb640 1 -- 172.21.15.37:0/395495437 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd3a002b30 con 0x55fd3a0aa9d0 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a45ffb640 10 client.16374 handle_command_reply: tid=0 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 1 client.16374 shutdown 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 1 -- 172.21.15.37:0/395495437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 msgr2=0x55fd3a097fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a097fa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009db0 tx=0x7f7a58004a40 comp rx=0 tx=0).stop 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 1 -- 172.21.15.37:0/395495437 shutdown_connections 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55fd3a0aa9d0 0x55fd3a0acdc0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a097fa0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a091d40 0x55fd3a097a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 1 --2- 172.21.15.37:0/395495437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a090fd0 0x55fd3a097520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a45ffb640 0 client.16374 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 1 -- 172.21.15.37:0/395495437 >> 172.21.15.37:0/395495437 conn(0x55fd3a0585b0 msgr2=0x55fd3a08da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 1 -- 172.21.15.37:0/395495437 shutdown_connections 2024-03-26T06:33:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 1 -- 172.21.15.37:0/395495437 wait complete. 2024-03-26T06:33:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.916+0000 7f7a677d5740 20 client.16374 trim_cache size 0 max 16384 2024-03-26T06:33:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.929+0000 7f7a677d5740 1 Processor -- start 2024-03-26T06:33:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.929+0000 7f7a677d5740 1 -- start start 2024-03-26T06:33:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 0x55fd3a0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a091d40 0x55fd3a095e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a096350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a677d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a601aeef0 con 0x55fd3a09a550 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a677d5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a601b0870 con 0x55fd3a090fd0 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a677d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a601af550 con 0x55fd3a091d40 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a096350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a096350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57716/0 (socket says 172.21.15.37:57716) 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5ed9d640 1 -- 172.21.15.37:0/3684593239 learned_addr learned my addr 172.21.15.37:0/3684593239 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/3684593239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 0x55fd3a0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5ed9d640 1 -- 172.21.15.37:0/3684593239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a091d40 msgr2=0x55fd3a095e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5dd9b640 1 --2- 172.21.15.37:0/3684593239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a091d40 0x55fd3a095e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5ed9d640 1 --2- 172.21.15.37:0/3684593239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a091d40 0x55fd3a095e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5ed9d640 1 -- 172.21.15.37:0/3684593239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 msgr2=0x55fd3a0b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5ed9d640 1 --2- 172.21.15.37:0/3684593239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 0x55fd3a0b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5ed9d640 1 -- 172.21.15.37:0/3684593239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd3a0902a0 con 0x55fd3a09a550 2024-03-26T06:33:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/3684593239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 0x55fd3a0b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5dd9b640 1 --2- 172.21.15.37:0/3684593239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a091d40 0x55fd3a095e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a5ed9d640 1 --2- 172.21.15.37:0/3684593239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a096350 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7a580098c0 tx=0x7f7a58009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5a60023142d006 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a47fff640 1 -- 172.21.15.37:0/3684593239 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a58025040 con 0x55fd3a09a550 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a47fff640 1 -- 172.21.15.37:0/3684593239 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a58038070 con 0x55fd3a09a550 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.930+0000 7f7a47fff640 1 -- 172.21.15.37:0/3684593239 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a580121d0 con 0x55fd3a09a550 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.931+0000 7f7a677d5740 1 -- 172.21.15.37:0/3684593239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 msgr2=0x55fd3a096350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.931+0000 7f7a677d5740 1 --2- 172.21.15.37:0/3684593239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a096350 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7a580098c0 tx=0x7f7a58009df0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.931+0000 7f7a677d5740 1 -- 172.21.15.37:0/3684593239 shutdown_connections 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.931+0000 7f7a677d5740 1 --2- 172.21.15.37:0/3684593239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a09a550 0x55fd3a096350 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.931+0000 7f7a677d5740 1 --2- 172.21.15.37:0/3684593239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a091d40 0x55fd3a095e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.931+0000 7f7a677d5740 1 --2- 172.21.15.37:0/3684593239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a090fd0 0x55fd3a0b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.931+0000 7f7a677d5740 1 -- 172.21.15.37:0/3684593239 >> 172.21.15.37:0/3684593239 conn(0x55fd3a0585b0 msgr2=0x55fd3a0928b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.931+0000 7f7a677d5740 1 -- 172.21.15.37:0/3684593239 shutdown_connections 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.931+0000 7f7a677d5740 1 -- 172.21.15.37:0/3684593239 wait complete. 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.931+0000 7f7a677d5740 1 -- start start 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.931+0000 7f7a677d5740 1 Processor -- start 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a090fd0 0x55fd3a0a1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a091d40 0x55fd3a09fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a677d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a09a550 0x55fd3a0a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a677d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a601b0210 con 0x55fd3a090fd0 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a677d5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a601afbb0 con 0x55fd3a091d40 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a677d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a601ad1b0 con 0x55fd3a09a550 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a677d5740 10 client.0 _ll_register_callbacks cb 0x7f7a540093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a090fd0 0x55fd3a0a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a090fd0 0x55fd3a0a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57732/0 (socket says 172.21.15.37:57732) 2024-03-26T06:33:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a5e59c640 1 -- 172.21.15.37:0/1433335104 learned_addr learned my addr 172.21.15.37:0/1433335104 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a5e59c640 1 -- 172.21.15.37:0/1433335104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a09a550 msgr2=0x55fd3a0a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a09a550 0x55fd3a0a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a5e59c640 1 -- 172.21.15.37:0/1433335104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a091d40 msgr2=0x55fd3a09fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a5dd9b640 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a091d40 0x55fd3a09fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a091d40 0x55fd3a09fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a5e59c640 1 -- 172.21.15.37:0/1433335104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd3a0a0aa0 con 0x55fd3a090fd0 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a5e59c640 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a090fd0 0x55fd3a0a1990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7a500b95b0 tx=0x7f7a5002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46eddf16b9ec4f5b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a45ffb640 1 -- 172.21.15.37:0/1433335104 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500c12a0 con 0x55fd3a090fd0 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a45ffb640 1 -- 172.21.15.37:0/1433335104 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7a500ee3b0 con 0x55fd3a090fd0 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a677d5740 1 -- 172.21.15.37:0/1433335104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a58038070 con 0x55fd3a090fd0 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a45ffb640 1 -- 172.21.15.37:0/1433335104 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a500c58f0 con 0x55fd3a090fd0 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a677d5740 10 client.16380 fetch_fsmap learned FSMap version 44 2024-03-26T06:33:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.932+0000 7f7a677d5740 1 -- 172.21.15.37:0/1433335104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55fd3a101e00 con 0x55fd3a090fd0 2024-03-26T06:33:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.933+0000 7f7a45ffb640 1 -- 172.21.15.37:0/1433335104 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a58038070 con 0x55fd3a090fd0 2024-03-26T06:33:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.933+0000 7f7a45ffb640 1 -- 172.21.15.37:0/1433335104 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7a500c5460 con 0x55fd3a090fd0 2024-03-26T06:33:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.933+0000 7f7a677d5740 10 client.16380 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:33:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.933+0000 7f7a677d5740 10 client.16380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:33:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.933+0000 7f7a677d5740 20 client.16380 populate_metadata read hostname 'smithi037' 2024-03-26T06:33:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.933+0000 7f7a677d5740 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55fd3a1024d0 0x55fd3a1048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.933+0000 7f7a677d5740 4 client.16380 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:33:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.933+0000 7f7a677d5740 1 -- 172.21.15.37:0/1433335104 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd39ffce80 con 0x55fd3a1024d0 2024-03-26T06:33:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.933+0000 7f7a5dd9b640 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55fd3a1024d0 0x55fd3a1048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.934+0000 7f7a45ffb640 10 client.16380 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.934+0000 7f7a5dd9b640 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55fd3a1024d0 0x55fd3a1048f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4134cefe5f3118e7 server_cookie=9b27f9374e17db3d in_seq=0 out_seq=0 2024-03-26T06:33:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.965+0000 7f7a45ffb640 1 -- 172.21.15.37:0/1433335104 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd39ffce80 con 0x55fd3a1024d0 2024-03-26T06:33:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.965+0000 7f7a45ffb640 10 client.16380 handle_command_reply: tid=0 2024-03-26T06:33:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.965+0000 7f7a677d5740 1 client.16380 shutdown 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a677d5740 1 -- 172.21.15.37:0/1433335104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a090fd0 msgr2=0x55fd3a0a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a677d5740 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a090fd0 0x55fd3a0a1990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7a500b95b0 tx=0x7f7a5002bcd0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a677d5740 1 -- 172.21.15.37:0/1433335104 shutdown_connections 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a677d5740 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55fd3a1024d0 0x55fd3a1048f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a677d5740 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd3a09a550 0x55fd3a0a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a45ffb640 0 client.16380 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a677d5740 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd3a091d40 0x55fd3a09fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a677d5740 1 --2- 172.21.15.37:0/1433335104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd3a090fd0 0x55fd3a0a1990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a677d5740 1 -- 172.21.15.37:0/1433335104 >> 172.21.15.37:0/1433335104 conn(0x55fd3a0585b0 msgr2=0x55fd3a0b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a677d5740 1 -- 172.21.15.37:0/1433335104 shutdown_connections 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a677d5740 1 -- 172.21.15.37:0/1433335104 wait complete. 2024-03-26T06:33:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a677d5740 20 client.16380 trim_cache size 0 max 16384 2024-03-26T06:33:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:33:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:33:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:33:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:33:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4099473813 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7a480a2a10 msgr2=0x7f7a480a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a64bf1640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7a480a2a10 0x7f7a480a4ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a58005c40 tx=0x7f7a58009280 comp rx=0 tx=0).stop 2024-03-26T06:33:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4099473813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a600736c0 msgr2=0x7f7a6019fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a64bf1640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a600736c0 0x7f7a6019fa70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005940 tx=0x7f7a5002cd10 comp rx=0 tx=0).stop 2024-03-26T06:33:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4099473813 shutdown_connections 2024-03-26T06:33:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a64bf1640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f7a480a2a10 0x7f7a480a4ed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a64bf1640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a6010db40 0x7f7a60199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a64bf1640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a60073fe0 0x7f7a6019ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a64bf1640 1 --2- 172.21.15.37:0/4099473813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a600736c0 0x7f7a6019fa70 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.966+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4099473813 >> 172.21.15.37:0/4099473813 conn(0x7f7a600fcdb0 msgr2=0x7f7a600fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.967+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4099473813 shutdown_connections 2024-03-26T06:33:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:45.967+0000 7f7a64bf1640 1 -- 172.21.15.37:0/4099473813 wait complete. 2024-03-26T06:33:46.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 412214 4.12 MB/sec execute 441 sec latency 3117.395 ms 2024-03-26T06:33:47.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:46 smithi066 ceph-mon[36504]: pgmap v335: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-03-26T06:33:47.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:46 smithi149 ceph-mon[36681]: pgmap v335: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-03-26T06:33:47.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:46 smithi037 ceph-mon[31974]: pgmap v335: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-03-26T06:33:47.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 412214 4.11 MB/sec execute 442 sec latency 1170.697 ms 2024-03-26T06:33:48.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:47 smithi066 ceph-mon[36504]: pgmap v336: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 618 KiB/s wr, 69 op/s 2024-03-26T06:33:48.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:47 smithi149 ceph-mon[36681]: pgmap v336: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 618 KiB/s wr, 69 op/s 2024-03-26T06:33:48.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:47 smithi037 ceph-mon[31974]: pgmap v336: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 618 KiB/s wr, 69 op/s 2024-03-26T06:33:48.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 412214 4.10 MB/sec execute 443 sec latency 2170.770 ms 2024-03-26T06:33:49.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 412214 4.09 MB/sec execute 444 sec latency 3170.846 ms 2024-03-26T06:33:50.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:49 smithi066 ceph-mon[36504]: pgmap v337: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 619 KiB/s wr, 69 op/s 2024-03-26T06:33:50.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:49 smithi149 ceph-mon[36681]: pgmap v337: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 619 KiB/s wr, 69 op/s 2024-03-26T06:33:50.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:49 smithi037 ceph-mon[31974]: pgmap v337: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 619 KiB/s wr, 69 op/s 2024-03-26T06:33:50.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 412214 4.08 MB/sec execute 445 sec latency 4170.920 ms 2024-03-26T06:33:51.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 412563 4.08 MB/sec execute 446 sec latency 4957.300 ms 2024-03-26T06:33:52.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:51 smithi066 ceph-mon[36504]: pgmap v338: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-03-26T06:33:52.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:51 smithi149 ceph-mon[36681]: pgmap v338: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-03-26T06:33:52.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:51 smithi037 ceph-mon[31974]: pgmap v338: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-03-26T06:33:52.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 414702 4.09 MB/sec execute 447 sec latency 17.217 ms 2024-03-26T06:33:53.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 415763 4.10 MB/sec execute 448 sec latency 128.423 ms 2024-03-26T06:33:54.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:53 smithi066 ceph-mon[36504]: pgmap v339: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 876 KiB/s wr, 76 op/s 2024-03-26T06:33:54.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:53 smithi149 ceph-mon[36681]: pgmap v339: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 876 KiB/s wr, 76 op/s 2024-03-26T06:33:54.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:53 smithi037 ceph-mon[31974]: pgmap v339: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 876 KiB/s wr, 76 op/s 2024-03-26T06:33:54.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 415763 4.09 MB/sec execute 449 sec latency 1128.463 ms 2024-03-26T06:33:55.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 415763 4.08 MB/sec execute 450 sec latency 2128.568 ms 2024-03-26T06:33:56.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:55 smithi066 ceph-mon[36504]: pgmap v340: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-03-26T06:33:56.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:55 smithi149 ceph-mon[36681]: pgmap v340: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-03-26T06:33:56.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:55 smithi037 ceph-mon[31974]: pgmap v340: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-03-26T06:33:56.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 415831 4.07 MB/sec execute 451 sec latency 2919.610 ms 2024-03-26T06:33:57.525 INFO:tasks.workunit.client.0.smithi037.stdout: 1 415831 4.06 MB/sec execute 452 sec latency 1161.706 ms 2024-03-26T06:33:58.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:57 smithi066 ceph-mon[36504]: pgmap v341: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-03-26T06:33:58.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:57 smithi149 ceph-mon[36681]: pgmap v341: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-03-26T06:33:58.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:57 smithi037 ceph-mon[31974]: pgmap v341: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-03-26T06:33:58.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 415831 4.05 MB/sec execute 453 sec latency 2161.774 ms 2024-03-26T06:33:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.061+0000 7fbf124d1640 1 -- 172.21.15.37:0/449456672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0c1001b0 msgr2=0x7fbf0c10e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.061+0000 7fbf124d1640 1 --2- 172.21.15.37:0/449456672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0c1001b0 0x7fbf0c10e180 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009cd0 tx=0x7fbf0402c0f0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.061+0000 7fbf124d1640 1 -- 172.21.15.37:0/449456672 shutdown_connections 2024-03-26T06:33:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.062+0000 7fbf124d1640 1 --2- 172.21.15.37:0/449456672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0c1001b0 0x7fbf0c10e180 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.062+0000 7fbf124d1640 1 --2- 172.21.15.37:0/449456672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c0ff7f0 0x7fbf0c0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.062+0000 7fbf124d1640 1 --2- 172.21.15.37:0/449456672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0c1062d0 0x7fbf0c1066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.062+0000 7fbf124d1640 1 -- 172.21.15.37:0/449456672 >> 172.21.15.37:0/449456672 conn(0x7fbf0c0767b0 msgr2=0x7fbf0c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.062+0000 7fbf124d1640 1 -- 172.21.15.37:0/449456672 shutdown_connections 2024-03-26T06:33:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.062+0000 7fbf124d1640 1 -- 172.21.15.37:0/449456672 wait complete. 2024-03-26T06:33:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.062+0000 7fbf124d1640 1 Processor -- start 2024-03-26T06:33:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.062+0000 7fbf124d1640 1 -- start start 2024-03-26T06:33:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf124d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0c0ff7f0 0x7fbf0c19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf124d1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c1001b0 0x7fbf0c19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf124d1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0c1062d0 0x7fbf0c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf124d1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c111790 con 0x7fbf0c0ff7f0 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf124d1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c111610 con 0x7fbf0c1062d0 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf124d1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c111910 con 0x7fbf0c1001b0 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf0bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0c0ff7f0 0x7fbf0c19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf0bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0c0ff7f0 0x7fbf0c19d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49562/0 (socket says 172.21.15.37:49562) 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf0bfff640 1 -- 172.21.15.37:0/4135270625 learned_addr learned my addr 172.21.15.37:0/4135270625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf0b7fe640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c1001b0 0x7fbf0c19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf10a6e640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0c1062d0 0x7fbf0c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf0bfff640 1 -- 172.21.15.37:0/4135270625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c1001b0 msgr2=0x7fbf0c19dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c1001b0 0x7fbf0c19dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf0bfff640 1 -- 172.21.15.37:0/4135270625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0c1062d0 msgr2=0x7fbf0c1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0c1062d0 0x7fbf0c1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf0bfff640 1 -- 172.21.15.37:0/4135270625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c198170 con 0x7fbf0c0ff7f0 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf0b7fe640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c1001b0 0x7fbf0c19dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf10a6e640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0c1062d0 0x7fbf0c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0c0ff7f0 0x7fbf0c19d7f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00d950 tx=0x7fbefc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf097fa640 1 -- 172.21.15.37:0/4135270625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc017070 con 0x7fbf0c0ff7f0 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf124d1640 1 -- 172.21.15.37:0/4135270625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0c198460 con 0x7fbf0c0ff7f0 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.063+0000 7fbf097fa640 1 -- 172.21.15.37:0/4135270625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbefc009c30 con 0x7fbf0c0ff7f0 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.064+0000 7fbf124d1640 1 -- 172.21.15.37:0/4135270625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0c1a46b0 con 0x7fbf0c0ff7f0 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.064+0000 7fbf097fa640 1 -- 172.21.15.37:0/4135270625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc020060 con 0x7fbf0c0ff7f0 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf097fa640 1 -- 172.21.15.37:0/4135270625 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fbefc0109d0 con 0x7fbf0c0ff7f0 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf15190740 1 Processor -- start 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf15190740 1 -- start start 2024-03-26T06:33:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf15190740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca118a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf15190740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca118fd0 0x5587ca119450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf15190740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca122550 0x5587ca124940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf15190740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ca07d5f0 con 0x5587ca119d40 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf15190740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ca026930 con 0x5587ca122550 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf15190740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ca079690 con 0x5587ca118fd0 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf10a6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca118a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf097fa640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbef40a2a10 0x7fbef40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf10a6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca118a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49572/0 (socket says 172.21.15.37:49572) 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf10a6e640 1 -- 172.21.15.37:0/2197078234 learned_addr learned my addr 172.21.15.37:0/2197078234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/2197078234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca118fd0 0x5587ca119450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.066+0000 7fbf0b7fe640 1 --2- 172.21.15.37:0/2197078234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca122550 0x5587ca124940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.067+0000 7fbf10a6e640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbef40a2a10 0x7fbef40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.067+0000 7fbf097fa640 1 -- 172.21.15.37:0/4135270625 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbefc0cebe0 con 0x7fbf0c0ff7f0 2024-03-26T06:33:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.067+0000 7fbf10a6e640 1 -- 172.21.15.37:0/2197078234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca118fd0 msgr2=0x5587ca119450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.067+0000 7fbf10a6e640 1 --2- 172.21.15.37:0/2197078234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca118fd0 0x5587ca119450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.067+0000 7fbf10a6e640 1 -- 172.21.15.37:0/2197078234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca122550 msgr2=0x5587ca124940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.067+0000 7fbf10a6e640 1 --2- 172.21.15.37:0/2197078234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca122550 0x5587ca124940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.067+0000 7fbf10a6e640 1 -- 172.21.15.37:0/2197078234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587c9d92210 con 0x5587ca119d40 2024-03-26T06:33:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.067+0000 7fbf10a6e640 1 --2- 172.21.15.37:0/2197078234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca118a90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009430 tx=0x7fbf04032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77536ac1adbfe4b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.067+0000 7fbef27fc640 1 -- 172.21.15.37:0/2197078234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04032890 con 0x5587ca119d40 2024-03-26T06:33:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.067+0000 7fbef27fc640 1 -- 172.21.15.37:0/2197078234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf04032a30 con 0x5587ca119d40 2024-03-26T06:33:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.067+0000 7fbef27fc640 1 -- 172.21.15.37:0/2197078234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04034740 con 0x5587ca119d40 2024-03-26T06:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.068+0000 7fbf15190740 1 -- 172.21.15.37:0/2197078234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 msgr2=0x5587ca118a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.068+0000 7fbf10a6e640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbef40a2a10 0x7fbef40a4ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf0404c000 tx=0x7fbf04048000 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.068+0000 7fbf15190740 1 --2- 172.21.15.37:0/2197078234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca118a90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009430 tx=0x7fbf04032f40 comp rx=0 tx=0).stop 2024-03-26T06:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.068+0000 7fbf15190740 1 -- 172.21.15.37:0/2197078234 shutdown_connections 2024-03-26T06:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.068+0000 7fbf15190740 1 --2- 172.21.15.37:0/2197078234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca122550 0x5587ca124940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.068+0000 7fbf15190740 1 --2- 172.21.15.37:0/2197078234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca118fd0 0x5587ca119450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.068+0000 7fbf15190740 1 --2- 172.21.15.37:0/2197078234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca118a90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.068+0000 7fbf15190740 1 -- 172.21.15.37:0/2197078234 >> 172.21.15.37:0/2197078234 conn(0x5587ca0e05b0 msgr2=0x5587ca113cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.068+0000 7fbf15190740 1 -- 172.21.15.37:0/2197078234 shutdown_connections 2024-03-26T06:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.068+0000 7fbf15190740 1 -- 172.21.15.37:0/2197078234 wait complete. 2024-03-26T06:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.068+0000 7fbf15190740 1 -- start start 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf15190740 1 Processor -- start 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf15190740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca118fd0 0x5587ca11f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf15190740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca119d40 0x5587ca11faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf15190740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca122550 0x5587ca120030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf15190740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ca125280 con 0x5587ca122550 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf15190740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ca039d90 con 0x5587ca118fd0 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf15190740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ca125400 con 0x5587ca119d40 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf15190740 10 client.0 _ll_register_callbacks cb 0x5587c9d91a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf10a6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca122550 0x5587ca120030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf10a6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca122550 0x5587ca120030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49582/0 (socket says 172.21.15.37:49582) 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf10a6e640 1 -- 172.21.15.37:0/2957550612 learned_addr learned my addr 172.21.15.37:0/2957550612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca118fd0 0x5587ca11f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf0b7fe640 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca119d40 0x5587ca11faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf10a6e640 1 -- 172.21.15.37:0/2957550612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca119d40 msgr2=0x5587ca11faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf10a6e640 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca119d40 0x5587ca11faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf10a6e640 1 -- 172.21.15.37:0/2957550612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca118fd0 msgr2=0x5587ca11f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf10a6e640 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca118fd0 0x5587ca11f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf10a6e640 1 -- 172.21.15.37:0/2957550612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587c9e5e620 con 0x5587ca122550 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca118fd0 0x5587ca11f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbf10a6e640 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca122550 0x5587ca120030 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbf04034cb0 tx=0x7fbf04034ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad261b7e8d3d18a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.069+0000 7fbeeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/2957550612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04039040 con 0x5587ca122550 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbf15190740 1 -- 172.21.15.37:0/2957550612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587ca08ab30 con 0x5587ca122550 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/2957550612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbf04003e00 con 0x5587ca122550 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/2957550612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf04032890 con 0x5587ca122550 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbf15190740 10 client.16392 fetch_fsmap learned FSMap version 44 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbf15190740 1 -- 172.21.15.37:0/2957550612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5587ca131590 con 0x5587ca122550 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/2957550612 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5587ca08ab30 con 0x5587ca122550 2024-03-26T06:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/2957550612 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbf040435f0 con 0x5587ca122550 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbf15190740 10 client.16392 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbf15190740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbf15190740 20 client.16392 populate_metadata read hostname 'smithi037' 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbf15190740 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5587ca1328e0 0x5587ca134cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbf15190740 4 client.16392 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.070+0000 7fbf15190740 1 -- 172.21.15.37:0/2957550612 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587ca084e80 con 0x5587ca1328e0 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.071+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5587ca1328e0 0x5587ca134cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.071+0000 7fbeeb7fe640 10 client.16392 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.071+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5587ca1328e0 0x5587ca134cd0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a6943495fde6944 server_cookie=d06a3d159bfb0efc in_seq=0 out_seq=0 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/2957550612 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587ca084e80 con 0x5587ca1328e0 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbeeb7fe640 10 client.16392 handle_command_reply: tid=0 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 1 client.16392 shutdown 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 1 -- 172.21.15.37:0/2957550612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca122550 msgr2=0x5587ca120030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca122550 0x5587ca120030 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbf04034cb0 tx=0x7fbf04034ce0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 1 -- 172.21.15.37:0/2957550612 shutdown_connections 2024-03-26T06:33:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5587ca1328e0 0x5587ca134cd0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca122550 0x5587ca120030 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbeeb7fe640 0 client.16392 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca119d40 0x5587ca11faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 1 --2- 172.21.15.37:0/2957550612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca118fd0 0x5587ca11f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 1 -- 172.21.15.37:0/2957550612 >> 172.21.15.37:0/2957550612 conn(0x5587ca0e05b0 msgr2=0x5587ca115a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 1 -- 172.21.15.37:0/2957550612 shutdown_connections 2024-03-26T06:33:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 1 -- 172.21.15.37:0/2957550612 wait complete. 2024-03-26T06:33:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.072+0000 7fbf15190740 20 client.16392 trim_cache size 0 max 16384 2024-03-26T06:33:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.085+0000 7fbf15190740 1 Processor -- start 2024-03-26T06:33:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.085+0000 7fbf15190740 1 -- start start 2024-03-26T06:33:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.085+0000 7fbf15190740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca118fd0 0x5587ca121ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.085+0000 7fbf15190740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca11c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.085+0000 7fbf15190740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca122550 0x5587ca11cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.085+0000 7fbf15190740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c1acbd0 con 0x5587ca119d40 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.085+0000 7fbf15190740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c1ae550 con 0x5587ca122550 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.085+0000 7fbf15190740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c1ad230 con 0x5587ca118fd0 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf0b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca11c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf0b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca11c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49592/0 (socket says 172.21.15.37:49592) 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf0b7fe640 1 -- 172.21.15.37:0/1095008772 learned_addr learned my addr 172.21.15.37:0/1095008772 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf10a6e640 1 --2- 172.21.15.37:0/1095008772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca122550 0x5587ca11cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/1095008772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca118fd0 0x5587ca121ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf0b7fe640 1 -- 172.21.15.37:0/1095008772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca118fd0 msgr2=0x5587ca121ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf0b7fe640 1 --2- 172.21.15.37:0/1095008772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca118fd0 0x5587ca121ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf0b7fe640 1 -- 172.21.15.37:0/1095008772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca122550 msgr2=0x5587ca11cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf0b7fe640 1 --2- 172.21.15.37:0/1095008772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca122550 0x5587ca11cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf0b7fe640 1 -- 172.21.15.37:0/1095008772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587ca11b1a0 con 0x5587ca119d40 2024-03-26T06:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/1095008772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca118fd0 0x5587ca121ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbf0b7fe640 1 --2- 172.21.15.37:0/1095008772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca11c830 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c105b00 tx=0x7fbf000123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27559e2b2eb06513 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbef1ffb640 1 -- 172.21.15.37:0/1095008772 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00012d70 con 0x5587ca119d40 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbef1ffb640 1 -- 172.21.15.37:0/1095008772 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf0001a8c0 con 0x5587ca119d40 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.086+0000 7fbef1ffb640 1 -- 172.21.15.37:0/1095008772 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00010a60 con 0x5587ca119d40 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.087+0000 7fbf15190740 1 -- 172.21.15.37:0/1095008772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 msgr2=0x5587ca11c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.087+0000 7fbf15190740 1 --2- 172.21.15.37:0/1095008772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca11c830 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c105b00 tx=0x7fbf000123f0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.087+0000 7fbf15190740 1 -- 172.21.15.37:0/1095008772 shutdown_connections 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.087+0000 7fbf15190740 1 --2- 172.21.15.37:0/1095008772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca122550 0x5587ca11cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.087+0000 7fbf15190740 1 --2- 172.21.15.37:0/1095008772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca119d40 0x5587ca11c830 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.087+0000 7fbf15190740 1 --2- 172.21.15.37:0/1095008772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca118fd0 0x5587ca121ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.087+0000 7fbf15190740 1 -- 172.21.15.37:0/1095008772 >> 172.21.15.37:0/1095008772 conn(0x5587ca0e05b0 msgr2=0x5587ca11de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.087+0000 7fbf15190740 1 -- 172.21.15.37:0/1095008772 shutdown_connections 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.087+0000 7fbf15190740 1 -- 172.21.15.37:0/1095008772 wait complete. 2024-03-26T06:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.087+0000 7fbf15190740 1 -- start start 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.087+0000 7fbf15190740 1 Processor -- start 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf15190740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca118fd0 0x5587ca128720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf15190740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca119d40 0x5587ca128c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf15190740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca122550 0x5587ca188b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf15190740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c1adef0 con 0x5587ca118fd0 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf15190740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c1ad890 con 0x5587ca119d40 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf15190740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c1aae90 con 0x5587ca122550 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf15190740 10 client.0 _ll_register_callbacks cb 0x7fbf04037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf0bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca118fd0 0x5587ca128720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf0bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca118fd0 0x5587ca128720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49606/0 (socket says 172.21.15.37:49606) 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf0bfff640 1 -- 172.21.15.37:0/992646478 learned_addr learned my addr 172.21.15.37:0/992646478 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf0b7fe640 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca119d40 0x5587ca128c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf0bfff640 1 -- 172.21.15.37:0/992646478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca122550 msgr2=0x5587ca188b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca122550 0x5587ca188b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf0bfff640 1 -- 172.21.15.37:0/992646478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca119d40 msgr2=0x5587ca128c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca119d40 0x5587ca128c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf0bfff640 1 -- 172.21.15.37:0/992646478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587ca1890a0 con 0x5587ca118fd0 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf0b7fe640 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca119d40 0x5587ca128c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf0bfff640 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca118fd0 0x5587ca128720 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0cc390 tx=0x7fbefc06ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a8965c93b087e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbeeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/992646478 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc09b240 con 0x5587ca118fd0 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/992646478 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbefc0cc5b0 con 0x5587ca118fd0 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.088+0000 7fbf15190740 1 -- 172.21.15.37:0/992646478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf0001a8c0 con 0x5587ca118fd0 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/992646478 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbefc09f390 con 0x5587ca118fd0 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbf15190740 10 client.16398 fetch_fsmap learned FSMap version 44 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbf15190740 1 -- 172.21.15.37:0/992646478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5587ca18a260 con 0x5587ca118fd0 2024-03-26T06:33:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/992646478 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf0001a8c0 con 0x5587ca118fd0 2024-03-26T06:33:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/992646478 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbefc09a690 con 0x5587ca118fd0 2024-03-26T06:33:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbf15190740 10 client.16398 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:33:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbf15190740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:33:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbf15190740 20 client.16398 populate_metadata read hostname 'smithi037' 2024-03-26T06:33:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbf15190740 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5587ca18a940 0x5587ca18cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:33:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbf15190740 4 client.16398 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:33:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbf15190740 1 -- 172.21.15.37:0/992646478 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5587ca003710 con 0x5587ca18a940 2024-03-26T06:33:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.089+0000 7fbf0b7fe640 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5587ca18a940 0x5587ca18cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:33:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.090+0000 7fbf0b7fe640 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5587ca18a940 0x5587ca18cd70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd054b6c7923b5a7 server_cookie=8687d89a671e01b3 in_seq=0 out_seq=0 2024-03-26T06:33:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.090+0000 7fbeeb7fe640 10 client.16398 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/992646478 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5587ca003710 con 0x5587ca18a940 2024-03-26T06:33:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbeeb7fe640 10 client.16398 handle_command_reply: tid=0 2024-03-26T06:33:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbf15190740 1 client.16398 shutdown 2024-03-26T06:33:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbf15190740 1 -- 172.21.15.37:0/992646478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca118fd0 msgr2=0x5587ca128720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbf15190740 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca118fd0 0x5587ca128720 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0cc390 tx=0x7fbefc06ad10 comp rx=0 tx=0).stop 2024-03-26T06:33:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbf15190740 1 -- 172.21.15.37:0/992646478 shutdown_connections 2024-03-26T06:33:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbf15190740 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5587ca18a940 0x5587ca18cd70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbf15190740 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587ca122550 0x5587ca188b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbf15190740 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587ca119d40 0x5587ca128c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbeeb7fe640 0 client.16398 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:33:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbf15190740 1 --2- 172.21.15.37:0/992646478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587ca118fd0 0x5587ca128720 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbf15190740 1 -- 172.21.15.37:0/992646478 >> 172.21.15.37:0/992646478 conn(0x5587ca0e05b0 msgr2=0x5587ca115020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbf15190740 1 -- 172.21.15.37:0/992646478 shutdown_connections 2024-03-26T06:33:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.125+0000 7fbf15190740 1 -- 172.21.15.37:0/992646478 wait complete. 2024-03-26T06:33:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf15190740 20 client.16398 trim_cache size 0 max 16384 2024-03-26T06:33:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:33:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:33:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:33:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:33:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 -- 172.21.15.37:0/4135270625 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbef40a2a10 msgr2=0x7fbef40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbef40a2a10 0x7fbef40a4ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf0404c000 tx=0x7fbf04048000 comp rx=0 tx=0).stop 2024-03-26T06:33:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 -- 172.21.15.37:0/4135270625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0c0ff7f0 msgr2=0x7fbf0c19d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:33:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0c0ff7f0 0x7fbf0c19d7f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00d950 tx=0x7fbefc00de10 comp rx=0 tx=0).stop 2024-03-26T06:33:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 -- 172.21.15.37:0/4135270625 shutdown_connections 2024-03-26T06:33:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbef40a2a10 0x7fbef40a4ed0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0c1062d0 0x7fbf0c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c1001b0 0x7fbf0c19dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 --2- 172.21.15.37:0/4135270625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf0c0ff7f0 0x7fbf0c19d7f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:33:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 -- 172.21.15.37:0/4135270625 >> 172.21.15.37:0/4135270625 conn(0x7fbf0c0767b0 msgr2=0x7fbf0c1041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:33:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 -- 172.21.15.37:0/4135270625 shutdown_connections 2024-03-26T06:33:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:33:59.126+0000 7fbf124d1640 1 -- 172.21.15.37:0/4135270625 wait complete. 2024-03-26T06:33:59.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 415831 4.04 MB/sec execute 454 sec latency 3161.856 ms 2024-03-26T06:34:00.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:33:59 smithi066 ceph-mon[36504]: pgmap v342: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-03-26T06:34:00.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:33:59 smithi149 ceph-mon[36681]: pgmap v342: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-03-26T06:34:00.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:33:59 smithi037 ceph-mon[31974]: pgmap v342: 97 pgs: 97 active+clean; 151 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-03-26T06:34:00.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 415948 4.03 MB/sec execute 455 sec latency 3794.543 ms 2024-03-26T06:34:01.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 416058 4.03 MB/sec execute 456 sec latency 1087.310 ms 2024-03-26T06:34:02.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:01 smithi066 ceph-mon[36504]: pgmap v343: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-03-26T06:34:02.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:01 smithi149 ceph-mon[36681]: pgmap v343: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-03-26T06:34:02.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:01 smithi037 ceph-mon[31974]: pgmap v343: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-03-26T06:34:02.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 417636 4.03 MB/sec execute 457 sec latency 26.984 ms 2024-03-26T06:34:03.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 419110 4.05 MB/sec execute 458 sec latency 12.655 ms 2024-03-26T06:34:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:03 smithi066 ceph-mon[36504]: pgmap v344: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-03-26T06:34:04.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:03 smithi149 ceph-mon[36681]: pgmap v344: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-03-26T06:34:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:03 smithi037 ceph-mon[31974]: pgmap v344: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-03-26T06:34:04.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 419250 4.04 MB/sec execute 459 sec latency 890.124 ms 2024-03-26T06:34:05.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 419250 4.03 MB/sec execute 460 sec latency 1890.226 ms 2024-03-26T06:34:06.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:06 smithi066 ceph-mon[36504]: pgmap v345: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-03-26T06:34:06.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:06 smithi066 ceph-mon[36504]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T06:34:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:06 smithi149 ceph-mon[36681]: pgmap v345: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-03-26T06:34:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:06 smithi149 ceph-mon[36681]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T06:34:06.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:06 smithi037 ceph-mon[31974]: pgmap v345: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-03-26T06:34:06.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:06 smithi037 ceph-mon[31974]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T06:34:06.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 419306 4.02 MB/sec execute 461 sec latency 2683.534 ms 2024-03-26T06:34:07.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 419306 4.01 MB/sec execute 462 sec latency 1163.409 ms 2024-03-26T06:34:08.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:08 smithi066 ceph-mon[36504]: pgmap v347: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-03-26T06:34:08.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:08 smithi149 ceph-mon[36681]: pgmap v347: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-03-26T06:34:08.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:08 smithi037 ceph-mon[31974]: pgmap v347: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-03-26T06:34:08.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 419306 4.00 MB/sec execute 463 sec latency 2163.532 ms 2024-03-26T06:34:09.526 INFO:tasks.workunit.client.0.smithi037.stdout: 1 419306 4.00 MB/sec execute 464 sec latency 3163.648 ms 2024-03-26T06:34:10.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:10 smithi066 ceph-mon[36504]: pgmap v348: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-03-26T06:34:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:10 smithi149 ceph-mon[36681]: pgmap v348: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-03-26T06:34:10.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:10 smithi037 ceph-mon[31974]: pgmap v348: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-03-26T06:34:10.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 419306 3.99 MB/sec execute 465 sec latency 4163.758 ms 2024-03-26T06:34:11.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 419642 3.98 MB/sec execute 466 sec latency 4953.415 ms 2024-03-26T06:34:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.222+0000 7fe927bd1640 1 -- 172.21.15.37:0/3780911784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe920101e00 msgr2=0x7fe920102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.222+0000 7fe927bd1640 1 --2- 172.21.15.37:0/3780911784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe920101e00 0x7fe920102260 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe910009930 tx=0x7fe91002bc30 comp rx=0 tx=0).stop 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.222+0000 7fe927bd1640 1 -- 172.21.15.37:0/3780911784 shutdown_connections 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.222+0000 7fe927bd1640 1 --2- 172.21.15.37:0/3780911784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9201027a0 0x7fe92010d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.222+0000 7fe927bd1640 1 --2- 172.21.15.37:0/3780911784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe920101e00 0x7fe920102260 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.222+0000 7fe927bd1640 1 --2- 172.21.15.37:0/3780911784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe920100520 0x7fe920100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.222+0000 7fe927bd1640 1 -- 172.21.15.37:0/3780911784 >> 172.21.15.37:0/3780911784 conn(0x7fe920077310 msgr2=0x7fe920077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.222+0000 7fe927bd1640 1 -- 172.21.15.37:0/3780911784 shutdown_connections 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.222+0000 7fe927bd1640 1 -- 172.21.15.37:0/3780911784 wait complete. 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe927bd1640 1 Processor -- start 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe927bd1640 1 -- start start 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe927bd1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe920100520 0x7fe92019d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe927bd1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe920101e00 0x7fe92019de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe927bd1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9201027a0 0x7fe920197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe927bd1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9201106e0 con 0x7fe920100520 2024-03-26T06:34:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe927bd1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe920110560 con 0x7fe920101e00 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe927bd1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe920110860 con 0x7fe9201027a0 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe92596d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe920100520 0x7fe92019d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe92596d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe920100520 0x7fe92019d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48282/0 (socket says 172.21.15.37:48282) 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe92596d640 1 -- 172.21.15.37:0/1445372487 learned_addr learned my addr 172.21.15.37:0/1445372487 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe92516c640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe920101e00 0x7fe92019de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe92616e640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9201027a0 0x7fe920197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe92596d640 1 -- 172.21.15.37:0/1445372487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9201027a0 msgr2=0x7fe920197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe92596d640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9201027a0 0x7fe920197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe92596d640 1 -- 172.21.15.37:0/1445372487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe920101e00 msgr2=0x7fe92019de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe92596d640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe920101e00 0x7fe92019de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.223+0000 7fe92596d640 1 -- 172.21.15.37:0/1445372487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe920198270 con 0x7fe920100520 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.224+0000 7fe92596d640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe920100520 0x7fe92019d940 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00ea60 tx=0x7fe91c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.224+0000 7fe916ffd640 1 -- 172.21.15.37:0/1445372487 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00cde0 con 0x7fe920100520 2024-03-26T06:34:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.224+0000 7fe916ffd640 1 -- 172.21.15.37:0/1445372487 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe91c0035f0 con 0x7fe920100520 2024-03-26T06:34:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.224+0000 7fe927bd1640 1 -- 172.21.15.37:0/1445372487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe920198560 con 0x7fe920100520 2024-03-26T06:34:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.224+0000 7fe916ffd640 1 -- 172.21.15.37:0/1445372487 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c010670 con 0x7fe920100520 2024-03-26T06:34:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.224+0000 7fe927bd1640 1 -- 172.21.15.37:0/1445372487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9201a4800 con 0x7fe920100520 2024-03-26T06:34:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.226+0000 7fe916ffd640 1 -- 172.21.15.37:0/1445372487 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe91c0037a0 con 0x7fe920100520 2024-03-26T06:34:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92a7a8740 1 Processor -- start 2024-03-26T06:34:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92a7a8740 1 -- start start 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563488294d40 0x563488293a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488293fd0 0x563488294450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56348829d550 0x56348829f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634881f85f0 con 0x56348829d550 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634881a1930 con 0x563488293fd0 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634881f4690 con 0x563488294d40 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe916ffd640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe90c0a2a10 0x7fe90c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92596d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488293fd0 0x563488294450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92516c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56348829d550 0x56348829f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92616e640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe90c0a2a10 0x7fe90c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92516c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56348829d550 0x56348829f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48294/0 (socket says 172.21.15.37:48294) 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92516c640 1 -- 172.21.15.37:0/1732279730 learned_addr learned my addr 172.21.15.37:0/1732279730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92616e640 1 --2- 172.21.15.37:0/1732279730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563488294d40 0x563488293a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe916ffd640 1 -- 172.21.15.37:0/1445372487 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fe91c0cf0e0 con 0x7fe920100520 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92516c640 1 -- 172.21.15.37:0/1732279730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563488294d40 msgr2=0x563488293a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92516c640 1 --2- 172.21.15.37:0/1732279730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563488294d40 0x563488293a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92516c640 1 -- 172.21.15.37:0/1732279730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488293fd0 msgr2=0x563488294450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.227+0000 7fe92516c640 1 --2- 172.21.15.37:0/1732279730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488293fd0 0x563488294450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.228+0000 7fe92516c640 1 -- 172.21.15.37:0/1732279730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563487f0d210 con 0x56348829d550 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.228+0000 7fe92616e640 1 --2- 172.21.15.37:0/1732279730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563488294d40 0x563488293a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.228+0000 7fe92516c640 1 --2- 172.21.15.37:0/1732279730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56348829d550 0x56348829f940 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe910000c00 tx=0x7fe910009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873a8f6d0e5492c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.228+0000 7fe92616e640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe90c0a2a10 0x7fe90c0a4ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe918005fd0 tx=0x7fe918009b60 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.228+0000 7fe90bfff640 1 -- 172.21.15.37:0/1732279730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910045070 con 0x56348829d550 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.228+0000 7fe90bfff640 1 -- 172.21.15.37:0/1732279730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe91002c650 con 0x56348829d550 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.228+0000 7fe90bfff640 1 -- 172.21.15.37:0/1732279730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910004aa0 con 0x56348829d550 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.229+0000 7fe92a7a8740 1 -- 172.21.15.37:0/1732279730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56348829d550 msgr2=0x56348829f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.229+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/1732279730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56348829d550 0x56348829f940 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe910000c00 tx=0x7fe910009560 comp rx=0 tx=0).stop 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.229+0000 7fe92a7a8740 1 -- 172.21.15.37:0/1732279730 shutdown_connections 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.229+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/1732279730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56348829d550 0x56348829f940 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.229+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/1732279730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488293fd0 0x563488294450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.229+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/1732279730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563488294d40 0x563488293a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.229+0000 7fe92a7a8740 1 -- 172.21.15.37:0/1732279730 >> 172.21.15.37:0/1732279730 conn(0x56348825b5b0 msgr2=0x56348828ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.229+0000 7fe92a7a8740 1 -- 172.21.15.37:0/1732279730 shutdown_connections 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.229+0000 7fe92a7a8740 1 -- 172.21.15.37:0/1732279730 wait complete. 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.229+0000 7fe92a7a8740 1 -- start start 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.229+0000 7fe92a7a8740 1 Processor -- start 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x56348829a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488294d40 0x56348829aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x56348829af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634882a0220 con 0x563488293fd0 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634881b4d90 con 0x563488294d40 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634882a03a0 con 0x56348829d550 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92a7a8740 10 client.0 _ll_register_callbacks cb 0x563487f0ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92596d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x56348829a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92516c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488294d40 0x56348829aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92596d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x56348829a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48298/0 (socket says 172.21.15.37:48298) 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92596d640 1 -- 172.21.15.37:0/396200712 learned_addr learned my addr 172.21.15.37:0/396200712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92616e640 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x56348829af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92516c640 1 -- 172.21.15.37:0/396200712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 msgr2=0x56348829af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92516c640 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x56348829af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92516c640 1 -- 172.21.15.37:0/396200712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 msgr2=0x56348829a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92516c640 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x56348829a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92516c640 1 -- 172.21.15.37:0/396200712 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563487fd9620 con 0x563488294d40 2024-03-26T06:34:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92596d640 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x56348829a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92616e640 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x56348829af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe92516c640 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488294d40 0x56348829aa40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe910036040 tx=0x7fe91002cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc24ea1fe67d6f24 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.230+0000 7fe9097fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe9097fa640 1 -- 172.21.15.37:0/396200712 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910045040 con 0x563488294d40 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe92a7a8740 1 -- 172.21.15.37:0/396200712 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe91002c650 con 0x563488294d40 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe9097fa640 1 -- 172.21.15.37:0/396200712 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fe9100052f0 con 0x563488294d40 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe9097fa640 1 -- 172.21.15.37:0/396200712 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe910004c80 con 0x563488294d40 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe92a7a8740 10 client.25934 fetch_fsmap learned FSMap version 44 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe92a7a8740 1 -- 172.21.15.37:0/396200712 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5634882ac480 con 0x563488294d40 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe9097fa640 1 -- 172.21.15.37:0/396200712 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe91002c650 con 0x563488294d40 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe9097fa640 1 -- 172.21.15.37:0/396200712 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe91003f440 con 0x563488294d40 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe92a7a8740 10 client.25934 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe92a7a8740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:34:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe92a7a8740 20 client.25934 populate_metadata read hostname 'smithi037' 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5634882ad7d0 0x5634882afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe92a7a8740 4 client.25934 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe92a7a8740 1 -- 172.21.15.37:0/396200712 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563488205b30 con 0x5634882ad7d0 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.231+0000 7fe92596d640 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5634882ad7d0 0x5634882afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.232+0000 7fe92596d640 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5634882ad7d0 0x5634882afbc0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a3073835d31051 server_cookie=cd49b4410e8ec0da in_seq=0 out_seq=0 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.232+0000 7fe9097fa640 10 client.25934 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.232+0000 7fe9097fa640 1 -- 172.21.15.37:0/396200712 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563488205b30 con 0x5634882ad7d0 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.232+0000 7fe9097fa640 10 client.25934 handle_command_reply: tid=0 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 1 client.25934 shutdown 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 1 -- 172.21.15.37:0/396200712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488294d40 msgr2=0x56348829aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488294d40 0x56348829aa40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe910036040 tx=0x7fe91002cb30 comp rx=0 tx=0).stop 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 1 -- 172.21.15.37:0/396200712 shutdown_connections 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5634882ad7d0 0x5634882afbc0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x56348829af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488294d40 0x56348829aa40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/396200712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x56348829a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe9097fa640 0 client.25934 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 1 -- 172.21.15.37:0/396200712 >> 172.21.15.37:0/396200712 conn(0x56348825b5b0 msgr2=0x563488290a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 1 -- 172.21.15.37:0/396200712 shutdown_connections 2024-03-26T06:34:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 1 -- 172.21.15.37:0/396200712 wait complete. 2024-03-26T06:34:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.233+0000 7fe92a7a8740 20 client.25934 trim_cache size 0 max 16384 2024-03-26T06:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92a7a8740 1 Processor -- start 2024-03-26T06:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92a7a8740 1 -- start start 2024-03-26T06:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488293fd0 0x56348829cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488294d40 0x5634882a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x5634882a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9201accc0 con 0x563488294d40 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9201ae640 con 0x563488293fd0 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9201ad320 con 0x56348829d550 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92516c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488294d40 0x5634882a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92516c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488294d40 0x5634882a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48304/0 (socket says 172.21.15.37:48304) 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92516c640 1 -- 172.21.15.37:0/558790819 learned_addr learned my addr 172.21.15.37:0/558790819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92616e640 1 --2- 172.21.15.37:0/558790819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x5634882a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.248+0000 7fe92596d640 1 --2- 172.21.15.37:0/558790819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488293fd0 0x56348829cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.249+0000 7fe92516c640 1 -- 172.21.15.37:0/558790819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 msgr2=0x5634882a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.249+0000 7fe92516c640 1 --2- 172.21.15.37:0/558790819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x5634882a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.249+0000 7fe92516c640 1 -- 172.21.15.37:0/558790819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488293fd0 msgr2=0x56348829cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.249+0000 7fe92516c640 1 --2- 172.21.15.37:0/558790819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488293fd0 0x56348829cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.249+0000 7fe92516c640 1 -- 172.21.15.37:0/558790819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634882932a0 con 0x563488294d40 2024-03-26T06:34:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.249+0000 7fe92616e640 1 --2- 172.21.15.37:0/558790819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x5634882a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.249+0000 7fe92516c640 1 --2- 172.21.15.37:0/558790819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488294d40 0x5634882a4610 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe91002c170 tx=0x7fe9100047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cb70c45f8597ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.249+0000 7fe90b7fe640 1 -- 172.21.15.37:0/558790819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91003f440 con 0x563488294d40 2024-03-26T06:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.249+0000 7fe90b7fe640 1 -- 172.21.15.37:0/558790819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe91002cba0 con 0x563488294d40 2024-03-26T06:34:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.249+0000 7fe90b7fe640 1 -- 172.21.15.37:0/558790819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910005350 con 0x563488294d40 2024-03-26T06:34:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.250+0000 7fe92a7a8740 1 -- 172.21.15.37:0/558790819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488294d40 msgr2=0x5634882a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.250+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/558790819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488294d40 0x5634882a4610 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe91002c170 tx=0x7fe9100047d0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.250+0000 7fe92a7a8740 1 -- 172.21.15.37:0/558790819 shutdown_connections 2024-03-26T06:34:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.250+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/558790819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x5634882a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.250+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/558790819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488294d40 0x5634882a4610 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.250+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/558790819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488293fd0 0x56348829cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.250+0000 7fe92a7a8740 1 -- 172.21.15.37:0/558790819 >> 172.21.15.37:0/558790819 conn(0x56348825b5b0 msgr2=0x5634882b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.250+0000 7fe92a7a8740 1 -- 172.21.15.37:0/558790819 shutdown_connections 2024-03-26T06:34:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.250+0000 7fe92a7a8740 1 -- 172.21.15.37:0/558790819 wait complete. 2024-03-26T06:34:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.250+0000 7fe92a7a8740 1 -- start start 2024-03-26T06:34:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.250+0000 7fe92a7a8740 1 Processor -- start 2024-03-26T06:34:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x5634882a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488294d40 0x5634882a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92a7a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x5634882ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9201adfe0 con 0x563488293fd0 2024-03-26T06:34:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9201ad980 con 0x563488294d40 2024-03-26T06:34:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92a7a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9201aaf80 con 0x56348829d550 2024-03-26T06:34:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92a7a8740 10 client.0 _ll_register_callbacks cb 0x7fe9180093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:34:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92596d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x5634882a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92596d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x5634882a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48306/0 (socket says 172.21.15.37:48306) 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92596d640 1 -- 172.21.15.37:0/3212725015 learned_addr learned my addr 172.21.15.37:0/3212725015 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92516c640 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488294d40 0x5634882a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92616e640 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x5634882ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92596d640 1 -- 172.21.15.37:0/3212725015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 msgr2=0x5634882ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92596d640 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x5634882ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92596d640 1 -- 172.21.15.37:0/3212725015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488294d40 msgr2=0x5634882a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92596d640 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488294d40 0x5634882a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92596d640 1 -- 172.21.15.37:0/3212725015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563488300580 con 0x563488293fd0 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92596d640 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x5634882a0be0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe91c09b4c0 tx=0x7fe91c06b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5022ff9d14813d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe9097fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe9097fa640 1 -- 172.21.15.37:0/3212725015 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c09f910 con 0x563488293fd0 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe9097fa640 1 -- 172.21.15.37:0/3212725015 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fe91c003b30 con 0x563488293fd0 2024-03-26T06:34:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.251+0000 7fe92a7a8740 1 -- 172.21.15.37:0/3212725015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe91002cba0 con 0x563488293fd0 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe9097fa640 1 -- 172.21.15.37:0/3212725015 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe91c0a3f60 con 0x563488293fd0 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe92a7a8740 10 client.16416 fetch_fsmap learned FSMap version 44 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe92a7a8740 1 -- 172.21.15.37:0/3212725015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x563488308950 con 0x563488293fd0 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe9097fa640 1 -- 172.21.15.37:0/3212725015 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe91002cba0 con 0x563488293fd0 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe9097fa640 1 -- 172.21.15.37:0/3212725015 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe91c09ac10 con 0x563488293fd0 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe92a7a8740 10 client.16416 fetch_fsmap finished waiting for FSMap version 44 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe92a7a8740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe92a7a8740 20 client.16416 populate_metadata read hostname 'smithi037' 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563488308f80 0x56348830b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe92a7a8740 4 client.16416 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe92a7a8740 1 -- 172.21.15.37:0/3212725015 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5634881ffe80 con 0x563488308f80 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe92516c640 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563488308f80 0x56348830b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe9097fa640 10 client.16416 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.252+0000 7fe92516c640 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563488308f80 0x56348830b370 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d1796fbdb16f51d server_cookie=81c0b31e1d5e714f in_seq=0 out_seq=0 2024-03-26T06:34:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.280+0000 7fe9097fa640 1 -- 172.21.15.37:0/3212725015 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5634881ffe80 con 0x563488308f80 2024-03-26T06:34:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.280+0000 7fe9097fa640 10 client.16416 handle_command_reply: tid=0 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 1 client.16416 shutdown 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 1 -- 172.21.15.37:0/3212725015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 msgr2=0x5634882a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x5634882a0be0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe91c09b4c0 tx=0x7fe91c06b290 comp rx=0 tx=0).stop 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 1 -- 172.21.15.37:0/3212725015 shutdown_connections 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x563488308f80 0x56348830b370 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348829d550 0x5634882ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe9097fa640 0 client.16416 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563488294d40 0x5634882a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 1 --2- 172.21.15.37:0/3212725015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563488293fd0 0x5634882a0be0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 1 -- 172.21.15.37:0/3212725015 >> 172.21.15.37:0/3212725015 conn(0x56348825b5b0 msgr2=0x563488298d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 1 -- 172.21.15.37:0/3212725015 shutdown_connections 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 1 -- 172.21.15.37:0/3212725015 wait complete. 2024-03-26T06:34:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe92a7a8740 20 client.16416 trim_cache size 0 max 16384 2024-03-26T06:34:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:34:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:34:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:34:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:34:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe927bd1640 1 -- 172.21.15.37:0/1445372487 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe90c0a2a10 msgr2=0x7fe90c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe927bd1640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe90c0a2a10 0x7fe90c0a4ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe918005fd0 tx=0x7fe918009b60 comp rx=0 tx=0).stop 2024-03-26T06:34:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe927bd1640 1 -- 172.21.15.37:0/1445372487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe920100520 msgr2=0x7fe92019d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe927bd1640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe920100520 0x7fe92019d940 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00ea60 tx=0x7fe91c00ef20 comp rx=0 tx=0).stop 2024-03-26T06:34:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe927bd1640 1 -- 172.21.15.37:0/1445372487 shutdown_connections 2024-03-26T06:34:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe927bd1640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe90c0a2a10 0x7fe90c0a4ed0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe927bd1640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9201027a0 0x7fe920197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.281+0000 7fe927bd1640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe920101e00 0x7fe92019de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.282+0000 7fe927bd1640 1 --2- 172.21.15.37:0/1445372487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe920100520 0x7fe92019d940 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.282+0000 7fe927bd1640 1 -- 172.21.15.37:0/1445372487 >> 172.21.15.37:0/1445372487 conn(0x7fe920077310 msgr2=0x7fe9200ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.282+0000 7fe927bd1640 1 -- 172.21.15.37:0/1445372487 shutdown_connections 2024-03-26T06:34:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:12.282+0000 7fe927bd1640 1 -- 172.21.15.37:0/1445372487 wait complete. 2024-03-26T06:34:12.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:12 smithi066 ceph-mon[36504]: pgmap v349: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 928 KiB/s wr, 130 op/s 2024-03-26T06:34:12.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:12 smithi149 ceph-mon[36681]: pgmap v349: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 928 KiB/s wr, 130 op/s 2024-03-26T06:34:12.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:12 smithi037 ceph-mon[31974]: pgmap v349: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 928 KiB/s wr, 130 op/s 2024-03-26T06:34:12.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 421926 4.00 MB/sec execute 467 sec latency 15.542 ms 2024-03-26T06:34:12.911 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:34:12 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: ignoring --setuser ceph since I am not root 2024-03-26T06:34:12.911 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:34:12 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: ignoring --setgroup ceph since I am not root 2024-03-26T06:34:12.911 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:34:12 smithi149 ceph-mds[67761]: -- 172.21.15.149:0/2154870198 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f1473aba40 con 0x55f14737e800 2024-03-26T06:34:12.911 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:34:12 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: starting mds.c at 2024-03-26T06:34:13.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:13 smithi066 ceph-mon[36504]: daemon mds.c restarted 2024-03-26T06:34:13.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:13 smithi149 ceph-mon[36681]: daemon mds.c restarted 2024-03-26T06:34:13.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:13 smithi037 ceph-mon[31974]: daemon mds.c restarted 2024-03-26T06:34:13.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 423111 4.00 MB/sec execute 468 sec latency 16.003 ms 2024-03-26T06:34:14.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:14 smithi066 ceph-mon[36504]: pgmap v350: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 928 KiB/s wr, 130 op/s 2024-03-26T06:34:14.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:14 smithi066 ceph-mon[36504]: osdmap e99: 12 total, 12 up, 12 in 2024-03-26T06:34:14.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:14 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6836/1657798084,v1:172.21.15.149:6837/1657798084] up:boot 2024-03-26T06:34:14.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:14 smithi066 ceph-mon[36504]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:34:14.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:14 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:34:14.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:14 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:34:14.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:14 smithi149 ceph-mon[36681]: pgmap v350: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 928 KiB/s wr, 130 op/s 2024-03-26T06:34:14.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:14 smithi149 ceph-mon[36681]: osdmap e99: 12 total, 12 up, 12 in 2024-03-26T06:34:14.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:14 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6836/1657798084,v1:172.21.15.149:6837/1657798084] up:boot 2024-03-26T06:34:14.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:14 smithi149 ceph-mon[36681]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:34:14.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:14 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:34:14.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:14 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:34:14.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:14 smithi037 ceph-mon[31974]: pgmap v350: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 928 KiB/s wr, 130 op/s 2024-03-26T06:34:14.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:14 smithi037 ceph-mon[31974]: osdmap e99: 12 total, 12 up, 12 in 2024-03-26T06:34:14.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:14 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6836/1657798084,v1:172.21.15.149:6837/1657798084] up:boot 2024-03-26T06:34:14.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:14 smithi037 ceph-mon[31974]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:34:14.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:14 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:34:14.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:14 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:34:14.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 424562 4.01 MB/sec execute 469 sec latency 21.473 ms 2024-03-26T06:34:15.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 426104 4.03 MB/sec execute 470 sec latency 19.761 ms 2024-03-26T06:34:16.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:15 smithi066 ceph-mon[36504]: osdmap e100: 12 total, 12 up, 12 in 2024-03-26T06:34:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:15 smithi149 ceph-mon[36681]: osdmap e100: 12 total, 12 up, 12 in 2024-03-26T06:34:16.202 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:15 smithi037 ceph-mon[31974]: osdmap e100: 12 total, 12 up, 12 in 2024-03-26T06:34:16.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 426746 4.02 MB/sec execute 471 sec latency 497.409 ms 2024-03-26T06:34:17.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:16 smithi066 ceph-mon[36504]: pgmap v353: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 919 KiB/s wr, 135 op/s 2024-03-26T06:34:17.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:16 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:34:17.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:16 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:34:17.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:16 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:34:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:16 smithi149 ceph-mon[36681]: pgmap v353: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 919 KiB/s wr, 135 op/s 2024-03-26T06:34:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:16 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:34:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:16 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:34:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:16 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:34:17.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:16 smithi037 ceph-mon[31974]: pgmap v353: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 919 KiB/s wr, 135 op/s 2024-03-26T06:34:17.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:16 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:34:17.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:16 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:34:17.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:16 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:34:17.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 429013 4.04 MB/sec execute 472 sec latency 44.263 ms 2024-03-26T06:34:18.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:17 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:34:18.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:17 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:34:18.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:17 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:34:18.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:17 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:34:18.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:17 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:34:18.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:17 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:34:18.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:17 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:34:18.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:17 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:34:18.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:17 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:34:18.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 430013 4.04 MB/sec execute 473 sec latency 132.955 ms 2024-03-26T06:34:19.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:18 smithi066 ceph-mon[36504]: pgmap v354: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 919 KiB/s wr, 135 op/s 2024-03-26T06:34:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:18 smithi149 ceph-mon[36681]: pgmap v354: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 919 KiB/s wr, 135 op/s 2024-03-26T06:34:19.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:18 smithi037 ceph-mon[31974]: pgmap v354: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 919 KiB/s wr, 135 op/s 2024-03-26T06:34:19.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 430013 4.03 MB/sec execute 474 sec latency 1133.049 ms 2024-03-26T06:34:20.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 430013 4.02 MB/sec execute 475 sec latency 2133.103 ms 2024-03-26T06:34:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:20 smithi066 ceph-mon[36504]: pgmap v355: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 820 KiB/s wr, 108 op/s 2024-03-26T06:34:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:20 smithi149 ceph-mon[36681]: pgmap v355: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 820 KiB/s wr, 108 op/s 2024-03-26T06:34:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:20 smithi037 ceph-mon[31974]: pgmap v355: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 820 KiB/s wr, 108 op/s 2024-03-26T06:34:21.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 430377 4.02 MB/sec execute 476 sec latency 2920.908 ms 2024-03-26T06:34:22.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 432749 4.04 MB/sec execute 477 sec latency 19.324 ms 2024-03-26T06:34:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:22 smithi066 ceph-mon[36504]: pgmap v356: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-03-26T06:34:23.160 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:22 smithi149 ceph-mon[36681]: pgmap v356: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-03-26T06:34:23.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:22 smithi037 ceph-mon[31974]: pgmap v356: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-03-26T06:34:23.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 433556 4.04 MB/sec execute 478 sec latency 207.281 ms 2024-03-26T06:34:24.527 INFO:tasks.workunit.client.0.smithi037.stdout: 1 433556 4.03 MB/sec execute 479 sec latency 1207.421 ms 2024-03-26T06:34:25.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:24 smithi066 ceph-mon[36504]: pgmap v357: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-03-26T06:34:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:24 smithi149 ceph-mon[36681]: pgmap v357: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-03-26T06:34:25.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:24 smithi037 ceph-mon[31974]: pgmap v357: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-03-26T06:34:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.373+0000 7f896c91b640 1 -- 172.21.15.37:0/2045844585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8968100440 msgr2=0x7f89681008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.373+0000 7f896c91b640 1 --2- 172.21.15.37:0/2045844585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8968100440 0x7f89681008c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f895c009900 tx=0x7f895c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:34:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.373+0000 7f896c91b640 1 -- 172.21.15.37:0/2045844585 shutdown_connections 2024-03-26T06:34:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.373+0000 7f896c91b640 1 --2- 172.21.15.37:0/2045844585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8968100e00 0x7f896810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.373+0000 7f896c91b640 1 --2- 172.21.15.37:0/2045844585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8968100440 0x7f89681008c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.373+0000 7f896c91b640 1 --2- 172.21.15.37:0/2045844585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8968105190 0x7f8968105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.373+0000 7f896c91b640 1 -- 172.21.15.37:0/2045844585 >> 172.21.15.37:0/2045844585 conn(0x7f8968077040 msgr2=0x7f8968077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.373+0000 7f896c91b640 1 -- 172.21.15.37:0/2045844585 shutdown_connections 2024-03-26T06:34:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.373+0000 7f896c91b640 1 -- 172.21.15.37:0/2045844585 wait complete. 2024-03-26T06:34:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f896c91b640 1 Processor -- start 2024-03-26T06:34:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f896c91b640 1 -- start start 2024-03-26T06:34:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f896c91b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8968100440 0x7f896819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f896c91b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8968100e00 0x7f896819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f896c91b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8968105190 0x7f8968196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f896c91b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8968110650 con 0x7f8968105190 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f896c91b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681104d0 con 0x7f8968100440 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f896c91b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681107d0 con 0x7f8968100e00 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f8966d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8968105190 0x7f8968196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f896659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8968100440 0x7f896819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f8966d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8968105190 0x7f8968196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52098/0 (socket says 172.21.15.37:52098) 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f8966d9d640 1 -- 172.21.15.37:0/3313724198 learned_addr learned my addr 172.21.15.37:0/3313724198 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.374+0000 7f8965d9b640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8968100e00 0x7f896819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.375+0000 7f8966d9d640 1 -- 172.21.15.37:0/3313724198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8968100e00 msgr2=0x7f896819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.375+0000 7f8966d9d640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8968100e00 0x7f896819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.375+0000 7f8966d9d640 1 -- 172.21.15.37:0/3313724198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8968100440 msgr2=0x7f896819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.375+0000 7f8966d9d640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8968100440 0x7f896819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.375+0000 7f8966d9d640 1 -- 172.21.15.37:0/3313724198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8968197040 con 0x7f8968105190 2024-03-26T06:34:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.375+0000 7f8966d9d640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8968105190 0x7f8968196870 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8960007cb0 tx=0x7f896000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.375+0000 7f89577fe640 1 -- 172.21.15.37:0/3313724198 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8960007e60 con 0x7f8968105190 2024-03-26T06:34:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.375+0000 7f896c91b640 1 -- 172.21.15.37:0/3313724198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8968197330 con 0x7f8968105190 2024-03-26T06:34:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.375+0000 7f89577fe640 1 -- 172.21.15.37:0/3313724198 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f896000eb30 con 0x7f8968105190 2024-03-26T06:34:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.375+0000 7f89577fe640 1 -- 172.21.15.37:0/3313724198 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89600153f0 con 0x7f8968105190 2024-03-26T06:34:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.375+0000 7f896c91b640 1 -- 172.21.15.37:0/3313724198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8968068970 con 0x7f8968105190 2024-03-26T06:34:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.377+0000 7f89577fe640 1 -- 172.21.15.37:0/3313724198 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f896000a440 con 0x7f8968105190 2024-03-26T06:34:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.377+0000 7f896f5d2740 1 Processor -- start 2024-03-26T06:34:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896f5d2740 1 -- start start 2024-03-26T06:34:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558830570d40 0x55883056fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830570450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f89577fe640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f89500a2a10 0x7f89500a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830579550 0x55883057b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896f5d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588304d45f0 con 0x55883056ffd0 2024-03-26T06:34:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896f5d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883047d930 con 0x558830579550 2024-03-26T06:34:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896f5d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588304d0690 con 0x558830570d40 2024-03-26T06:34:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830570450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830570450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52100/0 (socket says 172.21.15.37:52100) 2024-03-26T06:34:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896659c640 1 -- 172.21.15.37:0/622839585 learned_addr learned my addr 172.21.15.37:0/622839585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f8965d9b640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f89500a2a10 0x7f89500a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f89577fe640 1 -- 172.21.15.37:0/3313724198 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f89600d09e0 con 0x7f8968105190 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f8966d9d640 1 --2- 172.21.15.37:0/622839585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830579550 0x55883057b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f8965d9b640 1 --2- 172.21.15.37:0/622839585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558830570d40 0x55883056fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896659c640 1 -- 172.21.15.37:0/622839585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558830570d40 msgr2=0x55883056fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896659c640 1 --2- 172.21.15.37:0/622839585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558830570d40 0x55883056fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896659c640 1 -- 172.21.15.37:0/622839585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830579550 msgr2=0x55883057b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896659c640 1 --2- 172.21.15.37:0/622839585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830579550 0x55883057b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896659c640 1 -- 172.21.15.37:0/622839585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588301e9210 con 0x55883056ffd0 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f8966d9d640 1 --2- 172.21.15.37:0/622839585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830579550 0x55883057b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.378+0000 7f896659c640 1 --2- 172.21.15.37:0/622839585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830570450 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8958009d20 tx=0x7f8958011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d2434649385117 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f8965d9b640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f89500a2a10 0x7f89500a4ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8968197b70 tx=0x7f895c005a40 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f8954ff9640 1 -- 172.21.15.37:0/622839585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895801e070 con 0x55883056ffd0 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f8954ff9640 1 -- 172.21.15.37:0/622839585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8958019b10 con 0x55883056ffd0 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f8954ff9640 1 -- 172.21.15.37:0/622839585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895800eb30 con 0x55883056ffd0 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f896f5d2740 1 -- 172.21.15.37:0/622839585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 msgr2=0x558830570450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f896f5d2740 1 --2- 172.21.15.37:0/622839585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830570450 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8958009d20 tx=0x7f8958011810 comp rx=0 tx=0).stop 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f896f5d2740 1 -- 172.21.15.37:0/622839585 shutdown_connections 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f896f5d2740 1 --2- 172.21.15.37:0/622839585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830579550 0x55883057b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f896f5d2740 1 --2- 172.21.15.37:0/622839585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830570450 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f896f5d2740 1 --2- 172.21.15.37:0/622839585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558830570d40 0x55883056fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f896f5d2740 1 -- 172.21.15.37:0/622839585 >> 172.21.15.37:0/622839585 conn(0x5588305375b0 msgr2=0x55883056acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.379+0000 7f896f5d2740 1 -- 172.21.15.37:0/622839585 shutdown_connections 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f896f5d2740 1 -- 172.21.15.37:0/622839585 wait complete. 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f896f5d2740 1 -- start start 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f896f5d2740 1 Processor -- start 2024-03-26T06:34:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55883056ffd0 0x5588305765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830570d40 0x558830576ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830579550 0x558830577020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f896f5d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883057c280 con 0x558830570d40 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f896f5d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558830490d90 con 0x558830579550 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f896f5d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55883057c400 con 0x55883056ffd0 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f896f5d2740 10 client.0 _ll_register_callbacks cb 0x5588301e8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f8965d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830570d40 0x558830576ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f8965d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830570d40 0x558830576ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52108/0 (socket says 172.21.15.37:52108) 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f8965d9b640 1 -- 172.21.15.37:0/2698284893 learned_addr learned my addr 172.21.15.37:0/2698284893 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f8966d9d640 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830579550 0x558830577020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f896659c640 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55883056ffd0 0x5588305765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f8965d9b640 1 -- 172.21.15.37:0/2698284893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55883056ffd0 msgr2=0x5588305765a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.380+0000 7f8965d9b640 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55883056ffd0 0x5588305765a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f8965d9b640 1 -- 172.21.15.37:0/2698284893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830579550 msgr2=0x558830577020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f8965d9b640 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830579550 0x558830577020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f8965d9b640 1 -- 172.21.15.37:0/2698284893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588302b5620 con 0x558830570d40 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f8965d9b640 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830570d40 0x558830576ae0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f895c035000 tx=0x7f895c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f959a13bf16e2140 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f894dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f894dffb640 1 -- 172.21.15.37:0/2698284893 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c051020 con 0x558830570d40 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f894dffb640 1 -- 172.21.15.37:0/2698284893 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f895c0046c0 con 0x558830570d40 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f896f5d2740 1 -- 172.21.15.37:0/2698284893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8958019b10 con 0x558830570d40 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f894dffb640 1 -- 172.21.15.37:0/2698284893 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f895c046070 con 0x558830570d40 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f896f5d2740 10 client.16434 fetch_fsmap learned FSMap version 46 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f896f5d2740 1 -- 172.21.15.37:0/2698284893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x558830588580 con 0x558830570d40 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f894dffb640 1 -- 172.21.15.37:0/2698284893 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8958019b10 con 0x558830570d40 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f894dffb640 1 -- 172.21.15.37:0/2698284893 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 46) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f895c003c70 con 0x558830570d40 2024-03-26T06:34:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f896f5d2740 10 client.16434 fetch_fsmap finished waiting for FSMap version 46 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f896f5d2740 10 client.16434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f896f5d2740 20 client.16434 populate_metadata read hostname 'smithi037' 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f896f5d2740 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558830589a90 0x55883058be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f896f5d2740 4 client.16434 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.381+0000 7f896f5d2740 1 -- 172.21.15.37:0/2698284893 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588304e1b30 con 0x558830589a90 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.382+0000 7f896659c640 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558830589a90 0x55883058be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.382+0000 7f896659c640 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558830589a90 0x55883058be80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da864af6e331810a server_cookie=18722e767ec86c7b in_seq=0 out_seq=0 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.382+0000 7f894dffb640 10 client.16434 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f894dffb640 1 -- 172.21.15.37:0/2698284893 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588304e1b30 con 0x558830589a90 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f894dffb640 10 client.16434 handle_command_reply: tid=0 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 1 client.16434 shutdown 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 1 -- 172.21.15.37:0/2698284893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830570d40 msgr2=0x558830576ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830570d40 0x558830576ae0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f895c035000 tx=0x7f895c009260 comp rx=0 tx=0).stop 2024-03-26T06:34:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 1 -- 172.21.15.37:0/2698284893 shutdown_connections 2024-03-26T06:34:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558830589a90 0x55883058be80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830579550 0x558830577020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830570d40 0x558830576ae0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f894dffb640 0 client.16434 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 1 --2- 172.21.15.37:0/2698284893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55883056ffd0 0x5588305765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 1 -- 172.21.15.37:0/2698284893 >> 172.21.15.37:0/2698284893 conn(0x5588305375b0 msgr2=0x55883056ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 1 -- 172.21.15.37:0/2698284893 shutdown_connections 2024-03-26T06:34:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 1 -- 172.21.15.37:0/2698284893 wait complete. 2024-03-26T06:34:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.383+0000 7f896f5d2740 20 client.16434 trim_cache size 0 max 16384 2024-03-26T06:34:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f896f5d2740 1 Processor -- start 2024-03-26T06:34:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f896f5d2740 1 -- start start 2024-03-26T06:34:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830578cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830570d40 0x558830574e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558830579550 0x5588305753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f896f5d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681aa960 con 0x55883056ffd0 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f896f5d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681ac2e0 con 0x558830570d40 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f896f5d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681aafc0 con 0x558830579550 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f896659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830578cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f896659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830578cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52118/0 (socket says 172.21.15.37:52118) 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f896659c640 1 -- 172.21.15.37:0/2143182672 learned_addr learned my addr 172.21.15.37:0/2143182672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.396+0000 7f8965d9b640 1 --2- 172.21.15.37:0/2143182672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830570d40 0x558830574e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f896659c640 1 -- 172.21.15.37:0/2143182672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558830579550 msgr2=0x5588305753b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f8966d9d640 1 --2- 172.21.15.37:0/2143182672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558830579550 0x5588305753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f896659c640 1 --2- 172.21.15.37:0/2143182672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558830579550 0x5588305753b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f896659c640 1 -- 172.21.15.37:0/2143182672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830570d40 msgr2=0x558830574e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f896659c640 1 --2- 172.21.15.37:0/2143182672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830570d40 0x558830574e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f896659c640 1 -- 172.21.15.37:0/2143182672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55883056f2a0 con 0x55883056ffd0 2024-03-26T06:34:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f8965d9b640 1 --2- 172.21.15.37:0/2143182672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830570d40 0x558830574e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f8966d9d640 1 --2- 172.21.15.37:0/2143182672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558830579550 0x5588305753b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f896659c640 1 --2- 172.21.15.37:0/2143182672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830578cd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8958013010 tx=0x7f895800e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a681ba9e848d05 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f894ffff640 1 -- 172.21.15.37:0/2143182672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895801e040 con 0x55883056ffd0 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f894ffff640 1 -- 172.21.15.37:0/2143182672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f895802e070 con 0x55883056ffd0 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f894ffff640 1 -- 172.21.15.37:0/2143182672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958019430 con 0x55883056ffd0 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f896f5d2740 1 -- 172.21.15.37:0/2143182672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 msgr2=0x558830578cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.397+0000 7f896f5d2740 1 --2- 172.21.15.37:0/2143182672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830578cd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8958013010 tx=0x7f895800e790 comp rx=0 tx=0).stop 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 -- 172.21.15.37:0/2143182672 shutdown_connections 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 --2- 172.21.15.37:0/2143182672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558830579550 0x5588305753b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 --2- 172.21.15.37:0/2143182672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830570d40 0x558830574e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 --2- 172.21.15.37:0/2143182672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55883056ffd0 0x558830578cd0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 -- 172.21.15.37:0/2143182672 >> 172.21.15.37:0/2143182672 conn(0x5588305375b0 msgr2=0x55883056a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 -- 172.21.15.37:0/2143182672 shutdown_connections 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 -- 172.21.15.37:0/2143182672 wait complete. 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 -- start start 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 Processor -- start 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55883056ffd0 0x558830580a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830570d40 0x55883057edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830579550 0x55883057f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681abc80 con 0x558830579550 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681ab620 con 0x558830570d40 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681a8c20 con 0x55883056ffd0 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896f5d2740 10 client.0 _ll_register_callbacks cb 0x7f8958009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f8966d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830579550 0x55883057f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f8966d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830579550 0x55883057f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52128/0 (socket says 172.21.15.37:52128) 2024-03-26T06:34:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f8966d9d640 1 -- 172.21.15.37:0/1811013758 learned_addr learned my addr 172.21.15.37:0/1811013758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f8965d9b640 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830570d40 0x55883057edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.398+0000 7f896659c640 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55883056ffd0 0x558830580a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f8966d9d640 1 -- 172.21.15.37:0/1811013758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55883056ffd0 msgr2=0x558830580a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f8966d9d640 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55883056ffd0 0x558830580a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f8966d9d640 1 -- 172.21.15.37:0/1811013758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830570d40 msgr2=0x55883057edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f8966d9d640 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830570d40 0x55883057edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f8966d9d640 1 -- 172.21.15.37:0/1811013758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55883057fb40 con 0x558830579550 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f8966d9d640 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830579550 0x55883057f330 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8960099a10 tx=0x7f896009d100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57cbb9b2a1142be server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f894dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f894dffb640 1 -- 172.21.15.37:0/1811013758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89600a34c0 con 0x558830579550 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f894dffb640 1 -- 172.21.15.37:0/1811013758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f896000a730 con 0x558830579550 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f896f5d2740 1 -- 172.21.15.37:0/1811013758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f895802e070 con 0x558830579550 2024-03-26T06:34:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f894dffb640 1 -- 172.21.15.37:0/1811013758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89600a3dd0 con 0x558830579550 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f896f5d2740 10 client.16440 fetch_fsmap learned FSMap version 46 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f896f5d2740 1 -- 172.21.15.37:0/1811013758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5588305e0f90 con 0x558830579550 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f894dffb640 1 -- 172.21.15.37:0/1811013758 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f895802e070 con 0x558830579550 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f894dffb640 1 -- 172.21.15.37:0/1811013758 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 46) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8960014070 con 0x558830579550 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f896f5d2740 10 client.16440 fetch_fsmap finished waiting for FSMap version 46 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f896f5d2740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f896f5d2740 20 client.16440 populate_metadata read hostname 'smithi037' 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f896f5d2740 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5588305e16b0 0x5588305e3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f896f5d2740 4 client.16440 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.399+0000 7f896f5d2740 1 -- 172.21.15.37:0/1811013758 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588304dbe80 con 0x5588305e16b0 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.400+0000 7f896659c640 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5588305e16b0 0x5588305e3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.400+0000 7f896659c640 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5588305e16b0 0x5588305e3ad0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e79490d3faa44e server_cookie=d98b0df30a58d0e0 in_seq=0 out_seq=0 2024-03-26T06:34:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.400+0000 7f894dffb640 10 client.16440 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f894dffb640 1 -- 172.21.15.37:0/1811013758 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588304dbe80 con 0x5588305e16b0 2024-03-26T06:34:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f894dffb640 10 client.16440 handle_command_reply: tid=0 2024-03-26T06:34:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 1 client.16440 shutdown 2024-03-26T06:34:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 1 -- 172.21.15.37:0/1811013758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830579550 msgr2=0x55883057f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830579550 0x55883057f330 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8960099a10 tx=0x7f896009d100 comp rx=0 tx=0).stop 2024-03-26T06:34:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 1 -- 172.21.15.37:0/1811013758 shutdown_connections 2024-03-26T06:34:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5588305e16b0 0x5588305e3ad0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558830579550 0x55883057f330 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558830570d40 0x55883057edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f894dffb640 0 client.16440 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 1 --2- 172.21.15.37:0/1811013758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55883056ffd0 0x558830580a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 1 -- 172.21.15.37:0/1811013758 >> 172.21.15.37:0/1811013758 conn(0x5588305375b0 msgr2=0x55883056a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 1 -- 172.21.15.37:0/1811013758 shutdown_connections 2024-03-26T06:34:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 1 -- 172.21.15.37:0/1811013758 wait complete. 2024-03-26T06:34:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.434+0000 7f896f5d2740 20 client.16440 trim_cache size 0 max 16384 2024-03-26T06:34:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:34:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:34:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:34:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:34:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 -- 172.21.15.37:0/3313724198 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f89500a2a10 msgr2=0x7f89500a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f89500a2a10 0x7f89500a4ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8968197b70 tx=0x7f895c005a40 comp rx=0 tx=0).stop 2024-03-26T06:34:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 -- 172.21.15.37:0/3313724198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8968105190 msgr2=0x7f8968196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8968105190 0x7f8968196870 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8960007cb0 tx=0x7f896000e420 comp rx=0 tx=0).stop 2024-03-26T06:34:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 -- 172.21.15.37:0/3313724198 shutdown_connections 2024-03-26T06:34:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f89500a2a10 0x7f89500a4ed0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8968105190 0x7f8968196870 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8968100e00 0x7f896819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 --2- 172.21.15.37:0/3313724198 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8968100440 0x7f896819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 -- 172.21.15.37:0/3313724198 >> 172.21.15.37:0/3313724198 conn(0x7f8968077040 msgr2=0x7f89680fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 -- 172.21.15.37:0/3313724198 shutdown_connections 2024-03-26T06:34:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:25.435+0000 7f896c91b640 1 -- 172.21.15.37:0/3313724198 wait complete. 2024-03-26T06:34:25.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 433556 4.02 MB/sec execute 480 sec latency 2207.505 ms 2024-03-26T06:34:26.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 433929 4.02 MB/sec execute 481 sec latency 2993.723 ms 2024-03-26T06:34:27.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:26 smithi066 ceph-mon[36504]: pgmap v358: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-03-26T06:34:27.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:26 smithi149 ceph-mon[36681]: pgmap v358: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-03-26T06:34:27.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:26 smithi037 ceph-mon[31974]: pgmap v358: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-03-26T06:34:27.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 436274 4.04 MB/sec execute 482 sec latency 15.294 ms 2024-03-26T06:34:28.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 436979 4.04 MB/sec execute 483 sec latency 320.535 ms 2024-03-26T06:34:29.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:28 smithi066 ceph-mon[36504]: pgmap v359: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-26T06:34:29.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:28 smithi149 ceph-mon[36681]: pgmap v359: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-26T06:34:29.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:28 smithi037 ceph-mon[31974]: pgmap v359: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-26T06:34:29.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 436979 4.03 MB/sec execute 484 sec latency 1320.670 ms 2024-03-26T06:34:30.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 436979 4.02 MB/sec execute 485 sec latency 2320.731 ms 2024-03-26T06:34:31.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:30 smithi066 ceph-mon[36504]: pgmap v360: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-26T06:34:31.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:30 smithi149 ceph-mon[36681]: pgmap v360: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-26T06:34:31.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:30 smithi037 ceph-mon[31974]: pgmap v360: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-26T06:34:31.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 437035 4.01 MB/sec execute 486 sec latency 3110.720 ms 2024-03-26T06:34:32.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 437035 4.00 MB/sec execute 487 sec latency 1170.022 ms 2024-03-26T06:34:33.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:32 smithi066 ceph-mon[36504]: pgmap v361: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-03-26T06:34:33.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:32 smithi149 ceph-mon[36681]: pgmap v361: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-03-26T06:34:33.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:32 smithi037 ceph-mon[31974]: pgmap v361: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-03-26T06:34:33.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 437035 4.00 MB/sec execute 488 sec latency 2170.126 ms 2024-03-26T06:34:34.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 437035 3.99 MB/sec execute 489 sec latency 3170.211 ms 2024-03-26T06:34:35.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:34 smithi066 ceph-mon[36504]: pgmap v362: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1001 KiB/s wr, 113 op/s 2024-03-26T06:34:35.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:34 smithi149 ceph-mon[36681]: pgmap v362: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1001 KiB/s wr, 113 op/s 2024-03-26T06:34:35.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:34 smithi037 ceph-mon[31974]: pgmap v362: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1001 KiB/s wr, 113 op/s 2024-03-26T06:34:35.528 INFO:tasks.workunit.client.0.smithi037.stdout: 1 437035 3.98 MB/sec execute 490 sec latency 4170.289 ms 2024-03-26T06:34:36.529 INFO:tasks.workunit.client.0.smithi037.stdout: 1 437373 3.97 MB/sec execute 491 sec latency 4961.545 ms 2024-03-26T06:34:37.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:36 smithi066 ceph-mon[36504]: pgmap v363: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-03-26T06:34:37.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:36 smithi149 ceph-mon[36681]: pgmap v363: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-03-26T06:34:37.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:36 smithi037 ceph-mon[31974]: pgmap v363: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-03-26T06:34:37.529 INFO:tasks.workunit.client.0.smithi037.stdout: 1 439653 3.99 MB/sec execute 492 sec latency 15.480 ms 2024-03-26T06:34:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.521+0000 7f85f831b640 1 -- 172.21.15.37:0/4019720306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85f0100e00 msgr2=0x7f85f010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.521+0000 7f85f831b640 1 --2- 172.21.15.37:0/4019720306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85f0100e00 0x7f85f010d020 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009ca0 tx=0x7f85e802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.521+0000 7f85f831b640 1 -- 172.21.15.37:0/4019720306 shutdown_connections 2024-03-26T06:34:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.521+0000 7f85f831b640 1 --2- 172.21.15.37:0/4019720306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85f0100e00 0x7f85f010d020 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.521+0000 7f85f831b640 1 --2- 172.21.15.37:0/4019720306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85f0100440 0x7f85f01008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.521+0000 7f85f831b640 1 --2- 172.21.15.37:0/4019720306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85f0105190 0x7f85f0105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.521+0000 7f85f831b640 1 -- 172.21.15.37:0/4019720306 >> 172.21.15.37:0/4019720306 conn(0x7f85f0077040 msgr2=0x7f85f0077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.521+0000 7f85f831b640 1 -- 172.21.15.37:0/4019720306 shutdown_connections 2024-03-26T06:34:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.521+0000 7f85f831b640 1 -- 172.21.15.37:0/4019720306 wait complete. 2024-03-26T06:34:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f831b640 1 Processor -- start 2024-03-26T06:34:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f831b640 1 -- start start 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f831b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85f0100440 0x7f85f010db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f831b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85f0100e00 0x7f85f010e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f831b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85f0105190 0x7f85f010e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f831b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0111790 con 0x7f85f0100440 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f831b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0111610 con 0x7f85f0100e00 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f831b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0111910 con 0x7f85f0105190 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f60b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85f0100440 0x7f85f010db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f60b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85f0100440 0x7f85f010db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47940/0 (socket says 172.21.15.37:47940) 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f60b7640 1 -- 172.21.15.37:0/1989914761 learned_addr learned my addr 172.21.15.37:0/1989914761 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f58b6640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85f0100e00 0x7f85f010e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f68b8640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85f0105190 0x7f85f010e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f60b7640 1 -- 172.21.15.37:0/1989914761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85f0105190 msgr2=0x7f85f010e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f60b7640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85f0105190 0x7f85f010e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f60b7640 1 -- 172.21.15.37:0/1989914761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85f0100e00 msgr2=0x7f85f010e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f60b7640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85f0100e00 0x7f85f010e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f60b7640 1 -- 172.21.15.37:0/1989914761 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f01a5f90 con 0x7f85f0100440 2024-03-26T06:34:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85f60b7640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85f0100440 0x7f85f010db60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00d950 tx=0x7f85ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.522+0000 7f85e77fe640 1 -- 172.21.15.37:0/1989914761 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec017070 con 0x7f85f0100440 2024-03-26T06:34:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.523+0000 7f85e77fe640 1 -- 172.21.15.37:0/1989914761 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85ec009c30 con 0x7f85f0100440 2024-03-26T06:34:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.523+0000 7f85f831b640 1 -- 172.21.15.37:0/1989914761 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f01a6280 con 0x7f85f0100440 2024-03-26T06:34:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.523+0000 7f85e77fe640 1 -- 172.21.15.37:0/1989914761 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec00f6b0 con 0x7f85f0100440 2024-03-26T06:34:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.523+0000 7f85f831b640 1 -- 172.21.15.37:0/1989914761 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f01a6820 con 0x7f85f0100440 2024-03-26T06:34:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.525+0000 7f85e77fe640 1 -- 172.21.15.37:0/1989914761 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f85ec0109d0 con 0x7f85f0100440 2024-03-26T06:34:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.525+0000 7f85fafd9740 1 Processor -- start 2024-03-26T06:34:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.525+0000 7f85fafd9740 1 -- start start 2024-03-26T06:34:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85e77fe640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85dc0a2ac0 0x7f85dc0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd68a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd68fd0 0x55a66bd69450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd72550 0x55a66bd74940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85fafd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66bccd5f0 con 0x55a66bd69d40 2024-03-26T06:34:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85fafd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66bc76930 con 0x55a66bd68fd0 2024-03-26T06:34:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85fafd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66bcc9690 con 0x55a66bd72550 2024-03-26T06:34:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f68b8640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85dc0a2ac0 0x7f85dc0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85e77fe640 1 -- 172.21.15.37:0/1989914761 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f85ec0ce900 con 0x7f85f0100440 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f60b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd68a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f60b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd68a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47946/0 (socket says 172.21.15.37:47946) 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f60b7640 1 -- 172.21.15.37:0/3027483363 learned_addr learned my addr 172.21.15.37:0/3027483363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f68b8640 1 --2- 172.21.15.37:0/3027483363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd72550 0x55a66bd74940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f58b6640 1 --2- 172.21.15.37:0/3027483363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd68fd0 0x55a66bd69450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f60b7640 1 -- 172.21.15.37:0/3027483363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd72550 msgr2=0x55a66bd74940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f60b7640 1 --2- 172.21.15.37:0/3027483363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd72550 0x55a66bd74940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f60b7640 1 -- 172.21.15.37:0/3027483363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd68fd0 msgr2=0x55a66bd69450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f60b7640 1 --2- 172.21.15.37:0/3027483363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd68fd0 0x55a66bd69450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f60b7640 1 -- 172.21.15.37:0/3027483363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a66b9e2210 con 0x55a66bd69d40 2024-03-26T06:34:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f68b8640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85dc0a2ac0 0x7f85dc0a4f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f85e8004af0 tx=0x7f85e8032e90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:38.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 440522 3.99 MB/sec execute 493 sec latency 227.760 ms 2024-03-26T06:34:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.526+0000 7f85f60b7640 1 --2- 172.21.15.37:0/3027483363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd68a90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0a31d0 tx=0x7f85ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bc3571f5c22a92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85e4bfe640 1 -- 172.21.15.37:0/3027483363 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec020070 con 0x55a66bd69d40 2024-03-26T06:34:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85e4bfe640 1 -- 172.21.15.37:0/3027483363 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85ec09b010 con 0x55a66bd69d40 2024-03-26T06:34:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85e4bfe640 1 -- 172.21.15.37:0/3027483363 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0990b0 con 0x55a66bd69d40 2024-03-26T06:34:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85fafd9740 1 -- 172.21.15.37:0/3027483363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 msgr2=0x55a66bd68a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3027483363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd68a90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0a31d0 tx=0x7f85ec005880 comp rx=0 tx=0).stop 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85fafd9740 1 -- 172.21.15.37:0/3027483363 shutdown_connections 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3027483363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd72550 0x55a66bd74940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3027483363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd68fd0 0x55a66bd69450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3027483363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd68a90 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85fafd9740 1 -- 172.21.15.37:0/3027483363 >> 172.21.15.37:0/3027483363 conn(0x55a66bd305b0 msgr2=0x55a66bd63cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85fafd9740 1 -- 172.21.15.37:0/3027483363 shutdown_connections 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.527+0000 7f85fafd9740 1 -- 172.21.15.37:0/3027483363 wait complete. 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85fafd9740 1 -- start start 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85fafd9740 1 Processor -- start 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd6f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd69d40 0x55a66bd6fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd70040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85fafd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66bd75280 con 0x55a66bd72550 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85fafd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66bc89d90 con 0x55a66bd69d40 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85fafd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66bd75400 con 0x55a66bd68fd0 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85fafd9740 10 client.0 _ll_register_callbacks cb 0x55a66b9e1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85f68b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd70040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85f68b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd70040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47956/0 (socket says 172.21.15.37:47956) 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85f68b8640 1 -- 172.21.15.37:0/1873580434 learned_addr learned my addr 172.21.15.37:0/1873580434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85f60b7640 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd6f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85f58b6640 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd69d40 0x55a66bd6fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85f68b8640 1 -- 172.21.15.37:0/1873580434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 msgr2=0x55a66bd6f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85f68b8640 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd6f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85f68b8640 1 -- 172.21.15.37:0/1873580434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd69d40 msgr2=0x55a66bd6fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85f68b8640 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd69d40 0x55a66bd6fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.528+0000 7f85f68b8640 1 -- 172.21.15.37:0/1873580434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a66baae620 con 0x55a66bd72550 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85f68b8640 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd70040 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f85e8032530 tx=0x7f85e802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34932b26019fa436 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85d9ffb640 1 -- 172.21.15.37:0/1873580434 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8052060 con 0x55a66bd72550 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85d9ffb640 1 -- 172.21.15.37:0/1873580434 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f85e8004450 con 0x55a66bd72550 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85fafd9740 1 -- 172.21.15.37:0/1873580434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85ec09b010 con 0x55a66bd72550 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85d9ffb640 1 -- 172.21.15.37:0/1873580434 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e8039040 con 0x55a66bd72550 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85fafd9740 10 client.16452 fetch_fsmap learned FSMap version 46 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85fafd9740 1 -- 172.21.15.37:0/1873580434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55a66bd815b0 con 0x55a66bd72550 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85d9ffb640 1 -- 172.21.15.37:0/1873580434 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85ec09b010 con 0x55a66bd72550 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85d9ffb640 1 -- 172.21.15.37:0/1873580434 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 46) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f85e803f070 con 0x55a66bd72550 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85fafd9740 10 client.16452 fetch_fsmap finished waiting for FSMap version 46 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85fafd9740 10 client.16452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:34:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85fafd9740 20 client.16452 populate_metadata read hostname 'smithi037' 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.529+0000 7f85fafd9740 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a66bd82a70 0x55a66bd84e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.530+0000 7f85fafd9740 4 client.16452 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.530+0000 7f85fafd9740 1 -- 172.21.15.37:0/1873580434 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a66bcdab30 con 0x55a66bd82a70 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.530+0000 7f85f60b7640 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a66bd82a70 0x55a66bd84e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.530+0000 7f85f60b7640 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a66bd82a70 0x55a66bd84e60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cb476cc24f6624e server_cookie=74e9ec417f858f9a in_seq=0 out_seq=0 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.530+0000 7f85d9ffb640 10 client.16452 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85d9ffb640 1 -- 172.21.15.37:0/1873580434 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a66bcdab30 con 0x55a66bd82a70 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85d9ffb640 10 client.16452 handle_command_reply: tid=0 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 1 client.16452 shutdown 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 1 -- 172.21.15.37:0/1873580434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 msgr2=0x55a66bd70040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd70040 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f85e8032530 tx=0x7f85e802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 1 -- 172.21.15.37:0/1873580434 shutdown_connections 2024-03-26T06:34:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a66bd82a70 0x55a66bd84e60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd70040 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd69d40 0x55a66bd6fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 1 --2- 172.21.15.37:0/1873580434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd6f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85d9ffb640 0 client.16452 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 1 -- 172.21.15.37:0/1873580434 >> 172.21.15.37:0/1873580434 conn(0x55a66bd305b0 msgr2=0x55a66bd65a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 1 -- 172.21.15.37:0/1873580434 shutdown_connections 2024-03-26T06:34:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 1 -- 172.21.15.37:0/1873580434 wait complete. 2024-03-26T06:34:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.531+0000 7f85fafd9740 20 client.16452 trim_cache size 0 max 16384 2024-03-26T06:34:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.544+0000 7f85fafd9740 1 Processor -- start 2024-03-26T06:34:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.544+0000 7f85fafd9740 1 -- start start 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.544+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd71cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.544+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd6c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.544+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd72550 0x55a66bd6cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.544+0000 7f85fafd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01aef10 con 0x55a66bd69d40 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.544+0000 7f85fafd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01b0890 con 0x55a66bd72550 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.544+0000 7f85fafd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01af570 con 0x55a66bd68fd0 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.544+0000 7f85f58b6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd6c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85f58b6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd6c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47970/0 (socket says 172.21.15.37:47970) 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85f58b6640 1 -- 172.21.15.37:0/3777061048 learned_addr learned my addr 172.21.15.37:0/3777061048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85f60b7640 1 --2- 172.21.15.37:0/3777061048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd71cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85f68b8640 1 --2- 172.21.15.37:0/3777061048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd72550 0x55a66bd6cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85f58b6640 1 -- 172.21.15.37:0/3777061048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 msgr2=0x55a66bd71cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85f58b6640 1 --2- 172.21.15.37:0/3777061048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd71cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85f58b6640 1 -- 172.21.15.37:0/3777061048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd72550 msgr2=0x55a66bd6cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85f58b6640 1 --2- 172.21.15.37:0/3777061048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd72550 0x55a66bd6cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85f58b6640 1 -- 172.21.15.37:0/3777061048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a66bd851d0 con 0x55a66bd69d40 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85f68b8640 1 --2- 172.21.15.37:0/3777061048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd72550 0x55a66bd6cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85f58b6640 1 --2- 172.21.15.37:0/3777061048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd6c860 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f85e0004620 tx=0x7f85e000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4d9e4be47159ca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85dbfff640 1 -- 172.21.15.37:0/3777061048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e000be50 con 0x55a66bd69d40 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85dbfff640 1 -- 172.21.15.37:0/3777061048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85e0011440 con 0x55a66bd69d40 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.545+0000 7f85dbfff640 1 -- 172.21.15.37:0/3777061048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0010690 con 0x55a66bd69d40 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 -- 172.21.15.37:0/3777061048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 msgr2=0x55a66bd6c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3777061048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd6c860 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f85e0004620 tx=0x7f85e000bc00 comp rx=0 tx=0).stop 2024-03-26T06:34:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 -- 172.21.15.37:0/3777061048 shutdown_connections 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3777061048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd72550 0x55a66bd6cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3777061048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd69d40 0x55a66bd6c860 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3777061048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd71cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 -- 172.21.15.37:0/3777061048 >> 172.21.15.37:0/3777061048 conn(0x55a66bd305b0 msgr2=0x55a66bd6deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 -- 172.21.15.37:0/3777061048 shutdown_connections 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 -- 172.21.15.37:0/3777061048 wait complete. 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 -- start start 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 Processor -- start 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd78100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd69d40 0x55a66bd78640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd78b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01b0230 con 0x55a66bd72550 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01afbd0 con 0x55a66bd69d40 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01ad1d0 con 0x55a66bd68fd0 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85fafd9740 10 client.0 _ll_register_callbacks cb 0x7f85ec00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85f68b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd78b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85f68b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd78b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47984/0 (socket says 172.21.15.37:47984) 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85f68b8640 1 -- 172.21.15.37:0/3464190105 learned_addr learned my addr 172.21.15.37:0/3464190105 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.546+0000 7f85f60b7640 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd78100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85f58b6640 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd69d40 0x55a66bd78640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85f68b8640 1 -- 172.21.15.37:0/3464190105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 msgr2=0x55a66bd78100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85f68b8640 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd78100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85f68b8640 1 -- 172.21.15.37:0/3464190105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd69d40 msgr2=0x55a66bd78640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85f68b8640 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd69d40 0x55a66bd78640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85f68b8640 1 -- 172.21.15.37:0/3464190105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a66bdd9040 con 0x55a66bd72550 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85f58b6640 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd69d40 0x55a66bd78640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85f68b8640 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd78b80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009a60 tx=0x7f85e8009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b825eea5a418b811 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85d9ffb640 1 -- 172.21.15.37:0/3464190105 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8052040 con 0x55a66bd72550 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85d9ffb640 1 -- 172.21.15.37:0/3464190105 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f85e8004450 con 0x55a66bd72550 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85fafd9740 1 -- 172.21.15.37:0/3464190105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85e0011440 con 0x55a66bd72550 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85d9ffb640 1 -- 172.21.15.37:0/3464190105 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e8005bd0 con 0x55a66bd72550 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85fafd9740 10 client.16458 fetch_fsmap learned FSMap version 46 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85fafd9740 1 -- 172.21.15.37:0/3464190105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55a66bdda350 con 0x55a66bd72550 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85d9ffb640 1 -- 172.21.15.37:0/3464190105 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85e0011440 con 0x55a66bd72550 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85d9ffb640 1 -- 172.21.15.37:0/3464190105 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 46) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f85e803f040 con 0x55a66bd72550 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85fafd9740 10 client.16458 fetch_fsmap finished waiting for FSMap version 46 2024-03-26T06:34:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85fafd9740 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:34:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85fafd9740 20 client.16458 populate_metadata read hostname 'smithi037' 2024-03-26T06:34:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a66bdda9f0 0x55a66bddce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85fafd9740 4 client.16458 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:34:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.547+0000 7f85fafd9740 1 -- 172.21.15.37:0/3464190105 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a66bcd4e80 con 0x55a66bdda9f0 2024-03-26T06:34:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.548+0000 7f85f60b7640 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a66bdda9f0 0x55a66bddce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.548+0000 7f85f60b7640 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a66bdda9f0 0x55a66bddce10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917d999b53eb5a3b server_cookie=81fa2b239665f3a1 in_seq=0 out_seq=0 2024-03-26T06:34:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.548+0000 7f85d9ffb640 10 client.16458 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.579+0000 7f85d9ffb640 1 -- 172.21.15.37:0/3464190105 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a66bcd4e80 con 0x55a66bdda9f0 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.579+0000 7f85d9ffb640 10 client.16458 handle_command_reply: tid=0 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 1 client.16458 shutdown 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 1 -- 172.21.15.37:0/3464190105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 msgr2=0x55a66bd78b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd78b80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009a60 tx=0x7f85e8009360 comp rx=0 tx=0).stop 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 1 -- 172.21.15.37:0/3464190105 shutdown_connections 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a66bdda9f0 0x55a66bddce10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a66bd72550 0x55a66bd78b80 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a66bd69d40 0x55a66bd78640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85d9ffb640 0 client.16458 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 1 --2- 172.21.15.37:0/3464190105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a66bd68fd0 0x55a66bd78100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 1 -- 172.21.15.37:0/3464190105 >> 172.21.15.37:0/3464190105 conn(0x55a66bd305b0 msgr2=0x55a66bd6f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 1 -- 172.21.15.37:0/3464190105 shutdown_connections 2024-03-26T06:34:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 1 -- 172.21.15.37:0/3464190105 wait complete. 2024-03-26T06:34:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85fafd9740 20 client.16458 trim_cache size 0 max 16384 2024-03-26T06:34:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:34:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:34:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:34:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:34:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85f831b640 1 -- 172.21.15.37:0/1989914761 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85dc0a2ac0 msgr2=0x7f85dc0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85f831b640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85dc0a2ac0 0x7f85dc0a4f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f85e8004af0 tx=0x7f85e8032e90 comp rx=0 tx=0).stop 2024-03-26T06:34:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85f831b640 1 -- 172.21.15.37:0/1989914761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85f0100440 msgr2=0x7f85f010db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85f831b640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85f0100440 0x7f85f010db60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00d950 tx=0x7f85ec00de10 comp rx=0 tx=0).stop 2024-03-26T06:34:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85f831b640 1 -- 172.21.15.37:0/1989914761 shutdown_connections 2024-03-26T06:34:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85f831b640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85dc0a2ac0 0x7f85dc0a4f80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85f831b640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85f0105190 0x7f85f010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85f831b640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85f0100e00 0x7f85f010e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85f831b640 1 --2- 172.21.15.37:0/1989914761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85f0100440 0x7f85f010db60 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.580+0000 7f85f831b640 1 -- 172.21.15.37:0/1989914761 >> 172.21.15.37:0/1989914761 conn(0x7f85f0077040 msgr2=0x7f85f00fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.581+0000 7f85f831b640 1 -- 172.21.15.37:0/1989914761 shutdown_connections 2024-03-26T06:34:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:38.581+0000 7f85f831b640 1 -- 172.21.15.37:0/1989914761 wait complete. 2024-03-26T06:34:39.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:38 smithi066 ceph-mon[36504]: pgmap v364: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 697 KiB/s wr, 77 op/s 2024-03-26T06:34:39.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:38 smithi149 ceph-mon[36681]: pgmap v364: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 697 KiB/s wr, 77 op/s 2024-03-26T06:34:39.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:38 smithi037 ceph-mon[31974]: pgmap v364: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 697 KiB/s wr, 77 op/s 2024-03-26T06:34:39.529 INFO:tasks.workunit.client.0.smithi037.stdout: 1 440522 3.99 MB/sec execute 494 sec latency 1227.857 ms 2024-03-26T06:34:40.529 INFO:tasks.workunit.client.0.smithi037.stdout: 1 440522 3.98 MB/sec execute 495 sec latency 2227.972 ms 2024-03-26T06:34:41.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:40 smithi066 ceph-mon[36504]: pgmap v365: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 697 KiB/s wr, 77 op/s 2024-03-26T06:34:41.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:40 smithi149 ceph-mon[36681]: pgmap v365: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 697 KiB/s wr, 77 op/s 2024-03-26T06:34:41.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:40 smithi037 ceph-mon[31974]: pgmap v365: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 697 KiB/s wr, 77 op/s 2024-03-26T06:34:41.529 INFO:tasks.workunit.client.0.smithi037.stdout: 1 440646 3.97 MB/sec execute 496 sec latency 3014.939 ms 2024-03-26T06:34:42.529 INFO:tasks.workunit.client.0.smithi037.stdout: 1 440646 3.96 MB/sec execute 497 sec latency 1116.730 ms 2024-03-26T06:34:43.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:42 smithi066 ceph-mon[36504]: pgmap v366: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-03-26T06:34:43.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:42 smithi149 ceph-mon[36681]: pgmap v366: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-03-26T06:34:43.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:42 smithi037 ceph-mon[31974]: pgmap v366: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-03-26T06:34:43.529 INFO:tasks.workunit.client.0.smithi037.stdout: 1 440646 3.95 MB/sec execute 498 sec latency 2116.820 ms 2024-03-26T06:34:44.529 INFO:tasks.workunit.client.0.smithi037.stdout: 1 440646 3.95 MB/sec execute 499 sec latency 3116.943 ms 2024-03-26T06:34:45.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:44 smithi066 ceph-mon[36504]: pgmap v367: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 697 KiB/s wr, 73 op/s 2024-03-26T06:34:45.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:44 smithi149 ceph-mon[36681]: pgmap v367: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 697 KiB/s wr, 73 op/s 2024-03-26T06:34:45.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:44 smithi037 ceph-mon[31974]: pgmap v367: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 697 KiB/s wr, 73 op/s 2024-03-26T06:34:45.529 INFO:tasks.workunit.client.0.smithi037.stdout: 1 440646 3.94 MB/sec execute 500 sec latency 4117.050 ms 2024-03-26T06:34:46.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:45 smithi066 ceph-mon[36504]: pgmap v368: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 756 KiB/s wr, 86 op/s 2024-03-26T06:34:46.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:45 smithi149 ceph-mon[36681]: pgmap v368: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 756 KiB/s wr, 86 op/s 2024-03-26T06:34:46.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:45 smithi037 ceph-mon[31974]: pgmap v368: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 756 KiB/s wr, 86 op/s 2024-03-26T06:34:46.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 441019 3.93 MB/sec execute 501 sec latency 4905.680 ms 2024-03-26T06:34:47.529 INFO:tasks.workunit.client.0.smithi037.stdout: 1 443218 3.95 MB/sec execute 502 sec latency 17.897 ms 2024-03-26T06:34:48.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:47 smithi066 ceph-mon[36504]: pgmap v369: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 712 KiB/s wr, 75 op/s 2024-03-26T06:34:48.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:47 smithi149 ceph-mon[36681]: pgmap v369: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 712 KiB/s wr, 75 op/s 2024-03-26T06:34:48.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:47 smithi037 ceph-mon[31974]: pgmap v369: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 712 KiB/s wr, 75 op/s 2024-03-26T06:34:48.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 444127 3.95 MB/sec execute 503 sec latency 240.816 ms 2024-03-26T06:34:49.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 444127 3.94 MB/sec execute 504 sec latency 1240.902 ms 2024-03-26T06:34:50.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:49 smithi066 ceph-mon[36504]: pgmap v370: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 712 KiB/s wr, 75 op/s 2024-03-26T06:34:50.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:49 smithi149 ceph-mon[36681]: pgmap v370: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 712 KiB/s wr, 75 op/s 2024-03-26T06:34:50.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:49 smithi037 ceph-mon[31974]: pgmap v370: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 712 KiB/s wr, 75 op/s 2024-03-26T06:34:50.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 444127 3.94 MB/sec execute 505 sec latency 2240.999 ms 2024-03-26T06:34:51.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 445189 3.94 MB/sec execute 506 sec latency 2633.009 ms 2024-03-26T06:34:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.678+0000 7faf5bcd7640 1 -- 172.21.15.37:0/2194473935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf541073e0 msgr2=0x7faf541077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.678+0000 7faf5bcd7640 1 --2- 172.21.15.37:0/2194473935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf541073e0 0x7faf541077c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faf50009860 tx=0x7faf5002bc00 comp rx=0 tx=0).stop 2024-03-26T06:34:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.678+0000 7faf5bcd7640 1 -- 172.21.15.37:0/2194473935 shutdown_connections 2024-03-26T06:34:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.678+0000 7faf5bcd7640 1 --2- 172.21.15.37:0/2194473935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf54069310 0x7faf5410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.678+0000 7faf5bcd7640 1 --2- 172.21.15.37:0/2194473935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54068970 0x7faf54068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.678+0000 7faf5bcd7640 1 --2- 172.21.15.37:0/2194473935 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf541073e0 0x7faf541077c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.678+0000 7faf5bcd7640 1 -- 172.21.15.37:0/2194473935 >> 172.21.15.37:0/2194473935 conn(0x7faf54077040 msgr2=0x7faf54077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.678+0000 7faf5bcd7640 1 -- 172.21.15.37:0/2194473935 shutdown_connections 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.678+0000 7faf5bcd7640 1 -- 172.21.15.37:0/2194473935 wait complete. 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf5bcd7640 1 Processor -- start 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf5bcd7640 1 -- start start 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf5bcd7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54068970 0x7faf54100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf5bcd7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf54069310 0x7faf54100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf5bcd7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf541073e0 0x7faf54100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf5bcd7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf54111660 con 0x7faf541073e0 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf5bcd7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541114e0 con 0x7faf54069310 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf5bcd7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541117e0 con 0x7faf54068970 2024-03-26T06:34:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf5a274640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf541073e0 0x7faf54100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf59272640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf54069310 0x7faf54100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf5a274640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf541073e0 0x7faf54100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40406/0 (socket says 172.21.15.37:40406) 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf5a274640 1 -- 172.21.15.37:0/3019670285 learned_addr learned my addr 172.21.15.37:0/3019670285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.679+0000 7faf59a73640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54068970 0x7faf54100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf59272640 1 -- 172.21.15.37:0/3019670285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54068970 msgr2=0x7faf54100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf59272640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54068970 0x7faf54100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf59272640 1 -- 172.21.15.37:0/3019670285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf541073e0 msgr2=0x7faf54100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf59272640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf541073e0 0x7faf54100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf59272640 1 -- 172.21.15.37:0/3019670285 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf541a5fc0 con 0x7faf54069310 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf5a274640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf541073e0 0x7faf54100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf59272640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf54069310 0x7faf54100a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf4400e9f0 tx=0x7faf4400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf4affd640 1 -- 172.21.15.37:0/3019670285 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400ccc0 con 0x7faf54069310 2024-03-26T06:34:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf4affd640 1 -- 172.21.15.37:0/3019670285 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf44005020 con 0x7faf54069310 2024-03-26T06:34:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf5bcd7640 1 -- 172.21.15.37:0/3019670285 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf541a62b0 con 0x7faf54069310 2024-03-26T06:34:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf4affd640 1 -- 172.21.15.37:0/3019670285 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf440049e0 con 0x7faf54069310 2024-03-26T06:34:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.680+0000 7faf5bcd7640 1 -- 172.21.15.37:0/3019670285 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf541a67b0 con 0x7faf54069310 2024-03-26T06:34:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.682+0000 7faf4affd640 1 -- 172.21.15.37:0/3019670285 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7faf44005560 con 0x7faf54069310 2024-03-26T06:34:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.683+0000 7faf5e98b740 1 Processor -- start 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.683+0000 7faf5e98b740 1 -- start start 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.683+0000 7faf4affd640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7faf400a2a40 0x7faf400a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 0x5614df7f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f7fd0 0x5614df7f8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df801550 0x5614df803940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf5e98b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df75c5f0 con 0x5614df7f7fd0 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf5e98b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df705930 con 0x5614df7f8d40 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf5e98b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df758690 con 0x5614df801550 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf5a274640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7faf400a2a40 0x7faf400a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf4affd640 1 -- 172.21.15.37:0/3019670285 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7faf440ced80 con 0x7faf54069310 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf59a73640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 0x5614df7f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf59a73640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 0x5614df7f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:54392/0 (socket says 172.21.15.37:54392) 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf59a73640 1 -- 172.21.15.37:0/1690054559 learned_addr learned my addr 172.21.15.37:0/1690054559 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf59272640 1 --2- 172.21.15.37:0/1690054559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f7fd0 0x5614df7f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf5a274640 1 --2- 172.21.15.37:0/1690054559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df801550 0x5614df803940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf59a73640 1 -- 172.21.15.37:0/1690054559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df801550 msgr2=0x5614df803940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf59a73640 1 --2- 172.21.15.37:0/1690054559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df801550 0x5614df803940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf59a73640 1 -- 172.21.15.37:0/1690054559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f7fd0 msgr2=0x5614df7f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf59a73640 1 --2- 172.21.15.37:0/1690054559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f7fd0 0x5614df7f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf59a73640 1 -- 172.21.15.37:0/1690054559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614df471210 con 0x5614df7f8d40 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf59a73640 1 --2- 172.21.15.37:0/1690054559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 0x5614df7f7a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf50009780 tx=0x7faf50009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bedcfcfbc158e5d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.684+0000 7faf5a274640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7faf400a2a40 0x7faf400a4f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x5614df7f3e50 tx=0x7faf4c00e500 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.685+0000 7faf3ffff640 1 -- 172.21.15.37:0/1690054559 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50043020 con 0x5614df7f8d40 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.685+0000 7faf3ffff640 1 -- 172.21.15.37:0/1690054559 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf50034440 con 0x5614df7f8d40 2024-03-26T06:34:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.685+0000 7faf3ffff640 1 -- 172.21.15.37:0/1690054559 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50039040 con 0x5614df7f8d40 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.685+0000 7faf5e98b740 1 -- 172.21.15.37:0/1690054559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 msgr2=0x5614df7f7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.685+0000 7faf5e98b740 1 --2- 172.21.15.37:0/1690054559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 0x5614df7f7a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf50009780 tx=0x7faf50009510 comp rx=0 tx=0).stop 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.685+0000 7faf5e98b740 1 -- 172.21.15.37:0/1690054559 shutdown_connections 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.685+0000 7faf5e98b740 1 --2- 172.21.15.37:0/1690054559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df801550 0x5614df803940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.685+0000 7faf5e98b740 1 --2- 172.21.15.37:0/1690054559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f7fd0 0x5614df7f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.685+0000 7faf5e98b740 1 --2- 172.21.15.37:0/1690054559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 0x5614df7f7a90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.685+0000 7faf5e98b740 1 -- 172.21.15.37:0/1690054559 >> 172.21.15.37:0/1690054559 conn(0x5614df7bf5b0 msgr2=0x5614df7f2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.686+0000 7faf5e98b740 1 -- 172.21.15.37:0/1690054559 shutdown_connections 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.686+0000 7faf5e98b740 1 -- 172.21.15.37:0/1690054559 wait complete. 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.686+0000 7faf5e98b740 1 -- start start 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.686+0000 7faf5e98b740 1 Processor -- start 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f7fd0 0x5614df7fe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f8d40 0x5614df7fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df801550 0x5614df7fef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf5e98b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df804220 con 0x5614df7f7fd0 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf5e98b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df718d90 con 0x5614df801550 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf5e98b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df8043a0 con 0x5614df7f8d40 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf5e98b740 10 client.0 _ll_register_callbacks cb 0x5614df470a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf59a73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f7fd0 0x5614df7fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf59272640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f8d40 0x5614df7fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf5a274640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df801550 0x5614df7fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf59a73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f7fd0 0x5614df7fe510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40432/0 (socket says 172.21.15.37:40432) 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf59272640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f8d40 0x5614df7fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:42458/0 (socket says 172.21.15.37:42458) 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf59a73640 1 -- 172.21.15.37:0/673373195 learned_addr learned my addr 172.21.15.37:0/673373195 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf59272640 1 -- 172.21.15.37:0/673373195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df801550 msgr2=0x5614df7fef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf59272640 1 --2- 172.21.15.37:0/673373195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df801550 0x5614df7fef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf59272640 1 -- 172.21.15.37:0/673373195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f7fd0 msgr2=0x5614df7fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf59272640 1 --2- 172.21.15.37:0/673373195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f7fd0 0x5614df7fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf59272640 1 -- 172.21.15.37:0/673373195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614df53d620 con 0x5614df7f8d40 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf59272640 1 --2- 172.21.15.37:0/673373195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f8d40 0x5614df7fea50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf440cc4d0 tx=0x7faf4409ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15aa21db528173d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf3d7fa640 1 -- 172.21.15.37:0/673373195 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4401c070 con 0x5614df7f8d40 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.687+0000 7faf3d7fa640 1 -- 172.21.15.37:0/673373195 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7faf44054d50 con 0x5614df7f8d40 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf5e98b740 1 -- 172.21.15.37:0/673373195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf50034440 con 0x5614df7f8d40 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf3d7fa640 1 -- 172.21.15.37:0/673373195 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf4409b3e0 con 0x5614df7f8d40 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf5e98b740 10 client.25855 fetch_fsmap learned FSMap version 46 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf5e98b740 1 -- 172.21.15.37:0/673373195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5614df810500 con 0x5614df7f8d40 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf3d7fa640 1 -- 172.21.15.37:0/673373195 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf50034440 con 0x5614df7f8d40 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf3d7fa640 1 -- 172.21.15.37:0/673373195 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 46) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faf440a36f0 con 0x5614df7f8d40 2024-03-26T06:34:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf5e98b740 10 client.25855 fetch_fsmap finished waiting for FSMap version 46 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf5e98b740 10 client.25855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf5e98b740 20 client.25855 populate_metadata read hostname 'smithi037' 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf5e98b740 1 --2- 172.21.15.37:0/673373195 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5614df8119c0 0x5614df813dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf5e98b740 4 client.25855 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf5e98b740 1 -- 172.21.15.37:0/673373195 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614df769b30 con 0x5614df8119c0 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.688+0000 7faf59a73640 1 --2- 172.21.15.37:0/673373195 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5614df8119c0 0x5614df813dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf59a73640 1 --2- 172.21.15.37:0/673373195 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5614df8119c0 0x5614df813dd0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ba063c82f88504 server_cookie=9d23eb964df289bc in_seq=0 out_seq=0 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf3d7fa640 10 client.25855 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf3d7fa640 1 -- 172.21.15.37:0/673373195 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5614df769b30 con 0x5614df8119c0 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf3d7fa640 10 client.25855 handle_command_reply: tid=0 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf5e98b740 1 client.25855 shutdown 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf5e98b740 1 -- 172.21.15.37:0/673373195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f8d40 msgr2=0x5614df7fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf5e98b740 1 --2- 172.21.15.37:0/673373195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f8d40 0x5614df7fea50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf440cc4d0 tx=0x7faf4409ae00 comp rx=0 tx=0).stop 2024-03-26T06:34:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf5e98b740 1 -- 172.21.15.37:0/673373195 shutdown_connections 2024-03-26T06:34:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf5e98b740 1 --2- 172.21.15.37:0/673373195 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5614df8119c0 0x5614df813dd0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf5e98b740 1 --2- 172.21.15.37:0/673373195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df801550 0x5614df7fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf5e98b740 1 --2- 172.21.15.37:0/673373195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f8d40 0x5614df7fea50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf3d7fa640 0 client.25855 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.689+0000 7faf5e98b740 1 --2- 172.21.15.37:0/673373195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f7fd0 0x5614df7fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.690+0000 7faf5e98b740 1 -- 172.21.15.37:0/673373195 >> 172.21.15.37:0/673373195 conn(0x5614df7bf5b0 msgr2=0x5614df7f4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.690+0000 7faf5e98b740 1 -- 172.21.15.37:0/673373195 shutdown_connections 2024-03-26T06:34:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.690+0000 7faf5e98b740 1 -- 172.21.15.37:0/673373195 wait complete. 2024-03-26T06:34:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.690+0000 7faf5e98b740 20 client.25855 trim_cache size 0 max 16384 2024-03-26T06:34:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.704+0000 7faf5e98b740 1 Processor -- start 2024-03-26T06:34:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.704+0000 7faf5e98b740 1 -- start start 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f7fd0 0x5614df800c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f8d40 0x5614df7fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df801550 0x5614df7fd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf5e98b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541aed70 con 0x5614df7f8d40 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf5e98b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541b06f0 con 0x5614df801550 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf5e98b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541af3d0 con 0x5614df7f7fd0 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf59272640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f8d40 0x5614df7fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf59272640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f8d40 0x5614df7fce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40440/0 (socket says 172.21.15.37:40440) 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf5a274640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df801550 0x5614df7fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf59272640 1 -- 172.21.15.37:0/3545393501 learned_addr learned my addr 172.21.15.37:0/3545393501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf59a73640 1 --2- 172.21.15.37:0/3545393501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f7fd0 0x5614df800c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf59272640 1 -- 172.21.15.37:0/3545393501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f7fd0 msgr2=0x5614df800c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf59272640 1 --2- 172.21.15.37:0/3545393501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f7fd0 0x5614df800c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf59272640 1 -- 172.21.15.37:0/3545393501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df801550 msgr2=0x5614df7fd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf59272640 1 --2- 172.21.15.37:0/3545393501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df801550 0x5614df7fd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf59272640 1 -- 172.21.15.37:0/3545393501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614df7f72a0 con 0x5614df7f8d40 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf5a274640 1 --2- 172.21.15.37:0/3545393501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df801550 0x5614df7fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf59a73640 1 --2- 172.21.15.37:0/3545393501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f7fd0 0x5614df800c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.705+0000 7faf59272640 1 --2- 172.21.15.37:0/3545393501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f8d40 0x5614df7fce20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7faf4409b370 tx=0x7faf440096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da62c463ab256f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf3f7fe640 1 -- 172.21.15.37:0/3545393501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4401c070 con 0x5614df7f8d40 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf3f7fe640 1 -- 172.21.15.37:0/3545393501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf440a3b80 con 0x5614df7f8d40 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf3f7fe640 1 -- 172.21.15.37:0/3545393501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf440a3890 con 0x5614df7f8d40 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf5e98b740 1 -- 172.21.15.37:0/3545393501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f8d40 msgr2=0x5614df7fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf5e98b740 1 --2- 172.21.15.37:0/3545393501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f8d40 0x5614df7fce20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7faf4409b370 tx=0x7faf440096e0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf5e98b740 1 -- 172.21.15.37:0/3545393501 shutdown_connections 2024-03-26T06:34:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf5e98b740 1 --2- 172.21.15.37:0/3545393501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df801550 0x5614df7fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf5e98b740 1 --2- 172.21.15.37:0/3545393501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df7f8d40 0x5614df7fce20 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf5e98b740 1 --2- 172.21.15.37:0/3545393501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f7fd0 0x5614df800c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf5e98b740 1 -- 172.21.15.37:0/3545393501 >> 172.21.15.37:0/3545393501 conn(0x5614df7bf5b0 msgr2=0x5614df7f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf5e98b740 1 -- 172.21.15.37:0/3545393501 shutdown_connections 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.706+0000 7faf5e98b740 1 -- 172.21.15.37:0/3545393501 wait complete. 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5e98b740 1 -- start start 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5e98b740 1 Processor -- start 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f7fd0 0x5614df807050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 0x5614df807590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5e98b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df801550 0x5614df807ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5e98b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541b0090 con 0x5614df801550 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5e98b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541afa30 con 0x5614df7f8d40 2024-03-26T06:34:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5e98b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541ad030 con 0x5614df7f7fd0 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5e98b740 10 client.0 _ll_register_callbacks cb 0x7faf4c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5a274640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df801550 0x5614df807ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf59272640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 0x5614df807590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5a274640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df801550 0x5614df807ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40444/0 (socket says 172.21.15.37:40444) 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf5a274640 1 -- 172.21.15.37:0/161749019 learned_addr learned my addr 172.21.15.37:0/161749019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf59272640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 0x5614df807590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:54418/0 (socket says 172.21.15.37:54418) 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.707+0000 7faf59a73640 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f7fd0 0x5614df807050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf5a274640 1 -- 172.21.15.37:0/161749019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f7fd0 msgr2=0x5614df807050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf5a274640 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f7fd0 0x5614df807050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf5a274640 1 -- 172.21.15.37:0/161749019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 msgr2=0x5614df807590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf5a274640 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 0x5614df807590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf5a274640 1 -- 172.21.15.37:0/161749019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614df867f80 con 0x5614df801550 2024-03-26T06:34:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf5a274640 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df801550 0x5614df807ad0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7faf4c01f7c0 tx=0x7faf4c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb4faa2e695a422 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf3d7fa640 1 -- 172.21.15.37:0/161749019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c01d600 con 0x5614df801550 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf3d7fa640 1 -- 172.21.15.37:0/161749019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7faf4c017070 con 0x5614df801550 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf5e98b740 1 -- 172.21.15.37:0/161749019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf440a3b80 con 0x5614df801550 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf3d7fa640 1 -- 172.21.15.37:0/161749019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf4c01d7a0 con 0x5614df801550 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf5e98b740 10 client.16476 fetch_fsmap learned FSMap version 46 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.708+0000 7faf5e98b740 1 -- 172.21.15.37:0/161749019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5614df8692f0 con 0x5614df801550 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.709+0000 7faf3d7fa640 1 -- 172.21.15.37:0/161749019 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf440a3b80 con 0x5614df801550 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.709+0000 7faf3d7fa640 1 -- 172.21.15.37:0/161749019 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 46) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faf4c00b040 con 0x5614df801550 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.709+0000 7faf5e98b740 10 client.16476 fetch_fsmap finished waiting for FSMap version 46 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.709+0000 7faf5e98b740 10 client.16476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.709+0000 7faf5e98b740 20 client.16476 populate_metadata read hostname 'smithi037' 2024-03-26T06:34:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.709+0000 7faf5e98b740 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5614df869a50 0x5614df86be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:34:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.709+0000 7faf5e98b740 4 client.16476 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:34:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.709+0000 7faf5e98b740 1 -- 172.21.15.37:0/161749019 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5614df763e80 con 0x5614df869a50 2024-03-26T06:34:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.709+0000 7faf59a73640 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5614df869a50 0x5614df86be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:34:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.709+0000 7faf3d7fa640 10 client.16476 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.709+0000 7faf59a73640 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5614df869a50 0x5614df86be70 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce1f94363dfbdd87 server_cookie=8064b7bb27241c4c in_seq=0 out_seq=0 2024-03-26T06:34:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.736+0000 7faf3d7fa640 1 -- 172.21.15.37:0/161749019 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5614df763e80 con 0x5614df869a50 2024-03-26T06:34:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.736+0000 7faf3d7fa640 10 client.16476 handle_command_reply: tid=0 2024-03-26T06:34:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.736+0000 7faf5e98b740 1 client.16476 shutdown 2024-03-26T06:34:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.736+0000 7faf5e98b740 1 -- 172.21.15.37:0/161749019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df801550 msgr2=0x5614df807ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.737+0000 7faf5e98b740 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df801550 0x5614df807ad0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7faf4c01f7c0 tx=0x7faf4c0095c0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.737+0000 7faf5e98b740 1 -- 172.21.15.37:0/161749019 shutdown_connections 2024-03-26T06:34:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.737+0000 7faf5e98b740 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5614df869a50 0x5614df86be70 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.737+0000 7faf3d7fa640 0 client.16476 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:34:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.737+0000 7faf5e98b740 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614df801550 0x5614df807ad0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.737+0000 7faf5e98b740 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614df7f8d40 0x5614df807590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.737+0000 7faf5e98b740 1 --2- 172.21.15.37:0/161749019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614df7f7fd0 0x5614df807050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.737+0000 7faf5e98b740 1 -- 172.21.15.37:0/161749019 >> 172.21.15.37:0/161749019 conn(0x5614df7bf5b0 msgr2=0x5614df7fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.737+0000 7faf5e98b740 1 -- 172.21.15.37:0/161749019 shutdown_connections 2024-03-26T06:34:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.737+0000 7faf5e98b740 1 -- 172.21.15.37:0/161749019 wait complete. 2024-03-26T06:34:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.737+0000 7faf5e98b740 20 client.16476 trim_cache size 0 max 16384 2024-03-26T06:34:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:34:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:34:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:34:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:34:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 -- 172.21.15.37:0/3019670285 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7faf400a2a40 msgr2=0x7faf400a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7faf400a2a40 0x7faf400a4f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x5614df7f3e50 tx=0x7faf4c00e500 comp rx=0 tx=0).stop 2024-03-26T06:34:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 -- 172.21.15.37:0/3019670285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf54069310 msgr2=0x7faf54100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:34:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf54069310 0x7faf54100a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf4400e9f0 tx=0x7faf4400eeb0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 -- 172.21.15.37:0/3019670285 shutdown_connections 2024-03-26T06:34:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7faf400a2a40 0x7faf400a4f00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf541073e0 0x7faf54100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf54069310 0x7faf54100a50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 --2- 172.21.15.37:0/3019670285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54068970 0x7faf54100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:34:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 -- 172.21.15.37:0/3019670285 >> 172.21.15.37:0/3019670285 conn(0x7faf54077040 msgr2=0x7faf540ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:34:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 -- 172.21.15.37:0/3019670285 shutdown_connections 2024-03-26T06:34:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:34:51.738+0000 7faf5bcd7640 1 -- 172.21.15.37:0/3019670285 wait complete. 2024-03-26T06:34:52.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:51 smithi066 ceph-mon[36504]: pgmap v371: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-03-26T06:34:52.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:51 smithi149 ceph-mon[36681]: pgmap v371: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-03-26T06:34:52.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:51 smithi037 ceph-mon[31974]: pgmap v371: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-03-26T06:34:52.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 447258 3.96 MB/sec execute 507 sec latency 22.490 ms 2024-03-26T06:34:53.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 448480 3.96 MB/sec execute 508 sec latency 18.046 ms 2024-03-26T06:34:54.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:53 smithi066 ceph-mon[36504]: pgmap v372: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 879 KiB/s wr, 74 op/s 2024-03-26T06:34:54.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:53 smithi149 ceph-mon[36681]: pgmap v372: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 879 KiB/s wr, 74 op/s 2024-03-26T06:34:54.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:53 smithi037 ceph-mon[31974]: pgmap v372: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 879 KiB/s wr, 74 op/s 2024-03-26T06:34:54.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 450316 3.97 MB/sec execute 509 sec latency 14.598 ms 2024-03-26T06:34:55.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 451161 3.97 MB/sec execute 510 sec latency 243.084 ms 2024-03-26T06:34:56.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:55 smithi066 ceph-mon[36504]: pgmap v373: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-03-26T06:34:56.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:55 smithi149 ceph-mon[36681]: pgmap v373: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-03-26T06:34:56.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:55 smithi037 ceph-mon[31974]: pgmap v373: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-03-26T06:34:56.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 451488 3.97 MB/sec execute 511 sec latency 1031.986 ms 2024-03-26T06:34:57.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 453701 3.98 MB/sec execute 512 sec latency 20.602 ms 2024-03-26T06:34:58.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:34:57 smithi066 ceph-mon[36504]: pgmap v374: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:34:58.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:34:57 smithi149 ceph-mon[36681]: pgmap v374: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:34:58.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:34:57 smithi037 ceph-mon[31974]: pgmap v374: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:34:58.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 454760 3.99 MB/sec execute 513 sec latency 177.583 ms 2024-03-26T06:34:59.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 454760 3.98 MB/sec execute 514 sec latency 1177.671 ms 2024-03-26T06:35:00.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:00 smithi066 ceph-mon[36504]: pgmap v375: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:35:00.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:00 smithi149 ceph-mon[36681]: pgmap v375: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:35:00.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:00 smithi037 ceph-mon[31974]: pgmap v375: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-26T06:35:00.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 455053 3.97 MB/sec execute 515 sec latency 1893.248 ms 2024-03-26T06:35:01.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 456480 3.98 MB/sec execute 516 sec latency 27.111 ms 2024-03-26T06:35:02.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:02 smithi066 ceph-mon[36504]: pgmap v376: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-03-26T06:35:02.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:02 smithi149 ceph-mon[36681]: pgmap v376: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-03-26T06:35:02.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:02 smithi037 ceph-mon[31974]: pgmap v376: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-03-26T06:35:02.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 458049 3.99 MB/sec execute 517 sec latency 17.079 ms 2024-03-26T06:35:03.530 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459008 4.03 MB/sec execute 518 sec latency 54.204 ms 2024-03-26T06:35:04.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:04 smithi066 ceph-mon[36504]: pgmap v377: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-03-26T06:35:04.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:04 smithi149 ceph-mon[36681]: pgmap v377: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-03-26T06:35:04.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:04 smithi037 ceph-mon[31974]: pgmap v377: 97 pgs: 97 active+clean; 162 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-03-26T06:35:04.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 4.03 MB/sec execute 519 sec latency 493.356 ms 2024-03-26T06:35:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.838+0000 7fb52e873640 1 -- 172.21.15.37:0/3475660508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb528100de0 msgr2=0x7fb52810d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.838+0000 7fb52e873640 1 --2- 172.21.15.37:0/3475660508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb528100de0 0x7fb52810d020 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb520009ca0 tx=0x7fb52002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.839+0000 7fb52e873640 1 -- 172.21.15.37:0/3475660508 shutdown_connections 2024-03-26T06:35:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.839+0000 7fb52e873640 1 --2- 172.21.15.37:0/3475660508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb528100de0 0x7fb52810d020 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.839+0000 7fb52e873640 1 --2- 172.21.15.37:0/3475660508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb528100440 0x7fb5281008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.839+0000 7fb52e873640 1 --2- 172.21.15.37:0/3475660508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb528105170 0x7fb528105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.839+0000 7fb52e873640 1 -- 172.21.15.37:0/3475660508 >> 172.21.15.37:0/3475660508 conn(0x7fb528076fc0 msgr2=0x7fb5280773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.839+0000 7fb52e873640 1 -- 172.21.15.37:0/3475660508 shutdown_connections 2024-03-26T06:35:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.839+0000 7fb52e873640 1 -- 172.21.15.37:0/3475660508 wait complete. 2024-03-26T06:35:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.839+0000 7fb52e873640 1 Processor -- start 2024-03-26T06:35:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.839+0000 7fb52e873640 1 -- start start 2024-03-26T06:35:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb52e873640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb528100440 0x7fb528198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb52e873640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb528100de0 0x7fb528198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb52e873640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb528105170 0x7fb5281a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb52e873640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb528110620 con 0x7fb528100de0 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb52e873640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281104a0 con 0x7fb528105170 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb52e873640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281107a0 con 0x7fb528100440 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb528100de0 0x7fb528198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb528100de0 0x7fb528198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57300/0 (socket says 172.21.15.37:57300) 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb5277fe640 1 -- 172.21.15.37:0/2073334421 learned_addr learned my addr 172.21.15.37:0/2073334421 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb52ce10640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb528105170 0x7fb5281a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb527fff640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb528100440 0x7fb528198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb5277fe640 1 -- 172.21.15.37:0/2073334421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb528100440 msgr2=0x7fb528198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb5277fe640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb528100440 0x7fb528198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb5277fe640 1 -- 172.21.15.37:0/2073334421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb528105170 msgr2=0x7fb5281a39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb5277fe640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb528105170 0x7fb5281a39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb5277fe640 1 -- 172.21.15.37:0/2073334421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5281a3ee0 con 0x7fb528100de0 2024-03-26T06:35:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.840+0000 7fb5277fe640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb528100de0 0x7fb528198e60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb51000d950 tx=0x7fb51000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.841+0000 7fb5257fa640 1 -- 172.21.15.37:0/2073334421 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510017070 con 0x7fb528100de0 2024-03-26T06:35:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.841+0000 7fb5257fa640 1 -- 172.21.15.37:0/2073334421 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb510009c30 con 0x7fb528100de0 2024-03-26T06:35:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.841+0000 7fb52e873640 1 -- 172.21.15.37:0/2073334421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5281a40b0 con 0x7fb528100de0 2024-03-26T06:35:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.841+0000 7fb5257fa640 1 -- 172.21.15.37:0/2073334421 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51000f6b0 con 0x7fb528100de0 2024-03-26T06:35:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.841+0000 7fb52e873640 1 -- 172.21.15.37:0/2073334421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5281a45d0 con 0x7fb528100de0 2024-03-26T06:35:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.843+0000 7fb5257fa640 1 -- 172.21.15.37:0/2073334421 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb5100109d0 con 0x7fb528100de0 2024-03-26T06:35:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.843+0000 7fb531451740 1 Processor -- start 2024-03-26T06:35:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.843+0000 7fb531451740 1 -- start start 2024-03-26T06:35:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb531451740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca37d40 0x55739ca36a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb531451740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca37450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb531451740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca40550 0x55739ca42940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb531451740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739c99b5f0 con 0x55739ca36fd0 2024-03-26T06:35:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb531451740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739c944930 con 0x55739ca37d40 2024-03-26T06:35:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb531451740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739c997690 con 0x55739ca40550 2024-03-26T06:35:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb527fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca37450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb527fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca37450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57306/0 (socket says 172.21.15.37:57306) 2024-03-26T06:35:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb527fff640 1 -- 172.21.15.37:0/567758535 learned_addr learned my addr 172.21.15.37:0/567758535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb52ce10640 1 --2- 172.21.15.37:0/567758535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca37d40 0x55739ca36a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb5277fe640 1 --2- 172.21.15.37:0/567758535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca40550 0x55739ca42940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb5257fa640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5140a2a10 0x7fb5140a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb52ce10640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5140a2a10 0x7fb5140a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb527fff640 1 -- 172.21.15.37:0/567758535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca40550 msgr2=0x55739ca42940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb527fff640 1 --2- 172.21.15.37:0/567758535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca40550 0x55739ca42940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb527fff640 1 -- 172.21.15.37:0/567758535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca37d40 msgr2=0x55739ca36a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb527fff640 1 --2- 172.21.15.37:0/567758535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca37d40 0x55739ca36a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb527fff640 1 -- 172.21.15.37:0/567758535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55739c6b0210 con 0x55739ca36fd0 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb5257fa640 1 -- 172.21.15.37:0/2073334421 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb5100cead0 con 0x7fb528100de0 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb527fff640 1 --2- 172.21.15.37:0/567758535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca37450 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb51800bc50 tx=0x7fb5180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1fb38da60da8a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb52ce10640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5140a2a10 0x7fb5140a4ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb520004af0 tx=0x7fb52002c0f0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.844+0000 7fb51e7fc640 1 -- 172.21.15.37:0/567758535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51801c480 con 0x55739ca36fd0 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.845+0000 7fb51e7fc640 1 -- 172.21.15.37:0/567758535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb51801cb00 con 0x55739ca36fd0 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.845+0000 7fb51e7fc640 1 -- 172.21.15.37:0/567758535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518011b90 con 0x55739ca36fd0 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.845+0000 7fb531451740 1 -- 172.21.15.37:0/567758535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 msgr2=0x55739ca37450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:04.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.845+0000 7fb531451740 1 --2- 172.21.15.37:0/567758535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca37450 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb51800bc50 tx=0x7fb5180107a0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.845+0000 7fb531451740 1 -- 172.21.15.37:0/567758535 shutdown_connections 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.845+0000 7fb531451740 1 --2- 172.21.15.37:0/567758535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca40550 0x55739ca42940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.845+0000 7fb531451740 1 --2- 172.21.15.37:0/567758535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca37450 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.845+0000 7fb531451740 1 --2- 172.21.15.37:0/567758535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca37d40 0x55739ca36a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.845+0000 7fb531451740 1 -- 172.21.15.37:0/567758535 >> 172.21.15.37:0/567758535 conn(0x55739c9fe5b0 msgr2=0x55739ca31cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.845+0000 7fb531451740 1 -- 172.21.15.37:0/567758535 shutdown_connections 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.845+0000 7fb531451740 1 -- 172.21.15.37:0/567758535 wait complete. 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb531451740 1 -- start start 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb531451740 1 Processor -- start 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb531451740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca36fd0 0x55739ca3d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb531451740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca3dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb531451740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca40550 0x55739ca3e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb531451740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ca43280 con 0x55739ca37d40 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb531451740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739c957d90 con 0x55739ca40550 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb531451740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739ca43400 con 0x55739ca36fd0 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb531451740 10 client.0 _ll_register_callbacks cb 0x55739c6afa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca3dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca3dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57314/0 (socket says 172.21.15.37:57314) 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb5277fe640 1 -- 172.21.15.37:0/454411467 learned_addr learned my addr 172.21.15.37:0/454411467 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb527fff640 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca36fd0 0x55739ca3d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.846+0000 7fb52ce10640 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca40550 0x55739ca3e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb5277fe640 1 -- 172.21.15.37:0/454411467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca36fd0 msgr2=0x55739ca3d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb5277fe640 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca36fd0 0x55739ca3d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb5277fe640 1 -- 172.21.15.37:0/454411467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca40550 msgr2=0x55739ca3e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb5277fe640 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca40550 0x55739ca3e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb5277fe640 1 -- 172.21.15.37:0/454411467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55739c77c620 con 0x55739ca37d40 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb527fff640 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca36fd0 0x55739ca3d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb5277fe640 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca3dbc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb5100975e0 tx=0x7fb51000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54928d2a709aeb22 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb5077fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb5077fe640 1 -- 172.21.15.37:0/454411467 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510020040 con 0x55739ca37d40 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb531451740 1 -- 172.21.15.37:0/454411467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb51801cb00 con 0x55739ca37d40 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb5077fe640 1 -- 172.21.15.37:0/454411467 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb5100a33e0 con 0x55739ca37d40 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb5077fe640 1 -- 172.21.15.37:0/454411467 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb51009aae0 con 0x55739ca37d40 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb531451740 10 client.16488 fetch_fsmap learned FSMap version 47 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb531451740 1 -- 172.21.15.37:0/454411467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55739ca4f6c0 con 0x55739ca37d40 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.847+0000 7fb5077fe640 1 -- 172.21.15.37:0/454411467 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb51801cb00 con 0x55739ca37d40 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.848+0000 7fb5077fe640 1 -- 172.21.15.37:0/454411467 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb51009a5d0 con 0x55739ca37d40 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.848+0000 7fb531451740 10 client.16488 fetch_fsmap finished waiting for FSMap version 47 2024-03-26T06:35:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.848+0000 7fb531451740 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.848+0000 7fb531451740 20 client.16488 populate_metadata read hostname 'smithi037' 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.848+0000 7fb531451740 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55739ca50f40 0x55739ca533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.848+0000 7fb531451740 4 client.16488 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.848+0000 7fb531451740 1 -- 172.21.15.37:0/454411467 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55739c9a8b30 con 0x55739ca50f40 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.848+0000 7fb527fff640 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55739ca50f40 0x55739ca533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.849+0000 7fb527fff640 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55739ca50f40 0x55739ca533b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eae6c217fc1bd7d server_cookie=2b0505a881598ba2 in_seq=0 out_seq=0 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.849+0000 7fb5077fe640 10 client.16488 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.849+0000 7fb5077fe640 1 -- 172.21.15.37:0/454411467 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55739c9a8b30 con 0x55739ca50f40 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.849+0000 7fb5077fe640 10 client.16488 handle_command_reply: tid=0 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.849+0000 7fb531451740 1 client.16488 shutdown 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb531451740 1 -- 172.21.15.37:0/454411467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 msgr2=0x55739ca3dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb531451740 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca3dbc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb5100975e0 tx=0x7fb51000bf60 comp rx=0 tx=0).stop 2024-03-26T06:35:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb531451740 1 -- 172.21.15.37:0/454411467 shutdown_connections 2024-03-26T06:35:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb531451740 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55739ca50f40 0x55739ca533b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb531451740 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca40550 0x55739ca3e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb531451740 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca3dbc0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb531451740 1 --2- 172.21.15.37:0/454411467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca36fd0 0x55739ca3d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb5077fe640 0 client.16488 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb531451740 1 -- 172.21.15.37:0/454411467 >> 172.21.15.37:0/454411467 conn(0x55739c9fe5b0 msgr2=0x55739ca33a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb531451740 1 -- 172.21.15.37:0/454411467 shutdown_connections 2024-03-26T06:35:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb531451740 1 -- 172.21.15.37:0/454411467 wait complete. 2024-03-26T06:35:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.850+0000 7fb531451740 20 client.16488 trim_cache size 0 max 16384 2024-03-26T06:35:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.863+0000 7fb531451740 1 Processor -- start 2024-03-26T06:35:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.863+0000 7fb531451740 1 -- start start 2024-03-26T06:35:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.863+0000 7fb531451740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca3fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.863+0000 7fb531451740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca37d40 0x55739ca47670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.863+0000 7fb531451740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca40550 0x55739ca47bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.863+0000 7fb531451740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281accd0 con 0x55739ca36fd0 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.863+0000 7fb531451740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281ae680 con 0x55739ca40550 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.863+0000 7fb531451740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281ad330 con 0x55739ca37d40 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.863+0000 7fb527fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca3fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.863+0000 7fb527fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca3fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57324/0 (socket says 172.21.15.37:57324) 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.863+0000 7fb527fff640 1 -- 172.21.15.37:0/3575209693 learned_addr learned my addr 172.21.15.37:0/3575209693 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb5277fe640 1 --2- 172.21.15.37:0/3575209693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca37d40 0x55739ca47670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb52ce10640 1 --2- 172.21.15.37:0/3575209693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca40550 0x55739ca47bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb527fff640 1 -- 172.21.15.37:0/3575209693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca37d40 msgr2=0x55739ca47670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb527fff640 1 --2- 172.21.15.37:0/3575209693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca37d40 0x55739ca47670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb527fff640 1 -- 172.21.15.37:0/3575209693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca40550 msgr2=0x55739ca47bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb527fff640 1 --2- 172.21.15.37:0/3575209693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca40550 0x55739ca47bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb527fff640 1 -- 172.21.15.37:0/3575209693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55739ca585e0 con 0x55739ca36fd0 2024-03-26T06:35:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb5277fe640 1 --2- 172.21.15.37:0/3575209693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca37d40 0x55739ca47670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb52ce10640 1 --2- 172.21.15.37:0/3575209693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca40550 0x55739ca47bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb527fff640 1 --2- 172.21.15.37:0/3575209693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca3fdb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb51800bb70 tx=0x7fb518010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c30338bd04fe909 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb51dffb640 1 -- 172.21.15.37:0/3575209693 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518004020 con 0x55739ca36fd0 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb51dffb640 1 -- 172.21.15.37:0/3575209693 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb518005520 con 0x55739ca36fd0 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.864+0000 7fb51dffb640 1 -- 172.21.15.37:0/3575209693 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51801c890 con 0x55739ca36fd0 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.865+0000 7fb531451740 1 -- 172.21.15.37:0/3575209693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 msgr2=0x55739ca3fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.865+0000 7fb531451740 1 --2- 172.21.15.37:0/3575209693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca3fdb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb51800bb70 tx=0x7fb518010fd0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.865+0000 7fb531451740 1 -- 172.21.15.37:0/3575209693 shutdown_connections 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.865+0000 7fb531451740 1 --2- 172.21.15.37:0/3575209693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca40550 0x55739ca47bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.865+0000 7fb531451740 1 --2- 172.21.15.37:0/3575209693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca37d40 0x55739ca47670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.865+0000 7fb531451740 1 --2- 172.21.15.37:0/3575209693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca36fd0 0x55739ca3fdb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.865+0000 7fb531451740 1 -- 172.21.15.37:0/3575209693 >> 172.21.15.37:0/3575209693 conn(0x55739c9fe5b0 msgr2=0x55739ca427f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.865+0000 7fb531451740 1 -- 172.21.15.37:0/3575209693 shutdown_connections 2024-03-26T06:35:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.865+0000 7fb531451740 1 -- 172.21.15.37:0/3575209693 wait complete. 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.865+0000 7fb531451740 1 -- start start 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.865+0000 7fb531451740 1 Processor -- start 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb531451740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca36fd0 0x55739ca478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb531451740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca43c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb531451740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca40550 0x55739ca44150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb531451740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281ae020 con 0x55739ca37d40 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb531451740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281ad990 con 0x55739ca36fd0 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb531451740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5281aaf90 con 0x55739ca40550 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb531451740 10 client.0 _ll_register_callbacks cb 0x7fb51800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca43c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca43c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57338/0 (socket says 172.21.15.37:57338) 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb5277fe640 1 -- 172.21.15.37:0/3039969962 learned_addr learned my addr 172.21.15.37:0/3039969962 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb527fff640 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca36fd0 0x55739ca478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb52ce10640 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca40550 0x55739ca44150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb5277fe640 1 -- 172.21.15.37:0/3039969962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca40550 msgr2=0x55739ca44150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb5277fe640 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca40550 0x55739ca44150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb5277fe640 1 -- 172.21.15.37:0/3039969962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca36fd0 msgr2=0x55739ca478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb5277fe640 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca36fd0 0x55739ca478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb5277fe640 1 -- 172.21.15.37:0/3039969962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55739ca44790 con 0x55739ca37d40 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb52ce10640 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca40550 0x55739ca44150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb5277fe640 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca43c10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb510097b50 tx=0x7fb510005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343af465c41b905c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb5077fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.866+0000 7fb5077fe640 1 -- 172.21.15.37:0/3039969962 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510020040 con 0x55739ca37d40 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb5077fe640 1 -- 172.21.15.37:0/3039969962 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb5100a35e0 con 0x55739ca37d40 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb531451740 1 -- 172.21.15.37:0/3039969962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb518005520 con 0x55739ca37d40 2024-03-26T06:35:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb5077fe640 1 -- 172.21.15.37:0/3039969962 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb51009af50 con 0x55739ca37d40 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb531451740 10 client.16494 fetch_fsmap learned FSMap version 47 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb531451740 1 -- 172.21.15.37:0/3039969962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55739caa8350 con 0x55739ca37d40 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb5077fe640 1 -- 172.21.15.37:0/3039969962 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb518005520 con 0x55739ca37d40 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb5077fe640 1 -- 172.21.15.37:0/3039969962 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb51009a040 con 0x55739ca37d40 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb531451740 10 client.16494 fetch_fsmap finished waiting for FSMap version 47 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb531451740 10 client.16494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb531451740 20 client.16494 populate_metadata read hostname 'smithi037' 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb531451740 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55739caa89d0 0x55739caaadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb531451740 4 client.16494 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb531451740 1 -- 172.21.15.37:0/3039969962 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55739c9a2e80 con 0x55739caa89d0 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.867+0000 7fb527fff640 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55739caa89d0 0x55739caaadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.868+0000 7fb527fff640 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55739caa89d0 0x55739caaadf0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b04f879fa96f0b1 server_cookie=21f5eb2d286ea824 in_seq=0 out_seq=0 2024-03-26T06:35:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.868+0000 7fb5077fe640 10 client.16494 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb5077fe640 1 -- 172.21.15.37:0/3039969962 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55739c9a2e80 con 0x55739caa89d0 2024-03-26T06:35:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb5077fe640 10 client.16494 handle_command_reply: tid=0 2024-03-26T06:35:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb531451740 1 client.16494 shutdown 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb531451740 1 -- 172.21.15.37:0/3039969962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 msgr2=0x55739ca43c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb531451740 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca43c10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb510097b50 tx=0x7fb510005880 comp rx=0 tx=0).stop 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb531451740 1 -- 172.21.15.37:0/3039969962 shutdown_connections 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb531451740 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55739caa89d0 0x55739caaadf0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb531451740 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55739ca40550 0x55739ca44150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb531451740 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55739ca37d40 0x55739ca43c10 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb531451740 1 --2- 172.21.15.37:0/3039969962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739ca36fd0 0x55739ca478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb531451740 1 -- 172.21.15.37:0/3039969962 >> 172.21.15.37:0/3039969962 conn(0x55739c9fe5b0 msgr2=0x55739ca33010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb5077fe640 0 client.16494 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb531451740 1 -- 172.21.15.37:0/3039969962 shutdown_connections 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.915+0000 7fb531451740 1 -- 172.21.15.37:0/3039969962 wait complete. 2024-03-26T06:35:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb531451740 20 client.16494 trim_cache size 0 max 16384 2024-03-26T06:35:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:35:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:35:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:35:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:35:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 -- 172.21.15.37:0/2073334421 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5140a2a10 msgr2=0x7fb5140a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5140a2a10 0x7fb5140a4ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb520004af0 tx=0x7fb52002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 -- 172.21.15.37:0/2073334421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb528100de0 msgr2=0x7fb528198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb528100de0 0x7fb528198e60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb51000d950 tx=0x7fb51000de10 comp rx=0 tx=0).stop 2024-03-26T06:35:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 -- 172.21.15.37:0/2073334421 shutdown_connections 2024-03-26T06:35:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb5140a2a10 0x7fb5140a4ed0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb528105170 0x7fb5281a39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb528100de0 0x7fb528198e60 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 --2- 172.21.15.37:0/2073334421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb528100440 0x7fb528198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 -- 172.21.15.37:0/2073334421 >> 172.21.15.37:0/2073334421 conn(0x7fb528076fc0 msgr2=0x7fb5280fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 -- 172.21.15.37:0/2073334421 shutdown_connections 2024-03-26T06:35:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:04.916+0000 7fb52e873640 1 -- 172.21.15.37:0/2073334421 wait complete. 2024-03-26T06:35:05.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:05 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:05.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:05 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:05.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:05 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:05.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 4.02 MB/sec execute 520 sec latency 1493.501 ms 2024-03-26T06:35:06.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:06 smithi066 ceph-mon[36504]: pgmap v378: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-03-26T06:35:06.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:06 smithi066 ceph-mon[36504]: osdmap e101: 12 total, 12 up, 12 in 2024-03-26T06:35:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:06 smithi149 ceph-mon[36681]: pgmap v378: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-03-26T06:35:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:06 smithi149 ceph-mon[36681]: osdmap e101: 12 total, 12 up, 12 in 2024-03-26T06:35:06.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:06 smithi037 ceph-mon[31974]: pgmap v378: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-03-26T06:35:06.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:06 smithi037 ceph-mon[31974]: osdmap e101: 12 total, 12 up, 12 in 2024-03-26T06:35:06.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 4.01 MB/sec execute 521 sec latency 2493.599 ms 2024-03-26T06:35:07.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:07 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:07.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:07 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:07.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:07 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:07.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 4.01 MB/sec execute 522 sec latency 3493.711 ms 2024-03-26T06:35:08.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:08 smithi066 ceph-mon[36504]: pgmap v380: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-03-26T06:35:08.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:08 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:08.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:08 smithi149 ceph-mon[36681]: pgmap v380: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-03-26T06:35:08.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:08 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:08.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:08 smithi037 ceph-mon[31974]: pgmap v380: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-03-26T06:35:08.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:08 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:08.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 4.00 MB/sec execute 523 sec latency 4493.837 ms 2024-03-26T06:35:09.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 3.99 MB/sec execute 524 sec latency 5493.928 ms 2024-03-26T06:35:10.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:10 smithi066 ceph-mon[36504]: pgmap v381: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-03-26T06:35:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:10 smithi149 ceph-mon[36681]: pgmap v381: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-03-26T06:35:10.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:10 smithi037 ceph-mon[31974]: pgmap v381: 97 pgs: 97 active+clean; 169 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-03-26T06:35:10.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 3.98 MB/sec execute 525 sec latency 6494.023 ms 2024-03-26T06:35:11.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 3.97 MB/sec execute 526 sec latency 7494.080 ms 2024-03-26T06:35:12.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:12 smithi066 ceph-mon[36504]: pgmap v382: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-03-26T06:35:12.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:12 smithi149 ceph-mon[36681]: pgmap v382: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-03-26T06:35:12.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:12 smithi037 ceph-mon[31974]: pgmap v382: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-03-26T06:35:12.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 3.97 MB/sec execute 527 sec latency 8494.158 ms 2024-03-26T06:35:13.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 3.96 MB/sec execute 528 sec latency 9494.231 ms 2024-03-26T06:35:14.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:14 smithi066 ceph-mon[36504]: pgmap v383: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-03-26T06:35:14.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:14 smithi149 ceph-mon[36681]: pgmap v383: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-03-26T06:35:14.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:14 smithi037 ceph-mon[31974]: pgmap v383: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-03-26T06:35:14.531 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 3.95 MB/sec execute 529 sec latency 10494.354 ms 2024-03-26T06:35:15.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 3.94 MB/sec execute 530 sec latency 11494.441 ms 2024-03-26T06:35:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:15 smithi066 ceph-mon[36504]: osdmap e102: 12 total, 12 up, 12 in 2024-03-26T06:35:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:15 smithi149 ceph-mon[36681]: osdmap e102: 12 total, 12 up, 12 in 2024-03-26T06:35:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:15 smithi037 ceph-mon[31974]: osdmap e102: 12 total, 12 up, 12 in 2024-03-26T06:35:16.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 3.94 MB/sec execute 531 sec latency 12494.527 ms 2024-03-26T06:35:17.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:16 smithi066 ceph-mon[36504]: pgmap v385: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 532 KiB/s wr, 59 op/s 2024-03-26T06:35:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:16 smithi149 ceph-mon[36681]: pgmap v385: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 532 KiB/s wr, 59 op/s 2024-03-26T06:35:17.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:16 smithi037 ceph-mon[31974]: pgmap v385: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 532 KiB/s wr, 59 op/s 2024-03-26T06:35:17.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459987 3.93 MB/sec execute 532 sec latency 13494.597 ms 2024-03-26T06:35:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.004+0000 7fef8fb1b640 1 -- 172.21.15.37:0/929772347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88073fb0 msgr2=0x7fef88111d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.004+0000 7fef8fb1b640 1 --2- 172.21.15.37:0/929772347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88073fb0 0x7fef88111d60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fef78009900 tx=0x7fef7802bcd0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.004+0000 7fef8fb1b640 1 -- 172.21.15.37:0/929772347 shutdown_connections 2024-03-26T06:35:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.004+0000 7fef8fb1b640 1 --2- 172.21.15.37:0/929772347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef881122a0 0x7fef88114690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.004+0000 7fef8fb1b640 1 --2- 172.21.15.37:0/929772347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88073fb0 0x7fef88111d60 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.004+0000 7fef8fb1b640 1 --2- 172.21.15.37:0/929772347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef88073690 0x7fef88073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.004+0000 7fef8fb1b640 1 -- 172.21.15.37:0/929772347 >> 172.21.15.37:0/929772347 conn(0x7fef880fcd80 msgr2=0x7fef880ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.005+0000 7fef8fb1b640 1 -- 172.21.15.37:0/929772347 shutdown_connections 2024-03-26T06:35:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.005+0000 7fef8fb1b640 1 -- 172.21.15.37:0/929772347 wait complete. 2024-03-26T06:35:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.005+0000 7fef8fb1b640 1 Processor -- start 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.005+0000 7fef8fb1b640 1 -- start start 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.005+0000 7fef8fb1b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef88073690 0x7fef8819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.005+0000 7fef8fb1b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88073fb0 0x7fef8819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.005+0000 7fef8fb1b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef881122a0 0x7fef881a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.005+0000 7fef8fb1b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef88117ca0 con 0x7fef88073fb0 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.005+0000 7fef8fb1b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef88117b20 con 0x7fef881122a0 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.005+0000 7fef8fb1b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef88117e20 con 0x7fef88073690 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.005+0000 7fef8d0b6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88073fb0 0x7fef8819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8d8b7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef88073690 0x7fef8819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8d0b6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88073fb0 0x7fef8819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56354/0 (socket says 172.21.15.37:56354) 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8d0b6640 1 -- 172.21.15.37:0/621342832 learned_addr learned my addr 172.21.15.37:0/621342832 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8e0b8640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef881122a0 0x7fef881a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8d0b6640 1 -- 172.21.15.37:0/621342832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef88073690 msgr2=0x7fef8819e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8d0b6640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef88073690 0x7fef8819e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8d0b6640 1 -- 172.21.15.37:0/621342832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef881122a0 msgr2=0x7fef881a29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8d0b6640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef881122a0 0x7fef881a29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8d0b6640 1 -- 172.21.15.37:0/621342832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef881a30f0 con 0x7fef88073fb0 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8d8b7640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef88073690 0x7fef8819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8e0b8640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef881122a0 0x7fef881a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8d0b6640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88073fb0 0x7fef8819e640 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fef78005940 tx=0x7fef7802cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef7effd640 1 -- 172.21.15.37:0/621342832 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78041070 con 0x7fef88073fb0 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef7effd640 1 -- 172.21.15.37:0/621342832 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef78004450 con 0x7fef88073fb0 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8fb1b640 1 -- 172.21.15.37:0/621342832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef881a3380 con 0x7fef88073fb0 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef7effd640 1 -- 172.21.15.37:0/621342832 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78003d80 con 0x7fef88073fb0 2024-03-26T06:35:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.006+0000 7fef8fb1b640 1 -- 172.21.15.37:0/621342832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef881aac70 con 0x7fef88073fb0 2024-03-26T06:35:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.009+0000 7fef7effd640 1 -- 172.21.15.37:0/621342832 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fef78044070 con 0x7fef88073fb0 2024-03-26T06:35:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.009+0000 7fef927d9740 1 Processor -- start 2024-03-26T06:35:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.009+0000 7fef927d9740 1 -- start start 2024-03-26T06:35:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef927d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49bd40 0x55798e49aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef927d9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e49afd0 0x55798e49b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef927d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e4a4550 0x55798e4a6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef927d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798e3ff5f0 con 0x55798e49bd40 2024-03-26T06:35:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef927d9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798e3a8930 con 0x55798e49afd0 2024-03-26T06:35:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef927d9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798e3fb690 con 0x55798e4a4550 2024-03-26T06:35:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8e0b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49bd40 0x55798e49aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef7effd640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fef740a2a10 0x7fef740a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8d8b7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e49afd0 0x55798e49b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8e0b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49bd40 0x55798e49aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56356/0 (socket says 172.21.15.37:56356) 2024-03-26T06:35:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8e0b8640 1 -- 172.21.15.37:0/2507661285 learned_addr learned my addr 172.21.15.37:0/2507661285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8d0b6640 1 --2- 172.21.15.37:0/2507661285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e4a4550 0x55798e4a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8e0b8640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fef740a2a10 0x7fef740a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8d8b7640 1 -- 172.21.15.37:0/2507661285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e4a4550 msgr2=0x55798e4a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8d8b7640 1 --2- 172.21.15.37:0/2507661285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e4a4550 0x55798e4a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef7effd640 1 -- 172.21.15.37:0/621342832 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fef780f0830 con 0x7fef88073fb0 2024-03-26T06:35:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8d8b7640 1 -- 172.21.15.37:0/2507661285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49bd40 msgr2=0x55798e49aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8d8b7640 1 --2- 172.21.15.37:0/2507661285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49bd40 0x55798e49aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8d8b7640 1 -- 172.21.15.37:0/2507661285 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798e114210 con 0x55798e49afd0 2024-03-26T06:35:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8d8b7640 1 --2- 172.21.15.37:0/2507661285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e49afd0 0x55798e49b450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fef84005fd0 tx=0x7fef84011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699fa22d38239c98 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.010+0000 7fef8e0b8640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fef740a2a10 0x7fef740a4ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fef80009fd0 tx=0x7fef80004730 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.011+0000 7fef73fff640 1 -- 172.21.15.37:0/2507661285 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84003c30 con 0x55798e49afd0 2024-03-26T06:35:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.011+0000 7fef73fff640 1 -- 172.21.15.37:0/2507661285 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef8400fe80 con 0x55798e49afd0 2024-03-26T06:35:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.011+0000 7fef73fff640 1 -- 172.21.15.37:0/2507661285 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840193e0 con 0x55798e49afd0 2024-03-26T06:35:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.011+0000 7fef927d9740 1 -- 172.21.15.37:0/2507661285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e49afd0 msgr2=0x55798e49b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.011+0000 7fef927d9740 1 --2- 172.21.15.37:0/2507661285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e49afd0 0x55798e49b450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fef84005fd0 tx=0x7fef84011c70 comp rx=0 tx=0).stop 2024-03-26T06:35:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.011+0000 7fef927d9740 1 -- 172.21.15.37:0/2507661285 shutdown_connections 2024-03-26T06:35:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.011+0000 7fef927d9740 1 --2- 172.21.15.37:0/2507661285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e4a4550 0x55798e4a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.011+0000 7fef927d9740 1 --2- 172.21.15.37:0/2507661285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e49afd0 0x55798e49b450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.011+0000 7fef927d9740 1 --2- 172.21.15.37:0/2507661285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49bd40 0x55798e49aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.011+0000 7fef927d9740 1 -- 172.21.15.37:0/2507661285 >> 172.21.15.37:0/2507661285 conn(0x55798e4625b0 msgr2=0x55798e495cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef927d9740 1 -- 172.21.15.37:0/2507661285 shutdown_connections 2024-03-26T06:35:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef927d9740 1 -- 172.21.15.37:0/2507661285 wait complete. 2024-03-26T06:35:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef927d9740 1 -- start start 2024-03-26T06:35:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef927d9740 1 Processor -- start 2024-03-26T06:35:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef927d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4a1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef927d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49bd40 0x55798e4a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef927d9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e4a4550 0x55798e4a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef927d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798e4a7280 con 0x55798e49afd0 2024-03-26T06:35:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef927d9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798e3bbd90 con 0x55798e4a4550 2024-03-26T06:35:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef927d9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798e4a7400 con 0x55798e49bd40 2024-03-26T06:35:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef927d9740 10 client.0 _ll_register_callbacks cb 0x55798e113a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:35:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef8d8b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef8d8b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56370/0 (socket says 172.21.15.37:56370) 2024-03-26T06:35:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef8d8b7640 1 -- 172.21.15.37:0/393830856 learned_addr learned my addr 172.21.15.37:0/393830856 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.012+0000 7fef8d0b6640 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49bd40 0x55798e4a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef8e0b8640 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e4a4550 0x55798e4a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef8d8b7640 1 -- 172.21.15.37:0/393830856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49bd40 msgr2=0x55798e4a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef8d8b7640 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49bd40 0x55798e4a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef8d8b7640 1 -- 172.21.15.37:0/393830856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e4a4550 msgr2=0x55798e4a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef8d8b7640 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e4a4550 0x55798e4a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef8d8b7640 1 -- 172.21.15.37:0/393830856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55798e1e0620 con 0x55798e49afd0 2024-03-26T06:35:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef8d8b7640 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4a1530 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fef8400fe00 tx=0x7fef84005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1344721d98566ba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef717fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:35:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef717fa640 1 -- 172.21.15.37:0/393830856 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8401f040 con 0x55798e49afd0 2024-03-26T06:35:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef717fa640 1 -- 172.21.15.37:0/393830856 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fef840042d0 con 0x55798e49afd0 2024-03-26T06:35:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef927d9740 1 -- 172.21.15.37:0/393830856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55798e40cb30 con 0x55798e49afd0 2024-03-26T06:35:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef717fa640 1 -- 172.21.15.37:0/393830856 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef84032050 con 0x55798e49afd0 2024-03-26T06:35:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef927d9740 10 client.16506 fetch_fsmap learned FSMap version 50 2024-03-26T06:35:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef927d9740 1 -- 172.21.15.37:0/393830856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55798e4b3510 con 0x55798e49afd0 2024-03-26T06:35:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.013+0000 7fef717fa640 1 -- 172.21.15.37:0/393830856 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55798e40cb30 con 0x55798e49afd0 2024-03-26T06:35:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.014+0000 7fef717fa640 1 -- 172.21.15.37:0/393830856 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fef84018070 con 0x55798e49afd0 2024-03-26T06:35:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.014+0000 7fef927d9740 10 client.16506 fetch_fsmap finished waiting for FSMap version 50 2024-03-26T06:35:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.014+0000 7fef927d9740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:35:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.014+0000 7fef927d9740 20 client.16506 populate_metadata read hostname 'smithi037' 2024-03-26T06:35:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.014+0000 7fef927d9740 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55798e4b48c0 0x55798e4b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.014+0000 7fef927d9740 4 client.16506 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:35:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.014+0000 7fef927d9740 1 -- 172.21.15.37:0/393830856 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55798e406e80 con 0x55798e4b48c0 2024-03-26T06:35:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.014+0000 7fef8d0b6640 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55798e4b48c0 0x55798e4b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.014+0000 7fef717fa640 10 client.16506 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.014+0000 7fef8d0b6640 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55798e4b48c0 0x55798e4b6cb0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc639436a9b6e63 server_cookie=fb026dcb4ce6b2ef in_seq=0 out_seq=0 2024-03-26T06:35:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef717fa640 1 -- 172.21.15.37:0/393830856 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55798e406e80 con 0x55798e4b48c0 2024-03-26T06:35:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef717fa640 10 client.16506 handle_command_reply: tid=0 2024-03-26T06:35:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 1 client.16506 shutdown 2024-03-26T06:35:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 1 -- 172.21.15.37:0/393830856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 msgr2=0x55798e4a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4a1530 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fef8400fe00 tx=0x7fef84005ab0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 1 -- 172.21.15.37:0/393830856 shutdown_connections 2024-03-26T06:35:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55798e4b48c0 0x55798e4b6cb0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e4a4550 0x55798e4a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49bd40 0x55798e4a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 1 --2- 172.21.15.37:0/393830856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4a1530 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef717fa640 0 client.16506 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 1 -- 172.21.15.37:0/393830856 >> 172.21.15.37:0/393830856 conn(0x55798e4625b0 msgr2=0x55798e497a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 1 -- 172.21.15.37:0/393830856 shutdown_connections 2024-03-26T06:35:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 1 -- 172.21.15.37:0/393830856 wait complete. 2024-03-26T06:35:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.015+0000 7fef927d9740 20 client.16506 trim_cache size 0 max 16384 2024-03-26T06:35:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.028+0000 7fef927d9740 1 Processor -- start 2024-03-26T06:35:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.028+0000 7fef927d9740 1 -- start start 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.028+0000 7fef927d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49afd0 0x55798e4a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.028+0000 7fef927d9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e49bd40 0x55798e4ab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.028+0000 7fef927d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e4a4550 0x55798e4abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.028+0000 7fef927d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef881b32c0 con 0x55798e4a4550 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.028+0000 7fef927d9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef881b4ca0 con 0x55798e49bd40 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.028+0000 7fef927d9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef881b3920 con 0x55798e49afd0 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef8e0b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e4a4550 0x55798e4abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef8e0b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e4a4550 0x55798e4abbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56374/0 (socket says 172.21.15.37:56374) 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef8e0b8640 1 -- 172.21.15.37:0/2276760296 learned_addr learned my addr 172.21.15.37:0/2276760296 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef8d8b7640 1 --2- 172.21.15.37:0/2276760296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49afd0 0x55798e4a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef8d0b6640 1 --2- 172.21.15.37:0/2276760296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e49bd40 0x55798e4ab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef8e0b8640 1 -- 172.21.15.37:0/2276760296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49afd0 msgr2=0x55798e4a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef8e0b8640 1 --2- 172.21.15.37:0/2276760296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49afd0 0x55798e4a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef8e0b8640 1 -- 172.21.15.37:0/2276760296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e49bd40 msgr2=0x55798e4ab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef8e0b8640 1 --2- 172.21.15.37:0/2276760296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e49bd40 0x55798e4ab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef8e0b8640 1 -- 172.21.15.37:0/2276760296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798e49a2a0 con 0x55798e4a4550 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef8e0b8640 1 --2- 172.21.15.37:0/2276760296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e4a4550 0x55798e4abbb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fef80012390 tx=0x7fef80009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315ef5d65d408972 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef737fe640 1 -- 172.21.15.37:0/2276760296 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80003c70 con 0x55798e4a4550 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef737fe640 1 -- 172.21.15.37:0/2276760296 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef80003e10 con 0x55798e4a4550 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.029+0000 7fef737fe640 1 -- 172.21.15.37:0/2276760296 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8002b470 con 0x55798e4a4550 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.030+0000 7fef927d9740 1 -- 172.21.15.37:0/2276760296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e4a4550 msgr2=0x55798e4abbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.030+0000 7fef927d9740 1 --2- 172.21.15.37:0/2276760296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e4a4550 0x55798e4abbb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fef80012390 tx=0x7fef80009be0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.030+0000 7fef927d9740 1 -- 172.21.15.37:0/2276760296 shutdown_connections 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.030+0000 7fef927d9740 1 --2- 172.21.15.37:0/2276760296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e4a4550 0x55798e4abbb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.030+0000 7fef927d9740 1 --2- 172.21.15.37:0/2276760296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e49bd40 0x55798e4ab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.030+0000 7fef927d9740 1 --2- 172.21.15.37:0/2276760296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49afd0 0x55798e4a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.030+0000 7fef927d9740 1 -- 172.21.15.37:0/2276760296 >> 172.21.15.37:0/2276760296 conn(0x55798e4625b0 msgr2=0x55798e49c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.030+0000 7fef927d9740 1 -- 172.21.15.37:0/2276760296 shutdown_connections 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.030+0000 7fef927d9740 1 -- 172.21.15.37:0/2276760296 wait complete. 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.030+0000 7fef927d9740 1 -- start start 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.030+0000 7fef927d9740 1 Processor -- start 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef927d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4ab680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef927d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49bd40 0x55798e49d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef927d9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e4a4550 0x55798e49dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef927d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef881b4640 con 0x55798e49afd0 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef927d9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef881b3fb0 con 0x55798e4a4550 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef927d9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef881b1580 con 0x55798e49bd40 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef927d9740 10 client.0 _ll_register_callbacks cb 0x7fef84009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef8d8b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef8d8b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4ab680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56384/0 (socket says 172.21.15.37:56384) 2024-03-26T06:35:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef8d8b7640 1 -- 172.21.15.37:0/3620355469 learned_addr learned my addr 172.21.15.37:0/3620355469 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef8d0b6640 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49bd40 0x55798e49d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef8e0b8640 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e4a4550 0x55798e49dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef8d8b7640 1 -- 172.21.15.37:0/3620355469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49bd40 msgr2=0x55798e49d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef8d8b7640 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49bd40 0x55798e49d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef8d8b7640 1 -- 172.21.15.37:0/3620355469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e4a4550 msgr2=0x55798e49dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef8d8b7640 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e4a4550 0x55798e49dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef8d8b7640 1 -- 172.21.15.37:0/3620355469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55798e4b7210 con 0x55798e49afd0 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef8d8b7640 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4ab680 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fef84011730 tx=0x7fef84005220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d3faa4508bbeeb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef717fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef717fa640 1 -- 172.21.15.37:0/3620355469 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8403b060 con 0x55798e49afd0 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef717fa640 1 -- 172.21.15.37:0/3620355469 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fef84011040 con 0x55798e49afd0 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef927d9740 1 -- 172.21.15.37:0/3620355469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef80003e10 con 0x55798e49afd0 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef717fa640 1 -- 172.21.15.37:0/3620355469 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef84011240 con 0x55798e49afd0 2024-03-26T06:35:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.031+0000 7fef927d9740 10 client.16512 fetch_fsmap learned FSMap version 50 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef927d9740 1 -- 172.21.15.37:0/3620355469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55798e50b2a0 con 0x55798e49afd0 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef717fa640 1 -- 172.21.15.37:0/3620355469 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef80003e10 con 0x55798e49afd0 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef717fa640 1 -- 172.21.15.37:0/3620355469 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fef84018070 con 0x55798e49afd0 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef927d9740 10 client.16512 fetch_fsmap finished waiting for FSMap version 50 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef927d9740 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef927d9740 20 client.16512 populate_metadata read hostname 'smithi037' 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef927d9740 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55798e50b910 0x55798e50dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef927d9740 4 client.16512 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef927d9740 1 -- 172.21.15.37:0/3620355469 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55798e385710 con 0x55798e50b910 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef8d0b6640 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55798e50b910 0x55798e50dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef717fa640 10 client.16512 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.032+0000 7fef8d0b6640 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55798e50b910 0x55798e50dd30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea32312c61afbeba server_cookie=44e884f2e73fd2e0 in_seq=0 out_seq=0 2024-03-26T06:35:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef717fa640 1 -- 172.21.15.37:0/3620355469 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55798e385710 con 0x55798e50b910 2024-03-26T06:35:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef717fa640 10 client.16512 handle_command_reply: tid=0 2024-03-26T06:35:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 1 client.16512 shutdown 2024-03-26T06:35:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 1 -- 172.21.15.37:0/3620355469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 msgr2=0x55798e4ab680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4ab680 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fef84011730 tx=0x7fef84005220 comp rx=0 tx=0).stop 2024-03-26T06:35:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 1 -- 172.21.15.37:0/3620355469 shutdown_connections 2024-03-26T06:35:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55798e50b910 0x55798e50dd30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55798e4a4550 0x55798e49dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55798e49bd40 0x55798e49d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef717fa640 0 client.16512 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 1 --2- 172.21.15.37:0/3620355469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55798e49afd0 0x55798e4ab680 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 1 -- 172.21.15.37:0/3620355469 >> 172.21.15.37:0/3620355469 conn(0x55798e4625b0 msgr2=0x55798e4957f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 1 -- 172.21.15.37:0/3620355469 shutdown_connections 2024-03-26T06:35:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 1 -- 172.21.15.37:0/3620355469 wait complete. 2024-03-26T06:35:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.068+0000 7fef927d9740 20 client.16512 trim_cache size 0 max 16384 2024-03-26T06:35:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:35:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:35:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:35:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:35:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 -- 172.21.15.37:0/621342832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fef740a2a10 msgr2=0x7fef740a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fef740a2a10 0x7fef740a4ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fef80009fd0 tx=0x7fef80004730 comp rx=0 tx=0).stop 2024-03-26T06:35:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 -- 172.21.15.37:0/621342832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88073fb0 msgr2=0x7fef8819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88073fb0 0x7fef8819e640 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fef78005940 tx=0x7fef7802cbd0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 -- 172.21.15.37:0/621342832 shutdown_connections 2024-03-26T06:35:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fef740a2a10 0x7fef740a4ed0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef881122a0 0x7fef881a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88073fb0 0x7fef8819e640 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 --2- 172.21.15.37:0/621342832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef88073690 0x7fef8819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 -- 172.21.15.37:0/621342832 >> 172.21.15.37:0/621342832 conn(0x7fef880fcd80 msgr2=0x7fef880fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 -- 172.21.15.37:0/621342832 shutdown_connections 2024-03-26T06:35:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:18.069+0000 7fef8fb1b640 1 -- 172.21.15.37:0/621342832 wait complete. 2024-03-26T06:35:18.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:17 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:18.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:17 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:35:18.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:17 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:18.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:17 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:18.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:17 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:18.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:17 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:35:18.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:17 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:18.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:17 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:18.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:17 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:18.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:17 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:35:18.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:17 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:18.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:17 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:18.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 460772 3.93 MB/sec execute 533 sec latency 14185.762 ms 2024-03-26T06:35:19.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:18 smithi066 ceph-mon[36504]: pgmap v386: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 528 KiB/s wr, 58 op/s 2024-03-26T06:35:19.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:18 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:19.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:18 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:19.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:18 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:35:19.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:18 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:35:19.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:18 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:18 smithi149 ceph-mon[36681]: pgmap v386: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 528 KiB/s wr, 58 op/s 2024-03-26T06:35:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:18 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:18 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:18 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:35:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:18 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:35:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:18 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:19.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:18 smithi037 ceph-mon[31974]: pgmap v386: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 528 KiB/s wr, 58 op/s 2024-03-26T06:35:19.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:18 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:19.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:18 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:19.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:18 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:35:19.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:18 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:35:19.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:18 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:35:19.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 462438 3.94 MB/sec execute 534 sec latency 13.367 ms 2024-03-26T06:35:20.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 464473 3.95 MB/sec execute 535 sec latency 13.840 ms 2024-03-26T06:35:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:20 smithi066 ceph-mon[36504]: pgmap v387: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 528 KiB/s wr, 58 op/s 2024-03-26T06:35:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:20 smithi149 ceph-mon[36681]: pgmap v387: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 528 KiB/s wr, 58 op/s 2024-03-26T06:35:21.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:20 smithi037 ceph-mon[31974]: pgmap v387: 97 pgs: 97 active+clean; 165 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 528 KiB/s wr, 58 op/s 2024-03-26T06:35:21.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 466156 3.96 MB/sec execute 536 sec latency 11.012 ms 2024-03-26T06:35:22.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:21 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:22.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:21 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:22.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:21 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:22.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 468055 3.97 MB/sec execute 537 sec latency 16.118 ms 2024-03-26T06:35:23.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:22 smithi066 ceph-mon[36504]: pgmap v388: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 866 KiB/s wr, 83 op/s 2024-03-26T06:35:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:22 smithi149 ceph-mon[36681]: pgmap v388: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 866 KiB/s wr, 83 op/s 2024-03-26T06:35:23.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:22 smithi037 ceph-mon[31974]: pgmap v388: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 866 KiB/s wr, 83 op/s 2024-03-26T06:35:23.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 469660 3.98 MB/sec execute 538 sec latency 12.236 ms 2024-03-26T06:35:24.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 471875 4.00 MB/sec execute 539 sec latency 20.773 ms 2024-03-26T06:35:25.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:24 smithi066 ceph-mon[36504]: pgmap v389: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 866 KiB/s wr, 83 op/s 2024-03-26T06:35:25.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:24 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:24 smithi149 ceph-mon[36681]: pgmap v389: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 866 KiB/s wr, 83 op/s 2024-03-26T06:35:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:24 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:25.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:24 smithi037 ceph-mon[31974]: pgmap v389: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 866 KiB/s wr, 83 op/s 2024-03-26T06:35:25.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:24 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:25.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 473261 4.00 MB/sec execute 540 sec latency 14.614 ms 2024-03-26T06:35:26.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:25 smithi066 ceph-mon[36504]: pgmap v390: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-03-26T06:35:26.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:25 smithi149 ceph-mon[36681]: pgmap v390: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-03-26T06:35:26.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:25 smithi037 ceph-mon[31974]: pgmap v390: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-03-26T06:35:26.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 475432 4.02 MB/sec execute 541 sec latency 14.699 ms 2024-03-26T06:35:27.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 476798 4.02 MB/sec execute 542 sec latency 29.212 ms 2024-03-26T06:35:28.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:27 smithi066 ceph-mon[36504]: pgmap v391: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-03-26T06:35:28.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:27 smithi149 ceph-mon[36681]: pgmap v391: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-03-26T06:35:28.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:27 smithi037 ceph-mon[31974]: pgmap v391: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-03-26T06:35:28.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 479113 4.04 MB/sec execute 543 sec latency 13.044 ms 2024-03-26T06:35:29.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 480737 4.05 MB/sec execute 544 sec latency 11.756 ms 2024-03-26T06:35:30.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:29 smithi066 ceph-mon[36504]: pgmap v392: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-03-26T06:35:30.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:29 smithi149 ceph-mon[36681]: pgmap v392: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-03-26T06:35:30.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:29 smithi037 ceph-mon[31974]: pgmap v392: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-03-26T06:35:30.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 483057 4.06 MB/sec execute 545 sec latency 13.754 ms 2024-03-26T06:35:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.161+0000 7f8dffb44640 1 -- 172.21.15.37:0/2376880860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8df8100440 msgr2=0x7f8df81008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.161+0000 7f8dffb44640 1 --2- 172.21.15.37:0/2376880860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8df8100440 0x7f8df81008c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009900 tx=0x7f8de802bc30 comp rx=0 tx=0).stop 2024-03-26T06:35:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.161+0000 7f8dffb44640 1 -- 172.21.15.37:0/2376880860 shutdown_connections 2024-03-26T06:35:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.161+0000 7f8dffb44640 1 --2- 172.21.15.37:0/2376880860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df8100e00 0x7f8df810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.161+0000 7f8dffb44640 1 --2- 172.21.15.37:0/2376880860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8df8100440 0x7f8df81008c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.161+0000 7f8dffb44640 1 --2- 172.21.15.37:0/2376880860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df8105190 0x7f8df8105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.161+0000 7f8dffb44640 1 -- 172.21.15.37:0/2376880860 >> 172.21.15.37:0/2376880860 conn(0x7f8df8077040 msgr2=0x7f8df8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.161+0000 7f8dffb44640 1 -- 172.21.15.37:0/2376880860 shutdown_connections 2024-03-26T06:35:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.161+0000 7f8dffb44640 1 -- 172.21.15.37:0/2376880860 wait complete. 2024-03-26T06:35:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dffb44640 1 Processor -- start 2024-03-26T06:35:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dffb44640 1 -- start start 2024-03-26T06:35:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dffb44640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8df8100440 0x7f8df8101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dffb44640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df8100e00 0x7f8df81023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dffb44640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df8105190 0x7f8df81028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dffb44640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8110650 con 0x7f8df8100440 2024-03-26T06:35:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dffb44640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81104d0 con 0x7f8df8100e00 2024-03-26T06:35:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dffb44640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81107d0 con 0x7f8df8105190 2024-03-26T06:35:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfd8e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8df8100440 0x7f8df8101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfd8e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8df8100440 0x7f8df8101e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57920/0 (socket says 172.21.15.37:57920) 2024-03-26T06:35:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/3662378391 learned_addr learned my addr 172.21.15.37:0/3662378391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfe0e1640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df8105190 0x7f8df81028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfd0df640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df8100e00 0x7f8df81023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/3662378391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df8105190 msgr2=0x7f8df81028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df8105190 0x7f8df81028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/3662378391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df8100e00 msgr2=0x7f8df81023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df8100e00 0x7f8df81023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/3662378391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df81a60b0 con 0x7f8df8100440 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfe0e1640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df8105190 0x7f8df81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.162+0000 7f8dfd0df640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df8100e00 0x7f8df81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.163+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8df8100440 0x7f8df8101e60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8df400ea60 tx=0x7f8df400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.163+0000 7f8deeffd640 1 -- 172.21.15.37:0/3662378391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df400cde0 con 0x7f8df8100440 2024-03-26T06:35:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.163+0000 7f8deeffd640 1 -- 172.21.15.37:0/3662378391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8df40035f0 con 0x7f8df8100440 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.163+0000 7f8dffb44640 1 -- 172.21.15.37:0/3662378391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df81a63a0 con 0x7f8df8100440 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.163+0000 7f8deeffd640 1 -- 172.21.15.37:0/3662378391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4010670 con 0x7f8df8100440 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.163+0000 7f8dffb44640 1 -- 172.21.15.37:0/3662378391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df81a6940 con 0x7f8df8100440 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.165+0000 7f8deeffd640 1 -- 172.21.15.37:0/3662378391 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8df40037a0 con 0x7f8df8100440 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8e027f4740 1 Processor -- start 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8e027f4740 1 -- start start 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8deeffd640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f8de40a2ac0 0x7f8de40a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 0x560006f47a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f47fd0 0x560006f48450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f51550 0x560006f53940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8e027f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560006eac5f0 con 0x560006f51550 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8e027f4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560006e55930 con 0x560006f48d40 2024-03-26T06:35:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8e027f4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560006ea8690 con 0x560006f47fd0 2024-03-26T06:35:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8dfd8e0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 0x560006f47a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8dfe0e1640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f8de40a2ac0 0x7f8de40a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8dfd8e0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 0x560006f47a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:33622/0 (socket says 172.21.15.37:33622) 2024-03-26T06:35:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/835954715 learned_addr learned my addr 172.21.15.37:0/835954715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8dfe0e1640 1 --2- 172.21.15.37:0/835954715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f51550 0x560006f53940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8deeffd640 1 -- 172.21.15.37:0/3662378391 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8df40cf0e0 con 0x7f8df8100440 2024-03-26T06:35:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.166+0000 7f8dfd0df640 1 --2- 172.21.15.37:0/835954715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f47fd0 0x560006f48450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.167+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/835954715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f47fd0 msgr2=0x560006f48450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.167+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/835954715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f47fd0 0x560006f48450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.167+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/835954715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f51550 msgr2=0x560006f53940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.167+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/835954715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f51550 0x560006f53940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.167+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/835954715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560006bc1210 con 0x560006f48d40 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.167+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/835954715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 0x560006f47a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8df409b4e0 tx=0x7f8df4003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bd7427a7341fe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.167+0000 7f8dfe0e1640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f8de40a2ac0 0x7f8de40a4f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8df0005c40 tx=0x7f8df0009e50 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.167+0000 7f8de3fff640 1 -- 172.21.15.37:0/835954715 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df409b140 con 0x560006f48d40 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.167+0000 7f8de3fff640 1 -- 172.21.15.37:0/835954715 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8df40a4e40 con 0x560006f48d40 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.167+0000 7f8de3fff640 1 -- 172.21.15.37:0/835954715 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df409bee0 con 0x560006f48d40 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.168+0000 7f8e027f4740 1 -- 172.21.15.37:0/835954715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 msgr2=0x560006f47a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.168+0000 7f8e027f4740 1 --2- 172.21.15.37:0/835954715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 0x560006f47a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8df409b4e0 tx=0x7f8df4003f30 comp rx=0 tx=0).stop 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.168+0000 7f8e027f4740 1 -- 172.21.15.37:0/835954715 shutdown_connections 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.168+0000 7f8e027f4740 1 --2- 172.21.15.37:0/835954715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f51550 0x560006f53940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.168+0000 7f8e027f4740 1 --2- 172.21.15.37:0/835954715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f47fd0 0x560006f48450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.168+0000 7f8e027f4740 1 --2- 172.21.15.37:0/835954715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 0x560006f47a90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.168+0000 7f8e027f4740 1 -- 172.21.15.37:0/835954715 >> 172.21.15.37:0/835954715 conn(0x560006f0f5b0 msgr2=0x560006f42cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.168+0000 7f8e027f4740 1 -- 172.21.15.37:0/835954715 shutdown_connections 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.168+0000 7f8e027f4740 1 -- 172.21.15.37:0/835954715 wait complete. 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.168+0000 7f8e027f4740 1 -- start start 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.168+0000 7f8e027f4740 1 Processor -- start 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f4e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f48d40 0x560006f4eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f51550 0x560006f4f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8e027f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560006f54280 con 0x560006f47fd0 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8e027f4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560006e68d90 con 0x560006f51550 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8e027f4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560006f54400 con 0x560006f48d40 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8dfd8e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f4e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8e027f4740 10 client.0 _ll_register_callbacks cb 0x560006bc0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:35:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8dfd8e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f4e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57938/0 (socket says 172.21.15.37:57938) 2024-03-26T06:35:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/1966287216 learned_addr learned my addr 172.21.15.37:0/1966287216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8dfe0e1640 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f51550 0x560006f4f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8dfd0df640 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f48d40 0x560006f4eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/1966287216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f48d40 msgr2=0x560006f4eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f48d40 0x560006f4eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/1966287216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f51550 msgr2=0x560006f4f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f51550 0x560006f4f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/1966287216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560006c8d620 con 0x560006f47fd0 2024-03-26T06:35:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.169+0000 7f8dfe0e1640 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f51550 0x560006f4f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:35:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f4e660 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8df400ea30 tx=0x7f8df409b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de2f4f306f3ea05 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8de17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:35:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8de17fa640 1 -- 172.21.15.37:0/1966287216 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df409c910 con 0x560006f47fd0 2024-03-26T06:35:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8e027f4740 1 -- 172.21.15.37:0/1966287216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8df40a4e40 con 0x560006f47fd0 2024-03-26T06:35:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8de17fa640 1 -- 172.21.15.37:0/1966287216 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8df401fde0 con 0x560006f47fd0 2024-03-26T06:35:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8de17fa640 1 -- 172.21.15.37:0/1966287216 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8df409cb10 con 0x560006f47fd0 2024-03-26T06:35:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8e027f4740 10 client.16524 fetch_fsmap learned FSMap version 52 2024-03-26T06:35:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8e027f4740 1 -- 172.21.15.37:0/1966287216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x560006f60640 con 0x560006f47fd0 2024-03-26T06:35:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8de17fa640 1 -- 172.21.15.37:0/1966287216 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8df40a4e40 con 0x560006f47fd0 2024-03-26T06:35:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8de17fa640 1 -- 172.21.15.37:0/1966287216 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 52) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f8df40ccb50 con 0x560006f47fd0 2024-03-26T06:35:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8e027f4740 10 client.16524 fetch_fsmap finished waiting for FSMap version 52 2024-03-26T06:35:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8e027f4740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:35:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8e027f4740 20 client.16524 populate_metadata read hostname 'smithi037' 2024-03-26T06:35:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560006f61990 0x560006f63d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8e027f4740 4 client.16524 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:35:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.170+0000 7f8e027f4740 1 -- 172.21.15.37:0/1966287216 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560006eb9b30 con 0x560006f61990 2024-03-26T06:35:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.171+0000 7f8dfd0df640 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560006f61990 0x560006f63d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.171+0000 7f8de17fa640 10 client.16524 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.171+0000 7f8dfd0df640 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560006f61990 0x560006f63d80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c9fc86a89c2a61 server_cookie=1145be732e9e399c in_seq=0 out_seq=0 2024-03-26T06:35:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8de17fa640 1 -- 172.21.15.37:0/1966287216 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560006eb9b30 con 0x560006f61990 2024-03-26T06:35:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8de17fa640 10 client.16524 handle_command_reply: tid=0 2024-03-26T06:35:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 1 client.16524 shutdown 2024-03-26T06:35:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 1 -- 172.21.15.37:0/1966287216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 msgr2=0x560006f4e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f4e660 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8df400ea30 tx=0x7f8df409b140 comp rx=0 tx=0).stop 2024-03-26T06:35:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 1 -- 172.21.15.37:0/1966287216 shutdown_connections 2024-03-26T06:35:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560006f61990 0x560006f63d80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f51550 0x560006f4f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f48d40 0x560006f4eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8de17fa640 0 client.16524 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1966287216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f4e660 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 1 -- 172.21.15.37:0/1966287216 >> 172.21.15.37:0/1966287216 conn(0x560006f0f5b0 msgr2=0x560006f44a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 1 -- 172.21.15.37:0/1966287216 shutdown_connections 2024-03-26T06:35:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 1 -- 172.21.15.37:0/1966287216 wait complete. 2024-03-26T06:35:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.172+0000 7f8e027f4740 20 client.16524 trim_cache size 0 max 16384 2024-03-26T06:35:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8e027f4740 1 Processor -- start 2024-03-26T06:35:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8e027f4740 1 -- start start 2024-03-26T06:35:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 0x560006f42e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f51550 0x560006f47970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8e027f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81aef70 con 0x560006f47fd0 2024-03-26T06:35:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8e027f4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81b0a10 con 0x560006f48d40 2024-03-26T06:35:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8e027f4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81af5d0 con 0x560006f51550 2024-03-26T06:35:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8dfd8e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8dfd8e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f50d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57946/0 (socket says 172.21.15.37:57946) 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/3951419225 learned_addr learned my addr 172.21.15.37:0/3951419225 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8dfd0df640 1 --2- 172.21.15.37:0/3951419225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 0x560006f42e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8dfe0e1640 1 --2- 172.21.15.37:0/3951419225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f51550 0x560006f47970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/3951419225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f51550 msgr2=0x560006f47970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/3951419225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f51550 0x560006f47970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/3951419225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 msgr2=0x560006f42e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.186+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/3951419225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 0x560006f42e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.187+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/3951419225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560006f3e590 con 0x560006f47fd0 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.187+0000 7f8dfe0e1640 1 --2- 172.21.15.37:0/3951419225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f51550 0x560006f47970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.187+0000 7f8dfd0df640 1 --2- 172.21.15.37:0/3951419225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 0x560006f42e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.187+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/3951419225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f50d90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8df400c760 tx=0x7f8df409f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a212d785fdbe922c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.187+0000 7f8de37fe640 1 -- 172.21.15.37:0/3951419225 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df409bee0 con 0x560006f47fd0 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.187+0000 7f8de37fe640 1 -- 172.21.15.37:0/3951419225 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8df40a3f80 con 0x560006f47fd0 2024-03-26T06:35:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.187+0000 7f8de37fe640 1 -- 172.21.15.37:0/3951419225 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df40ccb50 con 0x560006f47fd0 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.188+0000 7f8e027f4740 1 -- 172.21.15.37:0/3951419225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 msgr2=0x560006f50d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.188+0000 7f8e027f4740 1 --2- 172.21.15.37:0/3951419225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f50d90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8df400c760 tx=0x7f8df409f950 comp rx=0 tx=0).stop 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.188+0000 7f8e027f4740 1 -- 172.21.15.37:0/3951419225 shutdown_connections 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.188+0000 7f8e027f4740 1 --2- 172.21.15.37:0/3951419225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f51550 0x560006f47970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.188+0000 7f8e027f4740 1 --2- 172.21.15.37:0/3951419225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f48d40 0x560006f42e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.188+0000 7f8e027f4740 1 --2- 172.21.15.37:0/3951419225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f47fd0 0x560006f50d90 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.188+0000 7f8e027f4740 1 -- 172.21.15.37:0/3951419225 >> 172.21.15.37:0/3951419225 conn(0x560006f0f5b0 msgr2=0x560006f415b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.188+0000 7f8e027f4740 1 -- 172.21.15.37:0/3951419225 shutdown_connections 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.188+0000 7f8e027f4740 1 -- 172.21.15.37:0/3951419225 wait complete. 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.188+0000 7f8e027f4740 1 -- start start 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.188+0000 7f8e027f4740 1 Processor -- start 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f47fd0 0x560006f65da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f48d40 0x560006f662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8e027f4740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f51550 0x560006f4ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8e027f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81b03b0 con 0x560006f48d40 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8e027f4740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81afcf0 con 0x560006f51550 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8e027f4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81ad230 con 0x560006f47fd0 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8e027f4740 10 client.0 _ll_register_callbacks cb 0x7f8df4003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfd0df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f48d40 0x560006f662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfd8e0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f47fd0 0x560006f65da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfe0e1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f51550 0x560006f4ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfd0df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f48d40 0x560006f662e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57962/0 (socket says 172.21.15.37:57962) 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfd0df640 1 -- 172.21.15.37:0/1264719525 learned_addr learned my addr 172.21.15.37:0/1264719525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/1264719525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f51550 msgr2=0x560006f4ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f51550 0x560006f4ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/1264719525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f48d40 msgr2=0x560006f662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f48d40 0x560006f662e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfd8e0640 1 -- 172.21.15.37:0/1264719525 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560006f4b5d0 con 0x560006f47fd0 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfd0df640 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f48d40 0x560006f662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8dfd8e0640 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f47fd0 0x560006f65da0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8df400ea30 tx=0x7f8df401ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0be6013423526bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.189+0000 7f8de17fa640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8de17fa640 1 -- 172.21.15.37:0/1264719525 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df409bee0 con 0x560006f47fd0 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8de17fa640 1 -- 172.21.15.37:0/1264719525 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8df40a3af0 con 0x560006f47fd0 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8e027f4740 1 -- 172.21.15.37:0/1264719525 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8df40a3f80 con 0x560006f47fd0 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8de17fa640 1 -- 172.21.15.37:0/1264719525 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8df409b140 con 0x560006f47fd0 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8e027f4740 10 client.25915 fetch_fsmap learned FSMap version 52 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8e027f4740 1 -- 172.21.15.37:0/1264719525 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x560006f4b5d0 con 0x560006f47fd0 2024-03-26T06:35:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8de17fa640 1 -- 172.21.15.37:0/1264719525 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8df40a3f80 con 0x560006f47fd0 2024-03-26T06:35:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8de17fa640 1 -- 172.21.15.37:0/1264719525 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 52) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f8df409ac30 con 0x560006f47fd0 2024-03-26T06:35:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8e027f4740 10 client.25915 fetch_fsmap finished waiting for FSMap version 52 2024-03-26T06:35:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8e027f4740 10 client.25915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:35:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8e027f4740 20 client.25915 populate_metadata read hostname 'smithi037' 2024-03-26T06:35:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560006fb88b0 0x560006fbaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8e027f4740 4 client.25915 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:35:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8e027f4740 1 -- 172.21.15.37:0/1264719525 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560006eb3e80 con 0x560006fb88b0 2024-03-26T06:35:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.190+0000 7f8dfd0df640 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560006fb88b0 0x560006fbaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.191+0000 7f8de17fa640 10 client.25915 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.191+0000 7f8dfd0df640 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560006fb88b0 0x560006fbaca0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262bfb7f8a699bcf server_cookie=73963974c8d539a1 in_seq=0 out_seq=0 2024-03-26T06:35:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8de17fa640 1 -- 172.21.15.37:0/1264719525 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560006eb3e80 con 0x560006fb88b0 2024-03-26T06:35:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8de17fa640 10 client.25915 handle_command_reply: tid=0 2024-03-26T06:35:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 1 client.25915 shutdown 2024-03-26T06:35:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 1 -- 172.21.15.37:0/1264719525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f47fd0 msgr2=0x560006f65da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f47fd0 0x560006f65da0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8df400ea30 tx=0x7f8df401ff80 comp rx=0 tx=0).stop 2024-03-26T06:35:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 1 -- 172.21.15.37:0/1264719525 shutdown_connections 2024-03-26T06:35:31.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x560006fb88b0 0x560006fbaca0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560006f51550 0x560006f4ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8de17fa640 0 client.25915 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560006f48d40 0x560006f662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 1 --2- 172.21.15.37:0/1264719525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560006f47fd0 0x560006f65da0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 1 -- 172.21.15.37:0/1264719525 >> 172.21.15.37:0/1264719525 conn(0x560006f0f5b0 msgr2=0x560006f440a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 1 -- 172.21.15.37:0/1264719525 shutdown_connections 2024-03-26T06:35:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 1 -- 172.21.15.37:0/1264719525 wait complete. 2024-03-26T06:35:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.243+0000 7f8e027f4740 20 client.25915 trim_cache size 0 max 16384 2024-03-26T06:35:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:35:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:35:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:35:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:35:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 -- 172.21.15.37:0/3662378391 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f8de40a2ac0 msgr2=0x7f8de40a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f8de40a2ac0 0x7f8de40a4f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8df0005c40 tx=0x7f8df0009e50 comp rx=0 tx=0).stop 2024-03-26T06:35:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 -- 172.21.15.37:0/3662378391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8df8100440 msgr2=0x7f8df8101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8df8100440 0x7f8df8101e60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8df400ea60 tx=0x7f8df400ef20 comp rx=0 tx=0).stop 2024-03-26T06:35:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 -- 172.21.15.37:0/3662378391 shutdown_connections 2024-03-26T06:35:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f8de40a2ac0 0x7f8de40a4f80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df8105190 0x7f8df81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8df8100e00 0x7f8df81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 --2- 172.21.15.37:0/3662378391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8df8100440 0x7f8df8101e60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 -- 172.21.15.37:0/3662378391 >> 172.21.15.37:0/3662378391 conn(0x7f8df8077040 msgr2=0x7f8df8068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 -- 172.21.15.37:0/3662378391 shutdown_connections 2024-03-26T06:35:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:31.244+0000 7f8dffb44640 1 -- 172.21.15.37:0/3662378391 wait complete. 2024-03-26T06:35:31.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 484397 4.07 MB/sec execute 546 sec latency 41.391 ms 2024-03-26T06:35:32.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:32 smithi066 ceph-mon[36504]: pgmap v393: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 270 op/s 2024-03-26T06:35:32.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:32 smithi149 ceph-mon[36681]: pgmap v393: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 270 op/s 2024-03-26T06:35:32.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:32 smithi037 ceph-mon[31974]: pgmap v393: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 270 op/s 2024-03-26T06:35:32.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 486579 4.08 MB/sec execute 547 sec latency 31.374 ms 2024-03-26T06:35:33.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 487944 4.09 MB/sec execute 548 sec latency 14.186 ms 2024-03-26T06:35:34.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:34 smithi066 ceph-mon[36504]: pgmap v394: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 221 op/s 2024-03-26T06:35:34.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:34 smithi149 ceph-mon[36681]: pgmap v394: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 221 op/s 2024-03-26T06:35:34.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:34 smithi037 ceph-mon[31974]: pgmap v394: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 221 op/s 2024-03-26T06:35:34.532 INFO:tasks.workunit.client.0.smithi037.stdout: 1 490137 4.10 MB/sec execute 549 sec latency 16.708 ms 2024-03-26T06:35:35.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 491492 4.11 MB/sec execute 550 sec latency 12.578 ms 2024-03-26T06:35:36.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:36 smithi066 ceph-mon[36504]: pgmap v395: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-03-26T06:35:36.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:36 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:36.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:36 smithi149 ceph-mon[36681]: pgmap v395: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-03-26T06:35:36.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:36 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:36.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:36 smithi037 ceph-mon[31974]: pgmap v395: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-03-26T06:35:36.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:36 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:36.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 493651 4.12 MB/sec execute 551 sec latency 17.728 ms 2024-03-26T06:35:37.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 494888 4.13 MB/sec execute 552 sec latency 19.478 ms 2024-03-26T06:35:38.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:38 smithi066 ceph-mon[36504]: pgmap v396: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-26T06:35:38.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:38 smithi149 ceph-mon[36681]: pgmap v396: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-26T06:35:38.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:38 smithi037 ceph-mon[31974]: pgmap v396: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-26T06:35:38.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 496841 4.14 MB/sec execute 553 sec latency 19.793 ms 2024-03-26T06:35:39.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 498226 4.15 MB/sec execute 554 sec latency 19.425 ms 2024-03-26T06:35:40.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:40 smithi066 ceph-mon[36504]: pgmap v397: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-26T06:35:40.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:40 smithi149 ceph-mon[36681]: pgmap v397: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-26T06:35:40.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:40 smithi037 ceph-mon[31974]: pgmap v397: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2024-03-26T06:35:40.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 500315 4.16 MB/sec execute 555 sec latency 23.496 ms 2024-03-26T06:35:41.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:41 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:41.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:41 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:41.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:41 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:35:41.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 501731 4.16 MB/sec execute 556 sec latency 17.211 ms 2024-03-26T06:35:42.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:42 smithi066 ceph-mon[36504]: pgmap v398: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 362 op/s 2024-03-26T06:35:42.350 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:42 smithi037 ceph-mon[31974]: pgmap v398: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 362 op/s 2024-03-26T06:35:42.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:42 smithi149 ceph-mon[36681]: pgmap v398: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 362 op/s 2024-03-26T06:35:42.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 503870 4.18 MB/sec execute 557 sec latency 17.662 ms 2024-03-26T06:35:43.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 505405 4.18 MB/sec execute 558 sec latency 11.221 ms 2024-03-26T06:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.335+0000 7f0308a30640 1 -- 172.21.15.37:0/326578805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0304105190 msgr2=0x7f0304105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.335+0000 7f0308a30640 1 --2- 172.21.15.37:0/326578805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0304105190 0x7f0304105570 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009900 tx=0x7f02f402bc30 comp rx=0 tx=0).stop 2024-03-26T06:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.336+0000 7f0308a30640 1 -- 172.21.15.37:0/326578805 shutdown_connections 2024-03-26T06:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.336+0000 7f0308a30640 1 --2- 172.21.15.37:0/326578805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0304100e00 0x7f030410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.336+0000 7f0308a30640 1 --2- 172.21.15.37:0/326578805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0304100440 0x7f03041008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.336+0000 7f0308a30640 1 --2- 172.21.15.37:0/326578805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0304105190 0x7f0304105570 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.336+0000 7f0308a30640 1 -- 172.21.15.37:0/326578805 >> 172.21.15.37:0/326578805 conn(0x7f0304077040 msgr2=0x7f0304077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.336+0000 7f0308a30640 1 -- 172.21.15.37:0/326578805 shutdown_connections 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.336+0000 7f0308a30640 1 -- 172.21.15.37:0/326578805 wait complete. 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.336+0000 7f0308a30640 1 Processor -- start 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.336+0000 7f0308a30640 1 -- start start 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.336+0000 7f0308a30640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0304100440 0x7f0304198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.336+0000 7f0308a30640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0304100e00 0x7f0304198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f0308a30640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0304105190 0x7f03041a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f0308a30640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0304110650 con 0x7f0304100440 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f0308a30640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041104d0 con 0x7f0304100e00 2024-03-26T06:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f0308a30640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041107d0 con 0x7f0304105190 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f030259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0304100440 0x7f0304198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f030259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0304100440 0x7f0304198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36472/0 (socket says 172.21.15.37:36472) 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f030259c640 1 -- 172.21.15.37:0/754249394 learned_addr learned my addr 172.21.15.37:0/754249394 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f02fbfff640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0304100e00 0x7f0304198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f0302d9d640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0304105190 0x7f03041a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f030259c640 1 -- 172.21.15.37:0/754249394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0304105190 msgr2=0x7f03041a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f030259c640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0304105190 0x7f03041a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f030259c640 1 -- 172.21.15.37:0/754249394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0304100e00 msgr2=0x7f0304198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f030259c640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0304100e00 0x7f0304198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f030259c640 1 -- 172.21.15.37:0/754249394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03041a3f30 con 0x7f0304100440 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f02fbfff640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0304100e00 0x7f0304198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f030259c640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0304100440 0x7f0304198970 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f02f4005940 tx=0x7f02f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f02fb7fe640 1 -- 172.21.15.37:0/754249394 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4041070 con 0x7f0304100440 2024-03-26T06:35:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f02fb7fe640 1 -- 172.21.15.37:0/754249394 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02f40054e0 con 0x7f0304100440 2024-03-26T06:35:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f0308a30640 1 -- 172.21.15.37:0/754249394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03041a4100 con 0x7f0304100440 2024-03-26T06:35:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f02fb7fe640 1 -- 172.21.15.37:0/754249394 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4003d00 con 0x7f0304100440 2024-03-26T06:35:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.337+0000 7f0308a30640 1 -- 172.21.15.37:0/754249394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03041a45c0 con 0x7f0304100440 2024-03-26T06:35:44.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:44 smithi066 ceph-mon[36504]: pgmap v399: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 248 op/s 2024-03-26T06:35:44.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.340+0000 7f030b60c740 1 Processor -- start 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.340+0000 7f030b60c740 1 -- start start 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.340+0000 7f02fb7fe640 1 -- 172.21.15.37:0/754249394 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f02f4044070 con 0x7f0304100440 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.340+0000 7f030b60c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c208ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.340+0000 7f030b60c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 0x5558c208d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f030b60c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c2096550 0x5558c2098940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f030b60c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c1ff15f0 con 0x5558c208cfd0 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f030b60c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c1f9a930 con 0x5558c2096550 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f030b60c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c1fed690 con 0x5558c208dd40 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f030259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 0x5558c208d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f0302d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c208ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f02fbfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c2096550 0x5558c2098940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f030259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 0x5558c208d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36488/0 (socket says 172.21.15.37:36488) 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f030259c640 1 -- 172.21.15.37:0/513784016 learned_addr learned my addr 172.21.15.37:0/513784016 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f02fb7fe640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f02f00a2a10 0x7f02f00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f0302d9d640 1 -- 172.21.15.37:0/513784016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c2096550 msgr2=0x5558c2098940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f0302d9d640 1 --2- 172.21.15.37:0/513784016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c2096550 0x5558c2098940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f0302d9d640 1 -- 172.21.15.37:0/513784016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 msgr2=0x5558c208d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f0302d9d640 1 --2- 172.21.15.37:0/513784016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 0x5558c208d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f0302d9d640 1 -- 172.21.15.37:0/513784016 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558c1d06210 con 0x5558c208dd40 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f02fb7fe640 1 -- 172.21.15.37:0/754249394 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f02f40f0990 con 0x7f0304100440 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f0302d9d640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f02f00a2a10 0x7f02f00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.341+0000 7f0302d9d640 1 --2- 172.21.15.37:0/513784016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c208ca90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009ef0 tx=0x7f02fc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e5843b9304fe89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f02f8ff9640 1 -- 172.21.15.37:0/513784016 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc004050 con 0x5558c208dd40 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f02f8ff9640 1 -- 172.21.15.37:0/513784016 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02fc003e30 con 0x5558c208dd40 2024-03-26T06:35:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f02f8ff9640 1 -- 172.21.15.37:0/513784016 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc00e3b0 con 0x5558c208dd40 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f030b60c740 1 -- 172.21.15.37:0/513784016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 msgr2=0x5558c208ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f0302d9d640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f02f00a2a10 0x7f02f00a4ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f02fc01d000 tx=0x7f02fc00e630 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f030b60c740 1 --2- 172.21.15.37:0/513784016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c208ca90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009ef0 tx=0x7f02fc011810 comp rx=0 tx=0).stop 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f030b60c740 1 -- 172.21.15.37:0/513784016 shutdown_connections 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f030b60c740 1 --2- 172.21.15.37:0/513784016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c2096550 0x5558c2098940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f030b60c740 1 --2- 172.21.15.37:0/513784016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 0x5558c208d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f030b60c740 1 --2- 172.21.15.37:0/513784016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c208ca90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f030b60c740 1 -- 172.21.15.37:0/513784016 >> 172.21.15.37:0/513784016 conn(0x5558c20545b0 msgr2=0x5558c2087cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f030b60c740 1 -- 172.21.15.37:0/513784016 shutdown_connections 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.342+0000 7f030b60c740 1 -- 172.21.15.37:0/513784016 wait complete. 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030b60c740 1 -- start start 2024-03-26T06:35:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030b60c740 1 Processor -- start 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030b60c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 0x5558c20935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030b60c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c2093b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030b60c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c2096550 0x5558c2094060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030b60c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c2099280 con 0x5558c208cfd0 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030b60c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c1fadd90 con 0x5558c2096550 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030b60c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c2099400 con 0x5558c208dd40 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030b60c740 10 client.0 _ll_register_callbacks cb 0x5558c1d05a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 0x5558c20935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 0x5558c20935e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36500/0 (socket says 172.21.15.37:36500) 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030259c640 1 -- 172.21.15.37:0/842425388 learned_addr learned my addr 172.21.15.37:0/842425388 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f02fbfff640 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c2093b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f0302d9d640 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c2096550 0x5558c2094060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030259c640 1 -- 172.21.15.37:0/842425388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 msgr2=0x5558c2093b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030259c640 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c2093b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030259c640 1 -- 172.21.15.37:0/842425388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c2096550 msgr2=0x5558c2094060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030259c640 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c2096550 0x5558c2094060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.343+0000 7f030259c640 1 -- 172.21.15.37:0/842425388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558c1dd2620 con 0x5558c208cfd0 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f030259c640 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 0x5558c20935e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f02f40c5370 tx=0x7f02f40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d36ecd94d158c0e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f02e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f02e9ffb640 1 -- 172.21.15.37:0/842425388 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f404b070 con 0x5558c208cfd0 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f02e9ffb640 1 -- 172.21.15.37:0/842425388 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f02f40b9820 con 0x5558c208cfd0 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f030b60c740 1 -- 172.21.15.37:0/842425388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02fc003e30 con 0x5558c208cfd0 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f02e9ffb640 1 -- 172.21.15.37:0/842425388 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02f40bcf80 con 0x5558c208cfd0 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f030b60c740 10 client.16542 fetch_fsmap learned FSMap version 54 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f030b60c740 1 -- 172.21.15.37:0/842425388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x5558c20a55c0 con 0x5558c208cfd0 2024-03-26T06:35:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f02e9ffb640 1 -- 172.21.15.37:0/842425388 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02fc003e30 con 0x5558c208cfd0 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f02e9ffb640 1 -- 172.21.15.37:0/842425388 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f02f40c5410 con 0x5558c208cfd0 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f030b60c740 10 client.16542 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f030b60c740 10 client.16542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f030b60c740 20 client.16542 populate_metadata read hostname 'smithi037' 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f030b60c740 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5558c20a6ab0 0x5558c20a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f030b60c740 4 client.16542 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.344+0000 7f030b60c740 1 -- 172.21.15.37:0/842425388 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558c1ffeb30 con 0x5558c20a6ab0 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.345+0000 7f02fbfff640 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5558c20a6ab0 0x5558c20a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.345+0000 7f02e9ffb640 10 client.16542 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.345+0000 7f02fbfff640 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5558c20a6ab0 0x5558c20a8f10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d23f0adb08ca9c74 server_cookie=e4b6083ac2d80d25 in_seq=0 out_seq=0 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f02e9ffb640 1 -- 172.21.15.37:0/842425388 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558c1ffeb30 con 0x5558c20a6ab0 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f02e9ffb640 10 client.16542 handle_command_reply: tid=0 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 1 client.16542 shutdown 2024-03-26T06:35:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 1 -- 172.21.15.37:0/842425388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 msgr2=0x5558c20935e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 0x5558c20935e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f02f40c5370 tx=0x7f02f40035f0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 1 -- 172.21.15.37:0/842425388 shutdown_connections 2024-03-26T06:35:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5558c20a6ab0 0x5558c20a8f10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c2096550 0x5558c2094060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c2093b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f02e9ffb640 0 client.16542 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 1 --2- 172.21.15.37:0/842425388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c208cfd0 0x5558c20935e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 1 -- 172.21.15.37:0/842425388 >> 172.21.15.37:0/842425388 conn(0x5558c20545b0 msgr2=0x5558c2089a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 1 -- 172.21.15.37:0/842425388 shutdown_connections 2024-03-26T06:35:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 1 -- 172.21.15.37:0/842425388 wait complete. 2024-03-26T06:35:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.346+0000 7f030b60c740 20 client.16542 trim_cache size 0 max 16384 2024-03-26T06:35:44.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f030b60c740 1 Processor -- start 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f030b60c740 1 -- start start 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f030b60c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208cfd0 0x5558c2095d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f030b60c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208dd40 0x5558c2090860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f030b60c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 0x5558c2090da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f030b60c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041acc40 con 0x5558c2096550 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f030b60c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041ae7a0 con 0x5558c208dd40 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f030b60c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041ad2a0 con 0x5558c208cfd0 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f0302d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 0x5558c2090da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f0302d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 0x5558c2090da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36502/0 (socket says 172.21.15.37:36502) 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f0302d9d640 1 -- 172.21.15.37:0/1937345266 learned_addr learned my addr 172.21.15.37:0/1937345266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f02fbfff640 1 --2- 172.21.15.37:0/1937345266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208dd40 0x5558c2090860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f030259c640 1 --2- 172.21.15.37:0/1937345266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208cfd0 0x5558c2095d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f0302d9d640 1 -- 172.21.15.37:0/1937345266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208cfd0 msgr2=0x5558c2095d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f0302d9d640 1 --2- 172.21.15.37:0/1937345266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208cfd0 0x5558c2095d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f0302d9d640 1 -- 172.21.15.37:0/1937345266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208dd40 msgr2=0x5558c2090860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f0302d9d640 1 --2- 172.21.15.37:0/1937345266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208dd40 0x5558c2090860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f0302d9d640 1 -- 172.21.15.37:0/1937345266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558c208f1a0 con 0x5558c2096550 2024-03-26T06:35:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f030259c640 1 --2- 172.21.15.37:0/1937345266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208cfd0 0x5558c2095d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:35:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.360+0000 7f02fbfff640 1 --2- 172.21.15.37:0/1937345266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208dd40 0x5558c2090860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:35:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.361+0000 7f0302d9d640 1 --2- 172.21.15.37:0/1937345266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 0x5558c2090da0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009ec0 tx=0x7f02fc028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34db9d0da9112a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.361+0000 7f02ebfff640 1 -- 172.21.15.37:0/1937345266 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc00e3b0 con 0x5558c2096550 2024-03-26T06:35:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.361+0000 7f02ebfff640 1 -- 172.21.15.37:0/1937345266 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02fc00ed00 con 0x5558c2096550 2024-03-26T06:35:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.361+0000 7f02ebfff640 1 -- 172.21.15.37:0/1937345266 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc004050 con 0x5558c2096550 2024-03-26T06:35:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 -- 172.21.15.37:0/1937345266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 msgr2=0x5558c2090da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 --2- 172.21.15.37:0/1937345266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 0x5558c2090da0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009ec0 tx=0x7f02fc028000 comp rx=0 tx=0).stop 2024-03-26T06:35:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 -- 172.21.15.37:0/1937345266 shutdown_connections 2024-03-26T06:35:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 --2- 172.21.15.37:0/1937345266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 0x5558c2090da0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 --2- 172.21.15.37:0/1937345266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208dd40 0x5558c2090860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 --2- 172.21.15.37:0/1937345266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208cfd0 0x5558c2095d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 -- 172.21.15.37:0/1937345266 >> 172.21.15.37:0/1937345266 conn(0x5558c20545b0 msgr2=0x5558c2091eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 -- 172.21.15.37:0/1937345266 shutdown_connections 2024-03-26T06:35:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 -- 172.21.15.37:0/1937345266 wait complete. 2024-03-26T06:35:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 -- start start 2024-03-26T06:35:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 Processor -- start 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208cfd0 0x5558c209c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c209cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 0x5558c20fd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041ae080 con 0x5558c2096550 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041ad9c0 con 0x5558c208cfd0 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041aaf00 con 0x5558c208dd40 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.362+0000 7f030b60c740 10 client.0 _ll_register_callbacks cb 0x7f02fc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f030259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208cfd0 0x5558c209c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f0302d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 0x5558c20fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f030259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208cfd0 0x5558c209c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:42884/0 (socket says 172.21.15.37:42884) 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f030259c640 1 -- 172.21.15.37:0/1400452232 learned_addr learned my addr 172.21.15.37:0/1400452232 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f02fbfff640 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c209cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f030259c640 1 -- 172.21.15.37:0/1400452232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 msgr2=0x5558c209cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f030259c640 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c209cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f030259c640 1 -- 172.21.15.37:0/1400452232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 msgr2=0x5558c20fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f030259c640 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 0x5558c20fd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f030259c640 1 -- 172.21.15.37:0/1400452232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558c20fd540 con 0x5558c208cfd0 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f02fbfff640 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c209cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f0302d9d640 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 0x5558c20fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f030259c640 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208cfd0 0x5558c209c740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f02f40f0e10 tx=0x7f02f402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=912f1c441317581f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f02e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f02e9ffb640 1 -- 172.21.15.37:0/1400452232 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f404b040 con 0x5558c208cfd0 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f030b60c740 1 -- 172.21.15.37:0/1400452232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02fc00ed00 con 0x5558c208cfd0 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f02e9ffb640 1 -- 172.21.15.37:0/1400452232 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f02f40bc600 con 0x5558c208cfd0 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.363+0000 7f02e9ffb640 1 -- 172.21.15.37:0/1400452232 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02f40bcb10 con 0x5558c208cfd0 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.364+0000 7f030b60c740 10 client.26069 fetch_fsmap learned FSMap version 54 2024-03-26T06:35:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.364+0000 7f030b60c740 1 -- 172.21.15.37:0/1400452232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x5558c20fe7c0 con 0x5558c208cfd0 2024-03-26T06:35:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.364+0000 7f02e9ffb640 1 -- 172.21.15.37:0/1400452232 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02fc00ed00 con 0x5558c208cfd0 2024-03-26T06:35:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.364+0000 7f02e9ffb640 1 -- 172.21.15.37:0/1400452232 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f02f40d9020 con 0x5558c208cfd0 2024-03-26T06:35:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.364+0000 7f030b60c740 10 client.26069 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:35:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.364+0000 7f030b60c740 10 client.26069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:35:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.364+0000 7f030b60c740 20 client.26069 populate_metadata read hostname 'smithi037' 2024-03-26T06:35:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.364+0000 7f030b60c740 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5558c20fed20 0x5558c2101140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.364+0000 7f030b60c740 4 client.26069 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:35:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.364+0000 7f030b60c740 1 -- 172.21.15.37:0/1400452232 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5558c1ff8e80 con 0x5558c20fed20 2024-03-26T06:35:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.364+0000 7f02fbfff640 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5558c20fed20 0x5558c2101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.365+0000 7f02e9ffb640 10 client.26069 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.365+0000 7f02fbfff640 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5558c20fed20 0x5558c2101140 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fcc4bacbbc9d677 server_cookie=7e68b5859f29d4a5 in_seq=0 out_seq=0 2024-03-26T06:35:44.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:44 smithi149 ceph-mon[36681]: pgmap v399: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 248 op/s 2024-03-26T06:35:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f02e9ffb640 1 -- 172.21.15.37:0/1400452232 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5558c1ff8e80 con 0x5558c20fed20 2024-03-26T06:35:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f02e9ffb640 10 client.26069 handle_command_reply: tid=0 2024-03-26T06:35:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 1 client.26069 shutdown 2024-03-26T06:35:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 1 -- 172.21.15.37:0/1400452232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208cfd0 msgr2=0x5558c209c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208cfd0 0x5558c209c740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f02f40f0e10 tx=0x7f02f402bc30 comp rx=0 tx=0).stop 2024-03-26T06:35:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 1 -- 172.21.15.37:0/1400452232 shutdown_connections 2024-03-26T06:35:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5558c20fed20 0x5558c2101140 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558c2096550 0x5558c20fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f02e9ffb640 0 client.26069 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558c208dd40 0x5558c209cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 1 --2- 172.21.15.37:0/1400452232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5558c208cfd0 0x5558c209c740 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 1 -- 172.21.15.37:0/1400452232 >> 172.21.15.37:0/1400452232 conn(0x5558c20545b0 msgr2=0x5558c20865b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 1 -- 172.21.15.37:0/1400452232 shutdown_connections 2024-03-26T06:35:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 1 -- 172.21.15.37:0/1400452232 wait complete. 2024-03-26T06:35:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.411+0000 7f030b60c740 20 client.26069 trim_cache size 0 max 16384 2024-03-26T06:35:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:35:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:35:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:35:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:35:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 -- 172.21.15.37:0/754249394 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f02f00a2a10 msgr2=0x7f02f00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f02f00a2a10 0x7f02f00a4ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f02fc01d000 tx=0x7f02fc00e630 comp rx=0 tx=0).stop 2024-03-26T06:35:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 -- 172.21.15.37:0/754249394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0304100440 msgr2=0x7f0304198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0304100440 0x7f0304198970 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f02f4005940 tx=0x7f02f4004270 comp rx=0 tx=0).stop 2024-03-26T06:35:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 -- 172.21.15.37:0/754249394 shutdown_connections 2024-03-26T06:35:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f02f00a2a10 0x7f02f00a4ed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0304105190 0x7f03041a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0304100e00 0x7f0304198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 --2- 172.21.15.37:0/754249394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0304100440 0x7f0304198970 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 -- 172.21.15.37:0/754249394 >> 172.21.15.37:0/754249394 conn(0x7f0304077040 msgr2=0x7f03040fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 -- 172.21.15.37:0/754249394 shutdown_connections 2024-03-26T06:35:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:44.412+0000 7f0308a30640 1 -- 172.21.15.37:0/754249394 wait complete. 2024-03-26T06:35:44.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:44 smithi037 ceph-mon[31974]: pgmap v399: 97 pgs: 97 active+clean; 206 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 248 op/s 2024-03-26T06:35:44.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 507421 4.20 MB/sec execute 559 sec latency 38.574 ms 2024-03-26T06:35:45.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 508904 4.20 MB/sec execute 560 sec latency 12.125 ms 2024-03-26T06:35:46.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:46 smithi066 ceph-mon[36504]: pgmap v400: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 365 op/s 2024-03-26T06:35:46.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:46 smithi149 ceph-mon[36681]: pgmap v400: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 365 op/s 2024-03-26T06:35:46.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:46 smithi037 ceph-mon[31974]: pgmap v400: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 365 op/s 2024-03-26T06:35:46.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 510957 4.22 MB/sec execute 561 sec latency 15.436 ms 2024-03-26T06:35:47.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 512247 4.22 MB/sec execute 562 sec latency 20.335 ms 2024-03-26T06:35:48.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:48 smithi066 ceph-mon[36504]: pgmap v401: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-03-26T06:35:48.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:48 smithi149 ceph-mon[36681]: pgmap v401: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-03-26T06:35:48.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:48 smithi037 ceph-mon[31974]: pgmap v401: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-03-26T06:35:48.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 514140 4.23 MB/sec execute 563 sec latency 20.695 ms 2024-03-26T06:35:49.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 515698 4.24 MB/sec execute 564 sec latency 11.182 ms 2024-03-26T06:35:50.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:50 smithi066 ceph-mon[36504]: pgmap v402: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-03-26T06:35:50.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:50 smithi149 ceph-mon[36681]: pgmap v402: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-03-26T06:35:50.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:50 smithi037 ceph-mon[31974]: pgmap v402: 97 pgs: 97 active+clean; 202 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-03-26T06:35:50.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 517578 4.25 MB/sec execute 565 sec latency 18.363 ms 2024-03-26T06:35:51.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 519206 4.26 MB/sec execute 566 sec latency 10.921 ms 2024-03-26T06:35:52.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:52 smithi066 ceph-mon[36504]: pgmap v403: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 320 op/s 2024-03-26T06:35:52.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:52 smithi149 ceph-mon[36681]: pgmap v403: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 320 op/s 2024-03-26T06:35:52.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:52 smithi037 ceph-mon[31974]: pgmap v403: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 320 op/s 2024-03-26T06:35:52.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 521034 4.26 MB/sec execute 567 sec latency 53.472 ms 2024-03-26T06:35:53.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 522640 4.28 MB/sec execute 568 sec latency 16.408 ms 2024-03-26T06:35:54.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:54 smithi066 ceph-mon[36504]: pgmap v404: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-03-26T06:35:54.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:54 smithi149 ceph-mon[36681]: pgmap v404: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-03-26T06:35:54.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:54 smithi037 ceph-mon[31974]: pgmap v404: 97 pgs: 97 active+clean; 210 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-03-26T06:35:54.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 523906 4.28 MB/sec execute 569 sec latency 204.895 ms 2024-03-26T06:35:55.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 524705 4.28 MB/sec execute 570 sec latency 855.889 ms 2024-03-26T06:35:56.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:56 smithi066 ceph-mon[36504]: pgmap v405: 97 pgs: 97 active+clean; 216 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-03-26T06:35:56.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:56 smithi149 ceph-mon[36681]: pgmap v405: 97 pgs: 97 active+clean; 216 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-03-26T06:35:56.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:56 smithi037 ceph-mon[31974]: pgmap v405: 97 pgs: 97 active+clean; 216 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-03-26T06:35:56.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 526352 4.29 MB/sec execute 571 sec latency 11.422 ms 2024-03-26T06:35:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.503+0000 7fb3fcb16640 1 -- 172.21.15.37:0/4264971639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f8100430 msgr2=0x7fb3f8100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.503+0000 7fb3fcb16640 1 --2- 172.21.15.37:0/4264971639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f8100430 0x7fb3f8100810 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009900 tx=0x7fb3e802bc30 comp rx=0 tx=0).stop 2024-03-26T06:35:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.503+0000 7fb3fcb16640 1 -- 172.21.15.37:0/4264971639 shutdown_connections 2024-03-26T06:35:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.503+0000 7fb3fcb16640 1 --2- 172.21.15.37:0/4264971639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f8105500 0x7fb3f810d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.503+0000 7fb3fcb16640 1 --2- 172.21.15.37:0/4264971639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3f8100de0 0x7fb3f8104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.503+0000 7fb3fcb16640 1 --2- 172.21.15.37:0/4264971639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f8100430 0x7fb3f8100810 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.503+0000 7fb3fcb16640 1 -- 172.21.15.37:0/4264971639 >> 172.21.15.37:0/4264971639 conn(0x7fb3f8077320 msgr2=0x7fb3f8077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.503+0000 7fb3fcb16640 1 -- 172.21.15.37:0/4264971639 shutdown_connections 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.503+0000 7fb3fcb16640 1 -- 172.21.15.37:0/4264971639 wait complete. 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3fcb16640 1 Processor -- start 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3fcb16640 1 -- start start 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3fcb16640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f8100430 0x7fb3f81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3fcb16640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f8105500 0x7fb3f8198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3fcb16640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3f8199420 0x7fb3f81a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3fcb16640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81105c0 con 0x7fb3f8100430 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3fcb16640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8110440 con 0x7fb3f8199420 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3fcb16640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8110740 con 0x7fb3f8105500 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3f659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f8100430 0x7fb3f81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3f659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f8100430 0x7fb3f81989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53262/0 (socket says 172.21.15.37:53262) 2024-03-26T06:35:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3f659c640 1 -- 172.21.15.37:0/322201333 learned_addr learned my addr 172.21.15.37:0/322201333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3f8199420 0x7fb3f81a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.504+0000 7fb3f5d9b640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f8105500 0x7fb3f8198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3f659c640 1 -- 172.21.15.37:0/322201333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f8105500 msgr2=0x7fb3f8198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3f659c640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f8105500 0x7fb3f8198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3f659c640 1 -- 172.21.15.37:0/322201333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3f8199420 msgr2=0x7fb3f81a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3f659c640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3f8199420 0x7fb3f81a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3f659c640 1 -- 172.21.15.37:0/322201333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f81a41a0 con 0x7fb3f8100430 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3f5d9b640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f8105500 0x7fb3f8198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3f659c640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f8100430 0x7fb3f81989a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006010 tx=0x7fb3e802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3e77fe640 1 -- 172.21.15.37:0/322201333 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8041070 con 0x7fb3f8100430 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3e77fe640 1 -- 172.21.15.37:0/322201333 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3e8004450 con 0x7fb3f8100430 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3e77fe640 1 -- 172.21.15.37:0/322201333 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8003e00 con 0x7fb3f8100430 2024-03-26T06:35:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3fcb16640 1 -- 172.21.15.37:0/322201333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f81a4430 con 0x7fb3f8100430 2024-03-26T06:35:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.505+0000 7fb3fcb16640 1 -- 172.21.15.37:0/322201333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f81a4970 con 0x7fb3f8100430 2024-03-26T06:35:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.507+0000 7fb3e77fe640 1 -- 172.21.15.37:0/322201333 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb3e8044070 con 0x7fb3f8100430 2024-03-26T06:35:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.507+0000 7fb3ff7ce740 1 Processor -- start 2024-03-26T06:35:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3ff7ce740 1 -- start start 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611acd40 0x55ca611aba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3e77fe640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb3e00a2a90 0x7fb3e00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611abfd0 0x55ca611ac450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611b5550 0x55ca611b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca611105f0 con 0x55ca611abfd0 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca610b9930 con 0x55ca611acd40 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6110c690 con 0x55ca611b5550 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f659c640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb3e00a2a90 0x7fb3e00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611abfd0 0x55ca611ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611acd40 0x55ca611aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611acd40 0x55ca611aba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:60172/0 (socket says 172.21.15.37:60172) 2024-03-26T06:35:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/2919167015 learned_addr learned my addr 172.21.15.37:0/2919167015 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/2919167015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611b5550 0x55ca611b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3e77fe640 1 -- 172.21.15.37:0/322201333 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb3e80f0930 con 0x7fb3f8100430 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/2919167015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611b5550 msgr2=0x55ca611b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/2919167015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611b5550 0x55ca611b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/2919167015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611abfd0 msgr2=0x55ca611ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/2919167015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611abfd0 0x55ca611ac450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/2919167015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca60e25210 con 0x55ca611acd40 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f659c640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb3e00a2a90 0x7fb3e00a4f50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80bcfd0 tx=0x7fb3e8042fc0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.508+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/2919167015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611acd40 0x55ca611aba90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0005f40 tx=0x7fb3f0005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa175c703f642d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3e4ff9640 1 -- 172.21.15.37:0/2919167015 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0011130 con 0x55ca611acd40 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3e4ff9640 1 -- 172.21.15.37:0/2919167015 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3f00112d0 con 0x55ca611acd40 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3e4ff9640 1 -- 172.21.15.37:0/2919167015 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0005260 con 0x55ca611acd40 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/2919167015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611acd40 msgr2=0x55ca611aba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/2919167015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611acd40 0x55ca611aba90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0005f40 tx=0x7fb3f0005bd0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/2919167015 shutdown_connections 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/2919167015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611b5550 0x55ca611b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/2919167015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611abfd0 0x55ca611ac450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/2919167015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611acd40 0x55ca611aba90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/2919167015 >> 172.21.15.37:0/2919167015 conn(0x55ca611735b0 msgr2=0x55ca611a6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/2919167015 shutdown_connections 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.509+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/2919167015 wait complete. 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3ff7ce740 1 -- start start 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3ff7ce740 1 Processor -- start 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611acd40 0x55ca611b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611b5550 0x55ca611b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca611b8280 con 0x55ca611acd40 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca610ccd90 con 0x55ca611abfd0 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca611b8400 con 0x55ca611b5550 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3ff7ce740 10 client.0 _ll_register_callbacks cb 0x55ca60e24a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3f5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611acd40 0x55ca611b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3f6d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3f5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611acd40 0x55ca611b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53286/0 (socket says 172.21.15.37:53286) 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3f5d9b640 1 -- 172.21.15.37:0/760499499 learned_addr learned my addr 172.21.15.37:0/760499499 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611b5550 0x55ca611b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3f5d9b640 1 -- 172.21.15.37:0/760499499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611b5550 msgr2=0x55ca611b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3f5d9b640 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611b5550 0x55ca611b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3f5d9b640 1 -- 172.21.15.37:0/760499499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 msgr2=0x55ca611b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3f5d9b640 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3f5d9b640 1 -- 172.21.15.37:0/760499499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca60ef1620 con 0x55ca611acd40 2024-03-26T06:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.510+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3f5d9b640 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611acd40 0x55ca611b2b00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00bf30 tx=0x7fb3ec0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d7a4d155b0a551 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ddffb640 1 -- 172.21.15.37:0/760499499 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0035f0 con 0x55ca611acd40 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ddffb640 1 -- 172.21.15.37:0/760499499 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb3ec003790 con 0x55ca611acd40 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/760499499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3f00112d0 con 0x55ca611acd40 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ddffb640 1 -- 172.21.15.37:0/760499499 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3ec003990 con 0x55ca611acd40 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ff7ce740 10 client.16560 fetch_fsmap learned FSMap version 54 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/760499499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55ca611c4640 con 0x55ca611acd40 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ddffb640 1 -- 172.21.15.37:0/760499499 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3f00112d0 con 0x55ca611acd40 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ddffb640 1 -- 172.21.15.37:0/760499499 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fb3ec019070 con 0x55ca611acd40 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ff7ce740 10 client.16560 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ff7ce740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ff7ce740 20 client.16560 populate_metadata read hostname 'smithi037' 2024-03-26T06:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ca611c5bd0 0x55ca611c8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ff7ce740 4 client.16560 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.511+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/760499499 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca6111db30 con 0x55ca611c5bd0 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.512+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ca611c5bd0 0x55ca611c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.512+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ca611c5bd0 0x55ca611c8010 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e14cbf3b661e593 server_cookie=54d1033f7915cc74 in_seq=0 out_seq=0 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.512+0000 7fb3ddffb640 10 client.16560 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.512+0000 7fb3ddffb640 1 -- 172.21.15.37:0/760499499 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca6111db30 con 0x55ca611c5bd0 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.512+0000 7fb3ddffb640 10 client.16560 handle_command_reply: tid=0 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.512+0000 7fb3ff7ce740 1 client.16560 shutdown 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/760499499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611acd40 msgr2=0x55ca611b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611acd40 0x55ca611b2b00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00bf30 tx=0x7fb3ec0128e0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/760499499 shutdown_connections 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ca611c5bd0 0x55ca611c8010 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611b5550 0x55ca611b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611acd40 0x55ca611b2b00 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/760499499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ddffb640 0 client.16560 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/760499499 >> 172.21.15.37:0/760499499 conn(0x55ca611735b0 msgr2=0x55ca611a8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/760499499 shutdown_connections 2024-03-26T06:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/760499499 wait complete. 2024-03-26T06:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.513+0000 7fb3ff7ce740 20 client.16560 trim_cache size 0 max 16384 2024-03-26T06:35:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3ff7ce740 1 Processor -- start 2024-03-26T06:35:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3ff7ce740 1 -- start start 2024-03-26T06:35:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611acd40 0x55ca611bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca611bcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81acce0 con 0x55ca611b5550 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81ae840 con 0x55ca611abfd0 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81ad340 con 0x55ca611acd40 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3f6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca611bcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3f6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca611bcbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53302/0 (socket says 172.21.15.37:53302) 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/2513862517 learned_addr learned my addr 172.21.15.37:0/2513862517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/2513862517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.526+0000 7fb3f5d9b640 1 --2- 172.21.15.37:0/2513862517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611acd40 0x55ca611bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/2513862517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611acd40 msgr2=0x55ca611bc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/2513862517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611acd40 0x55ca611bc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/2513862517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 msgr2=0x55ca611b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/2513862517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/2513862517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca611abd80 con 0x55ca611b5550 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/2513862517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca611bcbe0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000fdf0 tx=0x7fb3f000fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c821d1b32bbbf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3dffff640 1 -- 172.21.15.37:0/2513862517 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00112d0 con 0x55ca611b5550 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3dffff640 1 -- 172.21.15.37:0/2513862517 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3f002d400 con 0x55ca611b5550 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3dffff640 1 -- 172.21.15.37:0/2513862517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0005050 con 0x55ca611b5550 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/2513862517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 msgr2=0x55ca611bcbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/2513862517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca611bcbe0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000fdf0 tx=0x7fb3f000fec0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/2513862517 shutdown_connections 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/2513862517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca611bcbe0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/2513862517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611acd40 0x55ca611bc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/2513862517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/2513862517 >> 172.21.15.37:0/2513862517 conn(0x55ca611735b0 msgr2=0x55ca611a67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/2513862517 shutdown_connections 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.527+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/2513862517 wait complete. 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3ff7ce740 1 -- start start 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3ff7ce740 1 Processor -- start 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611acd40 0x55ca611b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3ff7ce740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca6121c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81ae120 con 0x55ca611b5550 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81ada60 con 0x55ca611abfd0 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3ff7ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81aafa0 con 0x55ca611acd40 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3ff7ce740 10 client.0 _ll_register_callbacks cb 0x7fb3f0009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3f6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca6121c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3f6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca6121c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53308/0 (socket says 172.21.15.37:53308) 2024-03-26T06:35:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/3292796132 learned_addr learned my addr 172.21.15.37:0/3292796132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3f5d9b640 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611acd40 0x55ca611b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/3292796132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611acd40 msgr2=0x55ca611b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611acd40 0x55ca611b9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/3292796132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 msgr2=0x55ca611b9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.528+0000 7fb3f6d9d640 1 -- 172.21.15.37:0/3292796132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca611ba200 con 0x55ca611b5550 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3f6d9d640 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca6121c0f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009a70 tx=0x7fb3f000ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2dc49f26d986ac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ddffb640 1 -- 172.21.15.37:0/3292796132 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0004a00 con 0x55ca611b5550 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/3292796132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3f002d400 con 0x55ca611b5550 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ddffb640 1 -- 172.21.15.37:0/3292796132 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb3f0004ba0 con 0x55ca611b5550 2024-03-26T06:35:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ddffb640 1 -- 172.21.15.37:0/3292796132 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3f0004da0 con 0x55ca611b5550 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ff7ce740 10 client.16566 fetch_fsmap learned FSMap version 54 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/3292796132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55ca6121d710 con 0x55ca611b5550 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ddffb640 1 -- 172.21.15.37:0/3292796132 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3f002d400 con 0x55ca611b5550 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ddffb640 1 -- 172.21.15.37:0/3292796132 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fb3f001d020 con 0x55ca611b5550 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ff7ce740 10 client.16566 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ff7ce740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ff7ce740 20 client.16566 populate_metadata read hostname 'smithi037' 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ca6121dc10 0x55ca61220030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ff7ce740 4 client.16566 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.529+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/3292796132 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca61117e80 con 0x55ca6121dc10 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.530+0000 7fb3f5d9b640 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ca6121dc10 0x55ca61220030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.530+0000 7fb3f5d9b640 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ca6121dc10 0x55ca61220030 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b94618e5ed03afb8 server_cookie=7744bc4ea63c9e6c in_seq=0 out_seq=0 2024-03-26T06:35:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.530+0000 7fb3ddffb640 10 client.16566 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:57.539 INFO:tasks.workunit.client.0.smithi037.stdout: 1 527456 4.29 MB/sec execute 572 sec latency 335.787 ms 2024-03-26T06:35:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.573+0000 7fb3ddffb640 1 -- 172.21.15.37:0/3292796132 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca61117e80 con 0x55ca6121dc10 2024-03-26T06:35:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.573+0000 7fb3ddffb640 10 client.16566 handle_command_reply: tid=0 2024-03-26T06:35:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.573+0000 7fb3ff7ce740 1 client.16566 shutdown 2024-03-26T06:35:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/3292796132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 msgr2=0x55ca6121c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca6121c0f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009a70 tx=0x7fb3f000ffd0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/3292796132 shutdown_connections 2024-03-26T06:35:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ca6121dc10 0x55ca61220030 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca611b5550 0x55ca6121c0f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca611acd40 0x55ca611b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ddffb640 0 client.16566 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:35:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ff7ce740 1 --2- 172.21.15.37:0/3292796132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca611abfd0 0x55ca611b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/3292796132 >> 172.21.15.37:0/3292796132 conn(0x55ca611735b0 msgr2=0x55ca611b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/3292796132 shutdown_connections 2024-03-26T06:35:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ff7ce740 1 -- 172.21.15.37:0/3292796132 wait complete. 2024-03-26T06:35:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3ff7ce740 20 client.16566 trim_cache size 0 max 16384 2024-03-26T06:35:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:35:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:35:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:35:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:35:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3fcb16640 1 -- 172.21.15.37:0/322201333 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb3e00a2a90 msgr2=0x7fb3e00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3fcb16640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb3e00a2a90 0x7fb3e00a4f50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80bcfd0 tx=0x7fb3e8042fc0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3fcb16640 1 -- 172.21.15.37:0/322201333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f8100430 msgr2=0x7fb3f81989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:35:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.574+0000 7fb3fcb16640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f8100430 0x7fb3f81989a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006010 tx=0x7fb3e802cb30 comp rx=0 tx=0).stop 2024-03-26T06:35:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.575+0000 7fb3fcb16640 1 -- 172.21.15.37:0/322201333 shutdown_connections 2024-03-26T06:35:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.575+0000 7fb3fcb16640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb3e00a2a90 0x7fb3e00a4f50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.575+0000 7fb3fcb16640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3f8199420 0x7fb3f81a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.575+0000 7fb3fcb16640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f8105500 0x7fb3f8198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.575+0000 7fb3fcb16640 1 --2- 172.21.15.37:0/322201333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f8100430 0x7fb3f81989a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:35:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.575+0000 7fb3fcb16640 1 -- 172.21.15.37:0/322201333 >> 172.21.15.37:0/322201333 conn(0x7fb3f8077320 msgr2=0x7fb3f80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:35:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.575+0000 7fb3fcb16640 1 -- 172.21.15.37:0/322201333 shutdown_connections 2024-03-26T06:35:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:35:57.575+0000 7fb3fcb16640 1 -- 172.21.15.37:0/322201333 wait complete. 2024-03-26T06:35:58.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:35:58 smithi066 ceph-mon[36504]: pgmap v406: 97 pgs: 97 active+clean; 216 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-03-26T06:35:58.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:35:58 smithi149 ceph-mon[36681]: pgmap v406: 97 pgs: 97 active+clean; 216 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-03-26T06:35:58.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:35:58 smithi037 ceph-mon[31974]: pgmap v406: 97 pgs: 97 active+clean; 216 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-03-26T06:35:58.533 INFO:tasks.workunit.client.0.smithi037.stdout: 1 527456 4.28 MB/sec execute 573 sec latency 1335.838 ms 2024-03-26T06:35:59.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 527456 4.28 MB/sec execute 574 sec latency 2335.975 ms 2024-03-26T06:36:00.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:00 smithi066 ceph-mon[36504]: pgmap v407: 97 pgs: 97 active+clean; 216 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-03-26T06:36:00.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:00 smithi149 ceph-mon[36681]: pgmap v407: 97 pgs: 97 active+clean; 216 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-03-26T06:36:00.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:00 smithi037 ceph-mon[31974]: pgmap v407: 97 pgs: 97 active+clean; 216 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-03-26T06:36:00.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 528048 4.27 MB/sec execute 575 sec latency 2991.933 ms 2024-03-26T06:36:01.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 529565 4.28 MB/sec execute 576 sec latency 26.902 ms 2024-03-26T06:36:02.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:02 smithi066 ceph-mon[36504]: pgmap v408: 97 pgs: 97 active+clean; 212 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 284 op/s 2024-03-26T06:36:02.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:02 smithi149 ceph-mon[36681]: pgmap v408: 97 pgs: 97 active+clean; 212 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 284 op/s 2024-03-26T06:36:02.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:02 smithi037 ceph-mon[31974]: pgmap v408: 97 pgs: 97 active+clean; 212 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 284 op/s 2024-03-26T06:36:02.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 530975 4.29 MB/sec execute 577 sec latency 24.329 ms 2024-03-26T06:36:03.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 530999 4.28 MB/sec execute 578 sec latency 961.449 ms 2024-03-26T06:36:04.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:04 smithi066 ceph-mon[36504]: pgmap v409: 97 pgs: 97 active+clean; 212 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 2.2 MiB/s wr, 190 op/s 2024-03-26T06:36:04.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:04 smithi149 ceph-mon[36681]: pgmap v409: 97 pgs: 97 active+clean; 212 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 2.2 MiB/s wr, 190 op/s 2024-03-26T06:36:04.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:04 smithi037 ceph-mon[31974]: pgmap v409: 97 pgs: 97 active+clean; 212 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 2.2 MiB/s wr, 190 op/s 2024-03-26T06:36:04.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 530999 4.27 MB/sec execute 579 sec latency 1961.561 ms 2024-03-26T06:36:05.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 531834 4.27 MB/sec execute 580 sec latency 2611.273 ms 2024-03-26T06:36:06.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:06 smithi066 ceph-mon[36504]: pgmap v410: 97 pgs: 97 active+clean; 215 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 302 op/s 2024-03-26T06:36:06.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:06 smithi066 ceph-mon[36504]: osdmap e103: 12 total, 12 up, 12 in 2024-03-26T06:36:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:06 smithi149 ceph-mon[36681]: pgmap v410: 97 pgs: 97 active+clean; 215 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 302 op/s 2024-03-26T06:36:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:06 smithi149 ceph-mon[36681]: osdmap e103: 12 total, 12 up, 12 in 2024-03-26T06:36:06.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:06 smithi037 ceph-mon[31974]: pgmap v410: 97 pgs: 97 active+clean; 215 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 302 op/s 2024-03-26T06:36:06.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:06 smithi037 ceph-mon[31974]: osdmap e103: 12 total, 12 up, 12 in 2024-03-26T06:36:06.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 533435 4.28 MB/sec execute 581 sec latency 13.433 ms 2024-03-26T06:36:07.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 534579 4.28 MB/sec execute 582 sec latency 326.974 ms 2024-03-26T06:36:08.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:08 smithi066 ceph-mon[36504]: pgmap v412: 97 pgs: 97 active+clean; 215 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-03-26T06:36:08.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:08 smithi149 ceph-mon[36681]: pgmap v412: 97 pgs: 97 active+clean; 215 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-03-26T06:36:08.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:08 smithi037 ceph-mon[31974]: pgmap v412: 97 pgs: 97 active+clean; 215 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-03-26T06:36:08.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 534579 4.28 MB/sec execute 583 sec latency 1327.076 ms 2024-03-26T06:36:09.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 534579 4.27 MB/sec execute 584 sec latency 2327.113 ms 2024-03-26T06:36:10.348 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:10 smithi066 ceph-mon[36504]: pgmap v413: 97 pgs: 97 active+clean; 215 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-03-26T06:36:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:10 smithi149 ceph-mon[36681]: pgmap v413: 97 pgs: 97 active+clean; 215 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-03-26T06:36:10.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:10 smithi037 ceph-mon[31974]: pgmap v413: 97 pgs: 97 active+clean; 215 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-03-26T06:36:10.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 535356 4.26 MB/sec execute 585 sec latency 2984.098 ms 2024-03-26T06:36:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.667+0000 7fc453243640 1 -- 172.21.15.37:0/1154963932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c100440 msgr2=0x7fc44c1008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.667+0000 7fc453243640 1 --2- 172.21.15.37:0/1154963932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c100440 0x7fc44c1008a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009890 tx=0x7fc43c02bb70 comp rx=0 tx=0).stop 2024-03-26T06:36:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.667+0000 7fc453243640 1 -- 172.21.15.37:0/1154963932 shutdown_connections 2024-03-26T06:36:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.667+0000 7fc453243640 1 --2- 172.21.15.37:0/1154963932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc44c100de0 0x7fc44c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.667+0000 7fc453243640 1 --2- 172.21.15.37:0/1154963932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c100440 0x7fc44c1008a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.667+0000 7fc453243640 1 --2- 172.21.15.37:0/1154963932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc44c1051b0 0x7fc44c105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.668+0000 7fc453243640 1 -- 172.21.15.37:0/1154963932 >> 172.21.15.37:0/1154963932 conn(0x7fc44c077050 msgr2=0x7fc44c077460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.668+0000 7fc453243640 1 -- 172.21.15.37:0/1154963932 shutdown_connections 2024-03-26T06:36:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.668+0000 7fc453243640 1 -- 172.21.15.37:0/1154963932 wait complete. 2024-03-26T06:36:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.668+0000 7fc453243640 1 Processor -- start 2024-03-26T06:36:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.668+0000 7fc453243640 1 -- start start 2024-03-26T06:36:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.668+0000 7fc453243640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc44c100440 0x7fc44c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.668+0000 7fc453243640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc44c100de0 0x7fc44c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.668+0000 7fc453243640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c1051b0 0x7fc44c1968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.668+0000 7fc453243640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c110610 con 0x7fc44c100440 2024-03-26T06:36:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.668+0000 7fc453243640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c110490 con 0x7fc44c1051b0 2024-03-26T06:36:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.668+0000 7fc453243640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c110790 con 0x7fc44c100de0 2024-03-26T06:36:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc443fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc44c100de0 0x7fc44c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc450fdf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc44c100440 0x7fc44c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc4517e0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c1051b0 0x7fc44c1968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc443fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc44c100de0 0x7fc44c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:50582/0 (socket says 172.21.15.37:50582) 2024-03-26T06:36:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc443fff640 1 -- 172.21.15.37:0/1445826003 learned_addr learned my addr 172.21.15.37:0/1445826003 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc443fff640 1 -- 172.21.15.37:0/1445826003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c1051b0 msgr2=0x7fc44c1968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc443fff640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c1051b0 0x7fc44c1968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc443fff640 1 -- 172.21.15.37:0/1445826003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc44c100440 msgr2=0x7fc44c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc443fff640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc44c100440 0x7fc44c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc443fff640 1 -- 172.21.15.37:0/1445826003 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c197050 con 0x7fc44c100de0 2024-03-26T06:36:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc4517e0640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c1051b0 0x7fc44c1968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:36:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc450fdf640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc44c100440 0x7fc44c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:36:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc443fff640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc44c100de0 0x7fc44c19cc10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc43c02c0b0 tx=0x7fc43c004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc441ffb640 1 -- 172.21.15.37:0/1445826003 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c041020 con 0x7fc44c100de0 2024-03-26T06:36:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc453243640 1 -- 172.21.15.37:0/1445826003 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c1972e0 con 0x7fc44c100de0 2024-03-26T06:36:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc441ffb640 1 -- 172.21.15.37:0/1445826003 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc43c02c9f0 con 0x7fc44c100de0 2024-03-26T06:36:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.669+0000 7fc441ffb640 1 -- 172.21.15.37:0/1445826003 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c039040 con 0x7fc44c100de0 2024-03-26T06:36:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.670+0000 7fc453243640 1 -- 172.21.15.37:0/1445826003 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c1a3590 con 0x7fc44c100de0 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.672+0000 7fc441ffb640 1 -- 172.21.15.37:0/1445826003 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc43c044020 con 0x7fc44c100de0 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.672+0000 7fc455e19740 1 Processor -- start 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.672+0000 7fc455e19740 1 -- start start 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.672+0000 7fc455e19740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff13aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.672+0000 7fc455e19740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff13b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.672+0000 7fc455e19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff148b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc455e19740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ff09f5f0 con 0x55e3ff13bd40 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc455e19740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ff048930 con 0x55e3ff13afd0 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc455e19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ff09b690 con 0x55e3ff146760 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc4517e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff13aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc4517e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff13aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53306/0 (socket says 172.21.15.37:53306) 2024-03-26T06:36:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc4517e0640 1 -- 172.21.15.37:0/1477543001 learned_addr learned my addr 172.21.15.37:0/1477543001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc441ffb640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc4340a2a10 0x7fc4340a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc443fff640 1 --2- 172.21.15.37:0/1477543001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc450fdf640 1 --2- 172.21.15.37:0/1477543001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff13b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc4517e0640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc4340a2a10 0x7fc4340a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc4517e0640 1 -- 172.21.15.37:0/1477543001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 msgr2=0x55e3ff148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc4517e0640 1 --2- 172.21.15.37:0/1477543001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff148b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc4517e0640 1 -- 172.21.15.37:0/1477543001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 msgr2=0x55e3ff13b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc4517e0640 1 --2- 172.21.15.37:0/1477543001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff13b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc4517e0640 1 -- 172.21.15.37:0/1477543001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fedb4210 con 0x55e3ff13bd40 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc441ffb640 1 -- 172.21.15.37:0/1445826003 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc43c0f06d0 con 0x7fc44c100de0 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc4517e0640 1 --2- 172.21.15.37:0/1477543001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff13aa90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc444009800 tx=0x7fc4440147e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468a963ccef1b1f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc432ffd640 1 -- 172.21.15.37:0/1477543001 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44400e450 con 0x55e3ff13bd40 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.673+0000 7fc432ffd640 1 -- 172.21.15.37:0/1477543001 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc44400ead0 con 0x55e3ff13bd40 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.674+0000 7fc432ffd640 1 -- 172.21.15.37:0/1477543001 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44401f720 con 0x55e3ff13bd40 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.674+0000 7fc455e19740 1 -- 172.21.15.37:0/1477543001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 msgr2=0x55e3ff13aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.674+0000 7fc4517e0640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc4340a2a10 0x7fc4340a4ed0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc444023000 tx=0x7fc444021000 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.674+0000 7fc455e19740 1 --2- 172.21.15.37:0/1477543001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff13aa90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc444009800 tx=0x7fc4440147e0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.674+0000 7fc455e19740 1 -- 172.21.15.37:0/1477543001 shutdown_connections 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.674+0000 7fc455e19740 1 --2- 172.21.15.37:0/1477543001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.674+0000 7fc455e19740 1 --2- 172.21.15.37:0/1477543001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff13b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.674+0000 7fc455e19740 1 --2- 172.21.15.37:0/1477543001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff13aa90 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.674+0000 7fc455e19740 1 -- 172.21.15.37:0/1477543001 >> 172.21.15.37:0/1477543001 conn(0x55e3ff1025b0 msgr2=0x55e3ff135cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.674+0000 7fc455e19740 1 -- 172.21.15.37:0/1477543001 shutdown_connections 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.674+0000 7fc455e19740 1 -- 172.21.15.37:0/1477543001 wait complete. 2024-03-26T06:36:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc455e19740 1 -- start start 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc455e19740 1 Processor -- start 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc455e19740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff1437c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc455e19740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff143d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc455e19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff144240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc455e19740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ff149490 con 0x55e3ff13bd40 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc455e19740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ff05bd90 con 0x55e3ff13afd0 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc455e19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ff149610 con 0x55e3ff146760 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc455e19740 10 client.0 _ll_register_callbacks cb 0x55e3fedb3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc443fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc450fdf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff1437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc443fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff143d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53312/0 (socket says 172.21.15.37:53312) 2024-03-26T06:36:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc450fdf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff1437c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:60680/0 (socket says 172.21.15.37:60680) 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc443fff640 1 -- 172.21.15.37:0/4281097715 learned_addr learned my addr 172.21.15.37:0/4281097715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.675+0000 7fc4517e0640 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc443fff640 1 -- 172.21.15.37:0/4281097715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 msgr2=0x55e3ff144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc443fff640 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff144240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc443fff640 1 -- 172.21.15.37:0/4281097715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 msgr2=0x55e3ff1437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc443fff640 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff1437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc443fff640 1 -- 172.21.15.37:0/4281097715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3fee80620 con 0x55e3ff13bd40 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc450fdf640 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff1437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc443fff640 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff143d00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc43c03d040 tx=0x7fc43c034780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d023cf8e822f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc42bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc42bfff640 1 -- 172.21.15.37:0/4281097715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c034890 con 0x55e3ff13bd40 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc42bfff640 1 -- 172.21.15.37:0/4281097715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc43c0c5130 con 0x55e3ff13bd40 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc455e19740 1 -- 172.21.15.37:0/4281097715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc44400ead0 con 0x55e3ff13bd40 2024-03-26T06:36:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc42bfff640 1 -- 172.21.15.37:0/4281097715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc43c0bd520 con 0x55e3ff13bd40 2024-03-26T06:36:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc455e19740 10 client.16578 fetch_fsmap learned FSMap version 54 2024-03-26T06:36:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc455e19740 1 -- 172.21.15.37:0/4281097715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55e3ff149490 con 0x55e3ff13bd40 2024-03-26T06:36:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc42bfff640 1 -- 172.21.15.37:0/4281097715 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc44400ead0 con 0x55e3ff13bd40 2024-03-26T06:36:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc42bfff640 1 -- 172.21.15.37:0/4281097715 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fc43c03c030 con 0x55e3ff13bd40 2024-03-26T06:36:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc455e19740 10 client.16578 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:36:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc455e19740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:36:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc455e19740 20 client.16578 populate_metadata read hostname 'smithi037' 2024-03-26T06:36:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc455e19740 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e3ff156a40 0x55e3ff158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc455e19740 4 client.16578 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.676+0000 7fc455e19740 1 -- 172.21.15.37:0/4281097715 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3ff0acb30 con 0x55e3ff156a40 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.677+0000 7fc450fdf640 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e3ff156a40 0x55e3ff158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.677+0000 7fc42bfff640 10 client.16578 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.677+0000 7fc450fdf640 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e3ff156a40 0x55e3ff158ea0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb87f8c8a9a04730 server_cookie=d02b95a07a2dc7ea in_seq=0 out_seq=0 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc42bfff640 1 -- 172.21.15.37:0/4281097715 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e3ff0acb30 con 0x55e3ff156a40 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc42bfff640 10 client.16578 handle_command_reply: tid=0 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 1 client.16578 shutdown 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 1 -- 172.21.15.37:0/4281097715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 msgr2=0x55e3ff143d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff143d00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc43c03d040 tx=0x7fc43c034780 comp rx=0 tx=0).stop 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 1 -- 172.21.15.37:0/4281097715 shutdown_connections 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e3ff156a40 0x55e3ff158ea0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13bd40 0x55e3ff143d00 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc42bfff640 0 client.16578 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 1 --2- 172.21.15.37:0/4281097715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff1437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 1 -- 172.21.15.37:0/4281097715 >> 172.21.15.37:0/4281097715 conn(0x55e3ff1025b0 msgr2=0x55e3ff137a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 1 -- 172.21.15.37:0/4281097715 shutdown_connections 2024-03-26T06:36:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 1 -- 172.21.15.37:0/4281097715 wait complete. 2024-03-26T06:36:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.678+0000 7fc455e19740 20 client.16578 trim_cache size 0 max 16384 2024-03-26T06:36:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc455e19740 1 Processor -- start 2024-03-26T06:36:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc455e19740 1 -- start start 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc455e19740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13afd0 0x55e3ff145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc455e19740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13bd40 0x55e3ff140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc455e19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc455e19740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1aaa00 con 0x55e3ff13afd0 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc455e19740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1ac560 con 0x55e3ff13bd40 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc455e19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1ab060 con 0x55e3ff146760 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc450fdf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13afd0 0x55e3ff145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc450fdf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13afd0 0x55e3ff145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53318/0 (socket says 172.21.15.37:53318) 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc450fdf640 1 -- 172.21.15.37:0/2103110183 learned_addr learned my addr 172.21.15.37:0/2103110183 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc443fff640 1 --2- 172.21.15.37:0/2103110183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13bd40 0x55e3ff140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc4517e0640 1 --2- 172.21.15.37:0/2103110183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc450fdf640 1 -- 172.21.15.37:0/2103110183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 msgr2=0x55e3ff140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc450fdf640 1 --2- 172.21.15.37:0/2103110183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc450fdf640 1 -- 172.21.15.37:0/2103110183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13bd40 msgr2=0x55e3ff140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc450fdf640 1 --2- 172.21.15.37:0/2103110183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13bd40 0x55e3ff140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc450fdf640 1 -- 172.21.15.37:0/2103110183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3ff13a2a0 con 0x55e3ff13afd0 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.692+0000 7fc4517e0640 1 --2- 172.21.15.37:0/2103110183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.693+0000 7fc450fdf640 1 --2- 172.21.15.37:0/2103110183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13afd0 0x55e3ff145ef0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0ff940 tx=0x7fc448009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c80284bd1acf60b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.693+0000 7fc4327fc640 1 -- 172.21.15.37:0/2103110183 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44800fca0 con 0x55e3ff13afd0 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.693+0000 7fc4327fc640 1 -- 172.21.15.37:0/2103110183 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc448025030 con 0x55e3ff13afd0 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.693+0000 7fc4327fc640 1 -- 172.21.15.37:0/2103110183 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448004d90 con 0x55e3ff13afd0 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.693+0000 7fc455e19740 1 -- 172.21.15.37:0/2103110183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13afd0 msgr2=0x55e3ff145ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.693+0000 7fc455e19740 1 --2- 172.21.15.37:0/2103110183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13afd0 0x55e3ff145ef0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0ff940 tx=0x7fc448009260 comp rx=0 tx=0).stop 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 -- 172.21.15.37:0/2103110183 shutdown_connections 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 --2- 172.21.15.37:0/2103110183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff146760 0x55e3ff140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 --2- 172.21.15.37:0/2103110183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13bd40 0x55e3ff140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 --2- 172.21.15.37:0/2103110183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff13afd0 0x55e3ff145ef0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 -- 172.21.15.37:0/2103110183 >> 172.21.15.37:0/2103110183 conn(0x55e3ff1025b0 msgr2=0x55e3ff142090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 -- 172.21.15.37:0/2103110183 shutdown_connections 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 -- 172.21.15.37:0/2103110183 wait complete. 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 -- start start 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 Processor -- start 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff14c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff13bd40 0x55e3ff14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff146760 0x55e3ff1acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1abe40 con 0x55e3ff146760 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1ab780 con 0x55e3ff13afd0 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1a8cc0 con 0x55e3ff13bd40 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc455e19740 10 client.0 _ll_register_callbacks cb 0x7fc444009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:36:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc450fdf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff14c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc4517e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff146760 0x55e3ff1acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.694+0000 7fc450fdf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff14c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:60692/0 (socket says 172.21.15.37:60692) 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc450fdf640 1 -- 172.21.15.37:0/3991948296 learned_addr learned my addr 172.21.15.37:0/3991948296 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc443fff640 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff13bd40 0x55e3ff14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc450fdf640 1 -- 172.21.15.37:0/3991948296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff13bd40 msgr2=0x55e3ff14cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc450fdf640 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff13bd40 0x55e3ff14cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc450fdf640 1 -- 172.21.15.37:0/3991948296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff146760 msgr2=0x55e3ff1acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc450fdf640 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff146760 0x55e3ff1acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc450fdf640 1 -- 172.21.15.37:0/3991948296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3ff1ad4d0 con 0x55e3ff13afd0 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc450fdf640 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff14c710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc448024040 tx=0x7fc448012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76ee6ef363b46f20 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc42bfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc42bfff640 1 -- 172.21.15.37:0/3991948296 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44800fea0 con 0x55e3ff13afd0 2024-03-26T06:36:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc42bfff640 1 -- 172.21.15.37:0/3991948296 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc448004b10 con 0x55e3ff13afd0 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc455e19740 1 -- 172.21.15.37:0/3991948296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc448025030 con 0x55e3ff13afd0 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc42bfff640 1 -- 172.21.15.37:0/3991948296 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc448004d10 con 0x55e3ff13afd0 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc455e19740 10 client.26105 fetch_fsmap learned FSMap version 54 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.695+0000 7fc455e19740 1 -- 172.21.15.37:0/3991948296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55e3ff13c4f0 con 0x55e3ff13afd0 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.696+0000 7fc42bfff640 1 -- 172.21.15.37:0/3991948296 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc448025030 con 0x55e3ff13afd0 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.696+0000 7fc42bfff640 1 -- 172.21.15.37:0/3991948296 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fc4480215e0 con 0x55e3ff13afd0 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.696+0000 7fc455e19740 10 client.26105 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.696+0000 7fc455e19740 10 client.26105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.696+0000 7fc455e19740 20 client.26105 populate_metadata read hostname 'smithi037' 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.696+0000 7fc455e19740 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e3ff13cc80 0x55e3ff13d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.696+0000 7fc455e19740 4 client.26105 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.696+0000 7fc455e19740 1 -- 172.21.15.37:0/3991948296 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e3ff0a6e80 con 0x55e3ff13cc80 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.696+0000 7fc443fff640 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e3ff13cc80 0x55e3ff13d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.696+0000 7fc443fff640 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e3ff13cc80 0x55e3ff13d090 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a595a1036ea80c8d server_cookie=a416f2ed2cf43503 in_seq=0 out_seq=0 2024-03-26T06:36:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.696+0000 7fc42bfff640 10 client.26105 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc42bfff640 1 -- 172.21.15.37:0/3991948296 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e3ff0a6e80 con 0x55e3ff13cc80 2024-03-26T06:36:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc42bfff640 10 client.26105 handle_command_reply: tid=0 2024-03-26T06:36:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 1 client.26105 shutdown 2024-03-26T06:36:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 1 -- 172.21.15.37:0/3991948296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 msgr2=0x55e3ff14c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff14c710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc448024040 tx=0x7fc448012b00 comp rx=0 tx=0).stop 2024-03-26T06:36:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 1 -- 172.21.15.37:0/3991948296 shutdown_connections 2024-03-26T06:36:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e3ff13cc80 0x55e3ff13d090 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ff146760 0x55e3ff1acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ff13bd40 0x55e3ff14cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc42bfff640 0 client.26105 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 1 --2- 172.21.15.37:0/3991948296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e3ff13afd0 0x55e3ff14c710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 1 -- 172.21.15.37:0/3991948296 >> 172.21.15.37:0/3991948296 conn(0x55e3ff1025b0 msgr2=0x55e3ff1345b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 1 -- 172.21.15.37:0/3991948296 shutdown_connections 2024-03-26T06:36:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 1 -- 172.21.15.37:0/3991948296 wait complete. 2024-03-26T06:36:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.734+0000 7fc455e19740 20 client.26105 trim_cache size 0 max 16384 2024-03-26T06:36:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:36:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:36:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:36:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:36:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 -- 172.21.15.37:0/1445826003 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc4340a2a10 msgr2=0x7fc4340a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc4340a2a10 0x7fc4340a4ed0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc444023000 tx=0x7fc444021000 comp rx=0 tx=0).stop 2024-03-26T06:36:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 -- 172.21.15.37:0/1445826003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc44c100de0 msgr2=0x7fc44c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc44c100de0 0x7fc44c19cc10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc43c02c0b0 tx=0x7fc43c004430 comp rx=0 tx=0).stop 2024-03-26T06:36:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 -- 172.21.15.37:0/1445826003 shutdown_connections 2024-03-26T06:36:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc4340a2a10 0x7fc4340a4ed0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc44c1051b0 0x7fc44c1968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc44c100de0 0x7fc44c19cc10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 --2- 172.21.15.37:0/1445826003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc44c100440 0x7fc44c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 -- 172.21.15.37:0/1445826003 >> 172.21.15.37:0/1445826003 conn(0x7fc44c077050 msgr2=0x7fc44c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 -- 172.21.15.37:0/1445826003 shutdown_connections 2024-03-26T06:36:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:10.735+0000 7fc453243640 1 -- 172.21.15.37:0/1445826003 wait complete. 2024-03-26T06:36:11.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 536957 4.28 MB/sec execute 586 sec latency 19.018 ms 2024-03-26T06:36:12.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:12 smithi066 ceph-mon[36504]: pgmap v414: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-03-26T06:36:12.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:12 smithi149 ceph-mon[36681]: pgmap v414: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-03-26T06:36:12.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:12 smithi037 ceph-mon[31974]: pgmap v414: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-03-26T06:36:12.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 538122 4.28 MB/sec execute 587 sec latency 295.915 ms 2024-03-26T06:36:13.534 INFO:tasks.workunit.client.0.smithi037.stdout: 1 538122 4.27 MB/sec execute 588 sec latency 1295.961 ms 2024-03-26T06:36:14.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:14 smithi066 ceph-mon[36504]: pgmap v415: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-03-26T06:36:14.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:14 smithi149 ceph-mon[36681]: pgmap v415: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-03-26T06:36:14.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:14 smithi037 ceph-mon[31974]: pgmap v415: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-03-26T06:36:14.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 538122 4.26 MB/sec execute 589 sec latency 2296.030 ms 2024-03-26T06:36:15.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 538898 4.26 MB/sec execute 590 sec latency 2958.147 ms 2024-03-26T06:36:16.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:15 smithi066 ceph-mon[36504]: osdmap e104: 12 total, 12 up, 12 in 2024-03-26T06:36:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:15 smithi149 ceph-mon[36681]: osdmap e104: 12 total, 12 up, 12 in 2024-03-26T06:36:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:15 smithi037 ceph-mon[31974]: osdmap e104: 12 total, 12 up, 12 in 2024-03-26T06:36:16.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 540533 4.27 MB/sec execute 591 sec latency 21.683 ms 2024-03-26T06:36:17.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:16 smithi066 ceph-mon[36504]: pgmap v417: 97 pgs: 97 active+clean; 217 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-03-26T06:36:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:16 smithi149 ceph-mon[36681]: pgmap v417: 97 pgs: 97 active+clean; 217 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-03-26T06:36:17.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:16 smithi037 ceph-mon[31974]: pgmap v417: 97 pgs: 97 active+clean; 217 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-03-26T06:36:17.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 541671 4.27 MB/sec execute 592 sec latency 346.991 ms 2024-03-26T06:36:18.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 541671 4.27 MB/sec execute 593 sec latency 1347.066 ms 2024-03-26T06:36:19.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:18 smithi066 ceph-mon[36504]: pgmap v418: 97 pgs: 97 active+clean; 217 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-03-26T06:36:19.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:18 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:36:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:18 smithi149 ceph-mon[36681]: pgmap v418: 97 pgs: 97 active+clean; 217 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-03-26T06:36:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:18 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:36:19.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:18 smithi037 ceph-mon[31974]: pgmap v418: 97 pgs: 97 active+clean; 217 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-03-26T06:36:19.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:18 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:36:19.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 541671 4.26 MB/sec execute 594 sec latency 2347.124 ms 2024-03-26T06:36:20.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:19 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:36:20.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:19 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:36:20.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:19 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:36:20.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:19 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:36:20.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:19 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:36:20.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:19 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:36:20.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:19 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:36:20.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:19 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:36:20.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:19 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:36:20.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 542388 4.26 MB/sec execute 595 sec latency 3003.054 ms 2024-03-26T06:36:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:20 smithi066 ceph-mon[36504]: pgmap v419: 97 pgs: 97 active+clean; 217 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-03-26T06:36:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:20 smithi149 ceph-mon[36681]: pgmap v419: 97 pgs: 97 active+clean; 217 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-03-26T06:36:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:20 smithi037 ceph-mon[31974]: pgmap v419: 97 pgs: 97 active+clean; 217 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-03-26T06:36:21.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 543979 4.27 MB/sec execute 596 sec latency 15.669 ms 2024-03-26T06:36:22.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 545212 4.27 MB/sec execute 597 sec latency 221.142 ms 2024-03-26T06:36:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:22 smithi066 ceph-mon[36504]: pgmap v420: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-03-26T06:36:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:22 smithi149 ceph-mon[36681]: pgmap v420: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-03-26T06:36:23.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:22 smithi037 ceph-mon[31974]: pgmap v420: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-03-26T06:36:23.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 545212 4.26 MB/sec execute 598 sec latency 1221.271 ms 2024-03-26T06:36:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.836+0000 7f6fb7110640 1 -- 172.21.15.37:0/2740740430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb0108d50 msgr2=0x7f6fb010c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.836+0000 7f6fb7110640 1 --2- 172.21.15.37:0/2740740430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb0108d50 0x7f6fb010c570 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009900 tx=0x7f6f9c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:36:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.836+0000 7f6fb7110640 1 -- 172.21.15.37:0/2740740430 shutdown_connections 2024-03-26T06:36:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.836+0000 7f6fb7110640 1 --2- 172.21.15.37:0/2740740430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb010cc50 0x7f6fb0114800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.836+0000 7f6fb7110640 1 --2- 172.21.15.37:0/2740740430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb0108d50 0x7f6fb010c570 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.836+0000 7f6fb7110640 1 --2- 172.21.15.37:0/2740740430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb01083a0 0x7f6fb0108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.836+0000 7f6fb7110640 1 -- 172.21.15.37:0/2740740430 >> 172.21.15.37:0/2740740430 conn(0x7f6fb0077050 msgr2=0x7f6fb0079470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.836+0000 7f6fb7110640 1 -- 172.21.15.37:0/2740740430 shutdown_connections 2024-03-26T06:36:23.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.836+0000 7f6fb7110640 1 -- 172.21.15.37:0/2740740430 wait complete. 2024-03-26T06:36:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb7110640 1 Processor -- start 2024-03-26T06:36:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb7110640 1 -- start start 2024-03-26T06:36:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb7110640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb01083a0 0x7f6fb019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb7110640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb010cc50 0x7f6fb019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb7110640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb01a2a00 0x7f6fb01a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb7110640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb0117d30 con 0x7f6fb01a2a00 2024-03-26T06:36:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb7110640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb0117bb0 con 0x7f6fb010cc50 2024-03-26T06:36:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb7110640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb0117eb0 con 0x7f6fb01083a0 2024-03-26T06:36:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb56ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb01a2a00 0x7f6fb01a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb56ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb01a2a00 0x7f6fb01a2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58586/0 (socket says 172.21.15.37:58586) 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb56ad640 1 -- 172.21.15.37:0/802872531 learned_addr learned my addr 172.21.15.37:0/802872531 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6faffff640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb010cc50 0x7f6fb019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.837+0000 7f6fb4eac640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb01083a0 0x7f6fb019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6fb56ad640 1 -- 172.21.15.37:0/802872531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb01083a0 msgr2=0x7f6fb019e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6fb56ad640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb01083a0 0x7f6fb019e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6fb56ad640 1 -- 172.21.15.37:0/802872531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb010cc50 msgr2=0x7f6fb019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6fb56ad640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb010cc50 0x7f6fb019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6fb56ad640 1 -- 172.21.15.37:0/802872531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb01a34f0 con 0x7f6fb01a2a00 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6faffff640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb010cc50 0x7f6fb019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6fb56ad640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb01a2a00 0x7f6fb01a2eb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800eb10 tx=0x7f6fa800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6fadffb640 1 -- 172.21.15.37:0/802872531 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8004b20 con 0x7f6fb01a2a00 2024-03-26T06:36:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6fb7110640 1 -- 172.21.15.37:0/802872531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb01a37e0 con 0x7f6fb01a2a00 2024-03-26T06:36:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6fadffb640 1 -- 172.21.15.37:0/802872531 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fa8004cc0 con 0x7f6fb01a2a00 2024-03-26T06:36:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6fadffb640 1 -- 172.21.15.37:0/802872531 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8010760 con 0x7f6fb01a2a00 2024-03-26T06:36:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.838+0000 7f6fb7110640 1 -- 172.21.15.37:0/802872531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb01aafa0 con 0x7f6fb01a2a00 2024-03-26T06:36:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.842+0000 7f6fb9dcd740 1 Processor -- start 2024-03-26T06:36:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.842+0000 7f6fadffb640 1 -- 172.21.15.37:0/802872531 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f6fa8005020 con 0x7f6fb01a2a00 2024-03-26T06:36:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.843+0000 7f6fb9dcd740 1 -- start start 2024-03-26T06:36:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.843+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e51623d40 0x558e51622a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.843+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 0x558e51623450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.843+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e5162e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.843+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e515875f0 con 0x558e5162c550 2024-03-26T06:36:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.843+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e51530930 con 0x558e51622fd0 2024-03-26T06:36:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.843+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e51583690 con 0x558e51623d40 2024-03-26T06:36:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.843+0000 7f6fadffb640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6fa00a2a10 0x7f6fa00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.844+0000 7f6faffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e5162e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.844+0000 7f6faffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e5162e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58592/0 (socket says 172.21.15.37:58592) 2024-03-26T06:36:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.844+0000 7f6faffff640 1 -- 172.21.15.37:0/2792028859 learned_addr learned my addr 172.21.15.37:0/2792028859 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.844+0000 7f6faffff640 1 --2- 172.21.15.37:0/2792028859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e51623d40 0x558e51622a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.844+0000 7f6fadffb640 1 -- 172.21.15.37:0/802872531 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6fa80cf370 con 0x7f6fb01a2a00 2024-03-26T06:36:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.844+0000 7f6fb56ad640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6fa00a2a10 0x7f6fa00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.844+0000 7f6faffff640 1 -- 172.21.15.37:0/2792028859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e51623d40 msgr2=0x558e51622a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.844+0000 7f6faffff640 1 --2- 172.21.15.37:0/2792028859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e51623d40 0x558e51622a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.844+0000 7f6faffff640 1 -- 172.21.15.37:0/2792028859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 msgr2=0x558e51623450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.844+0000 7f6faffff640 1 --2- 172.21.15.37:0/2792028859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 0x558e51623450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.844+0000 7f6faffff640 1 -- 172.21.15.37:0/2792028859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e5129c210 con 0x558e5162c550 2024-03-26T06:36:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.845+0000 7f6faffff640 1 --2- 172.21.15.37:0/2792028859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e5162e940 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c000c00 tx=0x7f6f9c004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e669d70cc3bbd18 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.845+0000 7f6fb56ad640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6fa00a2a10 0x7f6fa00a4ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80ccb70 tx=0x7f6fa8005b40 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.845+0000 7f6f9affd640 1 -- 172.21.15.37:0/2792028859 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c047020 con 0x558e5162c550 2024-03-26T06:36:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.845+0000 7f6f9affd640 1 -- 172.21.15.37:0/2792028859 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f9c004c00 con 0x558e5162c550 2024-03-26T06:36:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.845+0000 7f6f9affd640 1 -- 172.21.15.37:0/2792028859 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0051e0 con 0x558e5162c550 2024-03-26T06:36:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.846+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2792028859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 msgr2=0x558e5162e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.846+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/2792028859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e5162e940 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c000c00 tx=0x7f6f9c004e50 comp rx=0 tx=0).stop 2024-03-26T06:36:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.846+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2792028859 shutdown_connections 2024-03-26T06:36:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.846+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/2792028859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e5162e940 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.846+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/2792028859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 0x558e51623450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.846+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/2792028859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e51623d40 0x558e51622a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.846+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2792028859 >> 172.21.15.37:0/2792028859 conn(0x558e515ea5b0 msgr2=0x558e5161dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.846+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2792028859 shutdown_connections 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.846+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2792028859 wait complete. 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.847+0000 7f6fb9dcd740 1 -- start start 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.847+0000 7f6fb9dcd740 1 Processor -- start 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.847+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51622fd0 0x558e51629680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.847+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51623d40 0x558e51629bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.847+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e5162c550 0x558e5162a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.847+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5162f220 con 0x558e51622fd0 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.847+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e51543d90 con 0x558e51623d40 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.847+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5162f3a0 con 0x558e5162c550 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.847+0000 7f6fb9dcd740 10 client.0 _ll_register_callbacks cb 0x558e5129ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6faffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51622fd0 0x558e51629680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6faffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51622fd0 0x558e51629680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58608/0 (socket says 172.21.15.37:58608) 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6fb4eac640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51623d40 0x558e51629bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6faffff640 1 -- 172.21.15.37:0/2252482863 learned_addr learned my addr 172.21.15.37:0/2252482863 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6faffff640 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e5162c550 0x558e5162a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6faffff640 1 -- 172.21.15.37:0/2252482863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e5162c550 msgr2=0x558e5162a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6faffff640 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e5162c550 0x558e5162a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6faffff640 1 -- 172.21.15.37:0/2252482863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51623d40 msgr2=0x558e51629bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6faffff640 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51623d40 0x558e51629bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6faffff640 1 -- 172.21.15.37:0/2252482863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e51368620 con 0x558e51622fd0 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6fb4eac640 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51623d40 0x558e51629bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6faffff640 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51622fd0 0x558e51629680 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x558e5161eb20 tx=0x7f6f9c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97fb6b4596cadad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6f8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6f8ffff640 1 -- 172.21.15.37:0/2252482863 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c047030 con 0x558e51622fd0 2024-03-26T06:36:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6f8ffff640 1 -- 172.21.15.37:0/2252482863 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6f9c005180 con 0x558e51622fd0 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.848+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2252482863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f9c004c00 con 0x558e51622fd0 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6f8ffff640 1 -- 172.21.15.37:0/2252482863 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f9c005380 con 0x558e51622fd0 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6fb9dcd740 10 client.16596 fetch_fsmap learned FSMap version 54 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2252482863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x558e5163b610 con 0x558e51622fd0 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6f8ffff640 1 -- 172.21.15.37:0/2252482863 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f9c004c00 con 0x558e51622fd0 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6f8ffff640 1 -- 172.21.15.37:0/2252482863 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f6f9c003cd0 con 0x558e51622fd0 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6fb9dcd740 10 client.16596 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6fb9dcd740 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6fb9dcd740 20 client.16596 populate_metadata read hostname 'smithi037' 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558e5163c960 0x558e5163ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6fb9dcd740 4 client.16596 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:36:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2252482863 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e51594b30 con 0x558e5163c960 2024-03-26T06:36:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.849+0000 7f6fb4eac640 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558e5163c960 0x558e5163ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6f8ffff640 10 client.16596 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:23.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6fb4eac640 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558e5163c960 0x558e5163ed50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b599e19b3cf090eb server_cookie=c8f186dcadd72fd8 in_seq=0 out_seq=0 2024-03-26T06:36:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6f8ffff640 1 -- 172.21.15.37:0/2252482863 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e51594b30 con 0x558e5163c960 2024-03-26T06:36:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6f8ffff640 10 client.16596 handle_command_reply: tid=0 2024-03-26T06:36:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6fb9dcd740 1 client.16596 shutdown 2024-03-26T06:36:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2252482863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51622fd0 msgr2=0x558e51629680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51622fd0 0x558e51629680 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x558e5161eb20 tx=0x7f6f9c005b50 comp rx=0 tx=0).stop 2024-03-26T06:36:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2252482863 shutdown_connections 2024-03-26T06:36:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558e5163c960 0x558e5163ed50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e5162c550 0x558e5162a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51623d40 0x558e51629bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6f8ffff640 0 client.16596 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/2252482863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51622fd0 0x558e51629680 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.850+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2252482863 >> 172.21.15.37:0/2252482863 conn(0x558e515ea5b0 msgr2=0x558e5161fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.851+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2252482863 shutdown_connections 2024-03-26T06:36:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.851+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/2252482863 wait complete. 2024-03-26T06:36:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.851+0000 7f6fb9dcd740 20 client.16596 trim_cache size 0 max 16384 2024-03-26T06:36:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.864+0000 7f6fb9dcd740 1 Processor -- start 2024-03-26T06:36:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.864+0000 7f6fb9dcd740 1 -- start start 2024-03-26T06:36:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.864+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 0x558e5162bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.864+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e51623d40 0x558e51633610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.864+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e51633b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.864+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb01b3230 con 0x558e5162c550 2024-03-26T06:36:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.864+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb01b4d90 con 0x558e51622fd0 2024-03-26T06:36:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.864+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb01b3890 con 0x558e51623d40 2024-03-26T06:36:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.864+0000 7f6faffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e51633b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.864+0000 7f6faffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e51633b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58612/0 (socket says 172.21.15.37:58612) 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.864+0000 7f6faffff640 1 -- 172.21.15.37:0/1841319875 learned_addr learned my addr 172.21.15.37:0/1841319875 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.865+0000 7f6fb4eac640 1 --2- 172.21.15.37:0/1841319875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e51623d40 0x558e51633610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.865+0000 7f6faffff640 1 --2- 172.21.15.37:0/1841319875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 0x558e5162bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.865+0000 7f6faffff640 1 -- 172.21.15.37:0/1841319875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e51623d40 msgr2=0x558e51633610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.865+0000 7f6faffff640 1 --2- 172.21.15.37:0/1841319875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e51623d40 0x558e51633610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.865+0000 7f6faffff640 1 -- 172.21.15.37:0/1841319875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 msgr2=0x558e5162bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.865+0000 7f6faffff640 1 --2- 172.21.15.37:0/1841319875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 0x558e5162bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.865+0000 7f6faffff640 1 -- 172.21.15.37:0/1841319875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e516222a0 con 0x558e5162c550 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.865+0000 7f6faffff640 1 --2- 172.21.15.37:0/1841319875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e51633b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005c40 tx=0x7f6f9c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d96f107aaf9d8da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.865+0000 7f6f9a7fc640 1 -- 172.21.15.37:0/1841319875 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c047040 con 0x558e5162c550 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.865+0000 7f6f9a7fc640 1 -- 172.21.15.37:0/1841319875 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f9c009520 con 0x558e5162c550 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.865+0000 7f6f9a7fc640 1 -- 172.21.15.37:0/1841319875 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c02c5d0 con 0x558e5162c550 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.866+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1841319875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 msgr2=0x558e51633b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.866+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/1841319875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e51633b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005c40 tx=0x7f6f9c005520 comp rx=0 tx=0).stop 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.866+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1841319875 shutdown_connections 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.866+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/1841319875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5162c550 0x558e51633b50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.866+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/1841319875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e51623d40 0x558e51633610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.866+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/1841319875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 0x558e5162bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.866+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1841319875 >> 172.21.15.37:0/1841319875 conn(0x558e515ea5b0 msgr2=0x558e51628f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.866+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1841319875 shutdown_connections 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.866+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1841319875 wait complete. 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.866+0000 7f6fb9dcd740 1 -- start start 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.866+0000 7f6fb9dcd740 1 Processor -- start 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 0x558e51633620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51623d40 0x558e516259d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb9dcd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e5162c550 0x558e51625f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb01b4670 con 0x558e51623d40 2024-03-26T06:36:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb01b3fb0 con 0x558e51622fd0 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb9dcd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb01b14f0 con 0x558e5162c550 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb9dcd740 10 client.0 _ll_register_callbacks cb 0x7f6fa4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb4eac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51623d40 0x558e516259d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb4eac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51623d40 0x558e516259d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58628/0 (socket says 172.21.15.37:58628) 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb4eac640 1 -- 172.21.15.37:0/1153476679 learned_addr learned my addr 172.21.15.37:0/1153476679 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6faffff640 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 0x558e51633620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6faffff640 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e5162c550 0x558e51625f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb4eac640 1 -- 172.21.15.37:0/1153476679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e5162c550 msgr2=0x558e51625f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb4eac640 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e5162c550 0x558e51625f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb4eac640 1 -- 172.21.15.37:0/1153476679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 msgr2=0x558e51633620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb4eac640 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 0x558e51633620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb4eac640 1 -- 172.21.15.37:0/1153476679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e51633d60 con 0x558e51623d40 2024-03-26T06:36:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb4eac640 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51623d40 0x558e516259d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009f70 tx=0x7f6fa401e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badd952be7656f96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6f8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6f8ffff640 1 -- 172.21.15.37:0/1153476679 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa401e8e0 con 0x558e51623d40 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6f8ffff640 1 -- 172.21.15.37:0/1153476679 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6fa4010430 con 0x558e51623d40 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.867+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1153476679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f9c009520 con 0x558e51623d40 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6f8ffff640 1 -- 172.21.15.37:0/1153476679 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fa4010630 con 0x558e51623d40 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6fb9dcd740 10 client.16602 fetch_fsmap learned FSMap version 54 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1153476679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x558e51693630 con 0x558e51623d40 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6f8ffff640 1 -- 172.21.15.37:0/1153476679 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f9c009520 con 0x558e51623d40 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6f8ffff640 1 -- 172.21.15.37:0/1153476679 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f6fa4025070 con 0x558e51623d40 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6fb9dcd740 10 client.16602 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6fb9dcd740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6fb9dcd740 20 client.16602 populate_metadata read hostname 'smithi037' 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558e51693e00 0x558e51696220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6fb9dcd740 4 client.16602 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:36:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1153476679 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558e5158ee80 con 0x558e51693e00 2024-03-26T06:36:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.868+0000 7f6faffff640 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558e51693e00 0x558e51696220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.869+0000 7f6faffff640 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558e51693e00 0x558e51696220 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=767b271f44a02f62 server_cookie=6e62d4f47b8c65a6 in_seq=0 out_seq=0 2024-03-26T06:36:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.869+0000 7f6f8ffff640 10 client.16602 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6f8ffff640 1 -- 172.21.15.37:0/1153476679 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558e5158ee80 con 0x558e51693e00 2024-03-26T06:36:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6f8ffff640 10 client.16602 handle_command_reply: tid=0 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6fb9dcd740 1 client.16602 shutdown 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1153476679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51623d40 msgr2=0x558e516259d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51623d40 0x558e516259d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009f70 tx=0x7f6fa401e6f0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1153476679 shutdown_connections 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558e51693e00 0x558e51696220 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e5162c550 0x558e51625f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e51623d40 0x558e516259d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6fb9dcd740 1 --2- 172.21.15.37:0/1153476679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e51622fd0 0x558e51633620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6f8ffff640 0 client.16602 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1153476679 >> 172.21.15.37:0/1153476679 conn(0x558e515ea5b0 msgr2=0x558e5161d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1153476679 shutdown_connections 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.911+0000 7f6fb9dcd740 1 -- 172.21.15.37:0/1153476679 wait complete. 2024-03-26T06:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb9dcd740 20 client.16602 trim_cache size 0 max 16384 2024-03-26T06:36:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:36:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:36:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:36:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 -- 172.21.15.37:0/802872531 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6fa00a2a10 msgr2=0x7f6fa00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6fa00a2a10 0x7f6fa00a4ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80ccb70 tx=0x7f6fa8005b40 comp rx=0 tx=0).stop 2024-03-26T06:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 -- 172.21.15.37:0/802872531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb01a2a00 msgr2=0x7f6fb01a2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb01a2a00 0x7f6fb01a2eb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800eb10 tx=0x7f6fa800c2d0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 -- 172.21.15.37:0/802872531 shutdown_connections 2024-03-26T06:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f6fa00a2a10 0x7f6fa00a4ed0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fb01a2a00 0x7f6fb01a2eb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fb010cc50 0x7f6fb019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 --2- 172.21.15.37:0/802872531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb01083a0 0x7f6fb019e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 -- 172.21.15.37:0/802872531 >> 172.21.15.37:0/802872531 conn(0x7f6fb0077050 msgr2=0x7f6fb0079440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 -- 172.21.15.37:0/802872531 shutdown_connections 2024-03-26T06:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:23.912+0000 7f6fb7110640 1 -- 172.21.15.37:0/802872531 wait complete. 2024-03-26T06:36:24.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 545212 4.26 MB/sec execute 599 sec latency 2221.343 ms 2024-03-26T06:36:25.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:24 smithi066 ceph-mon[36504]: pgmap v421: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-03-26T06:36:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:24 smithi149 ceph-mon[36681]: pgmap v421: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-03-26T06:36:25.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:24 smithi037 ceph-mon[31974]: pgmap v421: 97 pgs: 97 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-03-26T06:36:25.535 INFO:tasks.workunit.client.0.smithi037.stdout: 1 cleanup 600 sec 2024-03-26T06:36:25.742 INFO:tasks.workunit.client.0.smithi037.stdout: 0 cleanup 600 sec 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: Operation Count AvgLat MaxLat 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: ---------------------------------------- 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: NTCreateX 81362 5.235 14185.755 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: Close 59799 0.002 0.088 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: Rename 3446 12.836 4948.165 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: Unlink 16402 1.682 1087.296 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: Deltree 2 82.589 110.980 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: Mkdir 1 0.004 0.004 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: Qpathinfo 73779 1.046 88.298 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: Qfileinfo 12932 0.002 0.031 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: Qfsinfo 13500 0.174 12.419 2024-03-26T06:36:25.743 INFO:tasks.workunit.client.0.smithi037.stdout: Sfileinfo 6638 0.015 7.450 2024-03-26T06:36:25.744 INFO:tasks.workunit.client.0.smithi037.stdout: Find 28502 0.200 27.131 2024-03-26T06:36:25.744 INFO:tasks.workunit.client.0.smithi037.stdout: WriteX 40570 0.090 31.033 2024-03-26T06:36:25.744 INFO:tasks.workunit.client.0.smithi037.stdout: ReadX 127399 0.009 3.875 2024-03-26T06:36:25.744 INFO:tasks.workunit.client.0.smithi037.stdout: LockX 264 0.681 1.346 2024-03-26T06:36:25.744 INFO:tasks.workunit.client.0.smithi037.stdout: UnlockX 264 0.620 1.433 2024-03-26T06:36:25.744 INFO:tasks.workunit.client.0.smithi037.stdout: Flush 5696 2.595 17.487 2024-03-26T06:36:25.744 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-03-26T06:36:25.744 INFO:tasks.workunit.client.0.smithi037.stdout:Throughput 4.25673 MB/sec 1 clients 1 procs max_latency=14185.762 ms 2024-03-26T06:36:25.744 INFO:tasks.workunit.client.0.smithi037.stderr:+ dbench 10 2024-03-26T06:36:25.745 INFO:tasks.workunit.client.0.smithi037.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-26T06:36:25.745 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-03-26T06:36:25.745 INFO:tasks.workunit.client.0.smithi037.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-26T06:36:25.745 INFO:tasks.workunit.client.0.smithi037.stdout:8 of 10 processes prepared for launch 0 sec 2024-03-26T06:36:25.844 INFO:tasks.workunit.client.0.smithi037.stdout:10 of 10 processes prepared for launch 0 sec 2024-03-26T06:36:25.844 INFO:tasks.workunit.client.0.smithi037.stdout:releasing clients 2024-03-26T06:36:26.844 INFO:tasks.workunit.client.0.smithi037.stdout: 10 373 133.57 MB/sec warmup 1 sec latency 628.380 ms 2024-03-26T06:36:27.101 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:26 smithi066 ceph-mon[36504]: pgmap v422: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-03-26T06:36:27.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:26 smithi149 ceph-mon[36681]: pgmap v422: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-03-26T06:36:27.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:26 smithi037 ceph-mon[31974]: pgmap v422: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-03-26T06:36:27.844 INFO:tasks.workunit.client.0.smithi037.stdout: 10 820 100.57 MB/sec warmup 2 sec latency 1628.441 ms 2024-03-26T06:36:28.844 INFO:tasks.workunit.client.0.smithi037.stdout: 10 1376 80.39 MB/sec warmup 3 sec latency 2000.960 ms 2024-03-26T06:36:29.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:28 smithi066 ceph-mon[36504]: pgmap v423: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-03-26T06:36:29.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:28 smithi149 ceph-mon[36681]: pgmap v423: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-03-26T06:36:29.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:28 smithi037 ceph-mon[31974]: pgmap v423: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-03-26T06:36:29.844 INFO:tasks.workunit.client.0.smithi037.stdout: 10 2397 69.80 MB/sec warmup 4 sec latency 1307.696 ms 2024-03-26T06:36:30.844 INFO:tasks.workunit.client.0.smithi037.stdout: 10 3632 75.44 MB/sec warmup 5 sec latency 85.677 ms 2024-03-26T06:36:31.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:30 smithi066 ceph-mon[36504]: pgmap v424: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-03-26T06:36:31.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:30 smithi149 ceph-mon[36681]: pgmap v424: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-03-26T06:36:31.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:30 smithi037 ceph-mon[31974]: pgmap v424: 97 pgs: 97 active+clean; 218 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-03-26T06:36:31.844 INFO:tasks.workunit.client.0.smithi037.stdout: 10 4614 69.01 MB/sec warmup 6 sec latency 210.487 ms 2024-03-26T06:36:32.844 INFO:tasks.workunit.client.0.smithi037.stdout: 10 6244 72.46 MB/sec warmup 7 sec latency 36.735 ms 2024-03-26T06:36:33.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:32 smithi066 ceph-mon[36504]: pgmap v425: 97 pgs: 97 active+clean; 229 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-03-26T06:36:33.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:32 smithi149 ceph-mon[36681]: pgmap v425: 97 pgs: 97 active+clean; 229 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-03-26T06:36:33.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:32 smithi037 ceph-mon[31974]: pgmap v425: 97 pgs: 97 active+clean; 229 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-03-26T06:36:33.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 7381 71.68 MB/sec warmup 8 sec latency 49.720 ms 2024-03-26T06:36:34.844 INFO:tasks.workunit.client.0.smithi037.stdout: 10 8453 69.42 MB/sec warmup 9 sec latency 129.289 ms 2024-03-26T06:36:35.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:34 smithi066 ceph-mon[36504]: pgmap v426: 97 pgs: 97 active+clean; 229 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.8 MiB/s wr, 240 op/s 2024-03-26T06:36:35.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:34 smithi149 ceph-mon[36681]: pgmap v426: 97 pgs: 97 active+clean; 229 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.8 MiB/s wr, 240 op/s 2024-03-26T06:36:35.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:34 smithi037 ceph-mon[31974]: pgmap v426: 97 pgs: 97 active+clean; 229 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.8 MiB/s wr, 240 op/s 2024-03-26T06:36:35.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 9891 70.32 MB/sec warmup 10 sec latency 41.366 ms 2024-03-26T06:36:36.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 11036 70.86 MB/sec warmup 11 sec latency 35.138 ms 2024-03-26T06:36:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.033+0000 7f252f412640 1 -- 172.21.15.37:0/2347392201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2528106600 msgr2=0x7f252810e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.033+0000 7f252f412640 1 --2- 172.21.15.37:0/2347392201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2528106600 0x7f252810e1d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2520009be0 tx=0x7f252002bef0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.034+0000 7f252f412640 1 -- 172.21.15.37:0/2347392201 shutdown_connections 2024-03-26T06:36:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.034+0000 7f252f412640 1 --2- 172.21.15.37:0/2347392201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2528106600 0x7f252810e1d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.034+0000 7f252f412640 1 --2- 172.21.15.37:0/2347392201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2528101ee0 0x7f2528105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.034+0000 7f252f412640 1 --2- 172.21.15.37:0/2347392201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2528101530 0x7f2528101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.034+0000 7f252f412640 1 -- 172.21.15.37:0/2347392201 >> 172.21.15.37:0/2347392201 conn(0x7f25280fcdd0 msgr2=0x7f25280ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.034+0000 7f252f412640 1 -- 172.21.15.37:0/2347392201 shutdown_connections 2024-03-26T06:36:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.034+0000 7f252f412640 1 -- 172.21.15.37:0/2347392201 wait complete. 2024-03-26T06:36:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252f412640 1 Processor -- start 2024-03-26T06:36:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252f412640 1 -- start start 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252f412640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2528101530 0x7f25280735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252f412640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2528106600 0x7f2528073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252f412640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2528074070 0x7f25281a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252f412640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25281116a0 con 0x7f2528074070 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252f412640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2528111520 con 0x7f2528101530 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252f412640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2528111820 con 0x7f2528106600 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252d1ae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2528101530 0x7f25280735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252d9af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2528074070 0x7f25281a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252d1ae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2528101530 0x7f25280735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:59684/0 (socket says 172.21.15.37:59684) 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252d1ae640 1 -- 172.21.15.37:0/2288092175 learned_addr learned my addr 172.21.15.37:0/2288092175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252c9ad640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2528106600 0x7f2528073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252d9af640 1 -- 172.21.15.37:0/2288092175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2528106600 msgr2=0x7f2528073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252d9af640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2528106600 0x7f2528073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252d9af640 1 -- 172.21.15.37:0/2288092175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2528101530 msgr2=0x7f25280735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.035+0000 7f252d9af640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2528101530 0x7f25280735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.036+0000 7f252d9af640 1 -- 172.21.15.37:0/2288092175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25281a63b0 con 0x7f2528074070 2024-03-26T06:36:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.036+0000 7f252d9af640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2528074070 0x7f25281a5d10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2520033420 tx=0x7f25200336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.036+0000 7f251e7fc640 1 -- 172.21.15.37:0/2288092175 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520043020 con 0x7f2528074070 2024-03-26T06:36:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.036+0000 7f251e7fc640 1 -- 172.21.15.37:0/2288092175 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f252002ca80 con 0x7f2528074070 2024-03-26T06:36:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.036+0000 7f252f412640 1 -- 172.21.15.37:0/2288092175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25281a6640 con 0x7f2528074070 2024-03-26T06:36:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.036+0000 7f252f412640 1 -- 172.21.15.37:0/2288092175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25281a6bb0 con 0x7f2528074070 2024-03-26T06:36:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.036+0000 7f251e7fc640 1 -- 172.21.15.37:0/2288092175 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252003e6b0 con 0x7f2528074070 2024-03-26T06:36:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.039+0000 7f2531ffa740 1 Processor -- start 2024-03-26T06:36:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.040+0000 7f251e7fc640 1 -- 172.21.15.37:0/2288092175 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2520046020 con 0x7f2528074070 2024-03-26T06:36:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.040+0000 7f2531ffa740 1 -- start start 2024-03-26T06:36:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.040+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b35f8d40 0x55a8b35f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.040+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35f8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.040+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b3601550 0x55a8b3603940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.040+0000 7f2531ffa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b355c5f0 con 0x55a8b35f8d40 2024-03-26T06:36:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.040+0000 7f2531ffa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b3505930 con 0x55a8b3601550 2024-03-26T06:36:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.040+0000 7f2531ffa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b3558690 con 0x55a8b35f7fd0 2024-03-26T06:36:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.041+0000 7f252c9ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b3601550 0x55a8b3603940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.041+0000 7f252d1ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.041+0000 7f252c9ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b35f8d40 0x55a8b35f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.041+0000 7f252d1ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35f8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:41832/0 (socket says 172.21.15.37:41832) 2024-03-26T06:36:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.041+0000 7f252d1ae640 1 -- 172.21.15.37:0/770013649 learned_addr learned my addr 172.21.15.37:0/770013649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.041+0000 7f251e7fc640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25140a2b10 0x7f25140a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.041+0000 7f252d1ae640 1 -- 172.21.15.37:0/770013649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b3601550 msgr2=0x55a8b3603940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.041+0000 7f252d1ae640 1 --2- 172.21.15.37:0/770013649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b3601550 0x55a8b3603940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.041+0000 7f252d1ae640 1 -- 172.21.15.37:0/770013649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b35f8d40 msgr2=0x55a8b35f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.041+0000 7f252d1ae640 1 --2- 172.21.15.37:0/770013649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b35f8d40 0x55a8b35f7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.041+0000 7f252d1ae640 1 -- 172.21.15.37:0/770013649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8b3271210 con 0x55a8b35f7fd0 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.042+0000 7f252d9af640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25140a2b10 0x7f25140a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.042+0000 7f252d1ae640 1 --2- 172.21.15.37:0/770013649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35f8450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2524009d80 tx=0x7f2524011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26920b2fe2264a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.042+0000 7f251e7fc640 1 -- 172.21.15.37:0/2288092175 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f25200f11e0 con 0x7f2528074070 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.042+0000 7f25137fe640 1 -- 172.21.15.37:0/770013649 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252401e070 con 0x55a8b35f7fd0 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.042+0000 7f25137fe640 1 -- 172.21.15.37:0/770013649 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25240198f0 con 0x55a8b35f7fd0 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.042+0000 7f25137fe640 1 -- 172.21.15.37:0/770013649 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524018830 con 0x55a8b35f7fd0 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.043+0000 7f2531ffa740 1 -- 172.21.15.37:0/770013649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 msgr2=0x55a8b35f8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.043+0000 7f2531ffa740 1 --2- 172.21.15.37:0/770013649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35f8450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2524009d80 tx=0x7f2524011810 comp rx=0 tx=0).stop 2024-03-26T06:36:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.043+0000 7f252d9af640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25140a2b10 0x7f25140a4fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2520009be0 tx=0x7f2520009500 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.044+0000 7f2531ffa740 1 -- 172.21.15.37:0/770013649 shutdown_connections 2024-03-26T06:36:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.044+0000 7f2531ffa740 1 --2- 172.21.15.37:0/770013649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b3601550 0x55a8b3603940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.044+0000 7f2531ffa740 1 --2- 172.21.15.37:0/770013649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35f8450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.044+0000 7f2531ffa740 1 --2- 172.21.15.37:0/770013649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b35f8d40 0x55a8b35f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.044+0000 7f2531ffa740 1 -- 172.21.15.37:0/770013649 >> 172.21.15.37:0/770013649 conn(0x55a8b35bf5b0 msgr2=0x55a8b35f2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.044+0000 7f2531ffa740 1 -- 172.21.15.37:0/770013649 shutdown_connections 2024-03-26T06:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.044+0000 7f2531ffa740 1 -- 172.21.15.37:0/770013649 wait complete. 2024-03-26T06:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.045+0000 7f2531ffa740 1 -- start start 2024-03-26T06:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.045+0000 7f2531ffa740 1 Processor -- start 2024-03-26T06:36:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35fe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f8d40 0x55a8b35feab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b3601550 0x55a8b35feff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f2531ffa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b3604280 con 0x55a8b3601550 2024-03-26T06:36:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f2531ffa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b3518d90 con 0x55a8b35f8d40 2024-03-26T06:36:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f2531ffa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b3604400 con 0x55a8b35f7fd0 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f2531ffa740 10 client.0 _ll_register_callbacks cb 0x55a8b3270a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f252c9ad640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f252d1ae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f8d40 0x55a8b35feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f252c9ad640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:41840/0 (socket says 172.21.15.37:41840) 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f252c9ad640 1 -- 172.21.15.37:0/1795312615 learned_addr learned my addr 172.21.15.37:0/1795312615 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f252c9ad640 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b3601550 0x55a8b35feff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f252c9ad640 1 -- 172.21.15.37:0/1795312615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f8d40 msgr2=0x55a8b35feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f252c9ad640 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f8d40 0x55a8b35feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f252c9ad640 1 -- 172.21.15.37:0/1795312615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b3601550 msgr2=0x55a8b35feff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f252c9ad640 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b3601550 0x55a8b35feff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f252c9ad640 1 -- 172.21.15.37:0/1795312615 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8b333d620 con 0x55a8b35f7fd0 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.046+0000 7f252d1ae640 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f8d40 0x55a8b35feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:36:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.047+0000 7f252c9ad640 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35fe570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2518010ea0 tx=0x7f2518009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55b7af71c5e16b7a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.047+0000 7f2510ff9640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.047+0000 7f2510ff9640 1 -- 172.21.15.37:0/1795312615 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518019040 con 0x55a8b35f7fd0 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.047+0000 7f2510ff9640 1 -- 172.21.15.37:0/1795312615 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2518004230 con 0x55a8b35f7fd0 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.047+0000 7f2531ffa740 1 -- 172.21.15.37:0/1795312615 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25240198f0 con 0x55a8b35f7fd0 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.047+0000 7f2510ff9640 1 -- 172.21.15.37:0/1795312615 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2518004430 con 0x55a8b35f7fd0 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.047+0000 7f2531ffa740 10 client.25999 fetch_fsmap learned FSMap version 54 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.047+0000 7f2531ffa740 1 -- 172.21.15.37:0/1795312615 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55a8b3618d80 con 0x55a8b35f7fd0 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.047+0000 7f2510ff9640 1 -- 172.21.15.37:0/1795312615 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25240198f0 con 0x55a8b35f7fd0 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.048+0000 7f2510ff9640 1 -- 172.21.15.37:0/1795312615 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f25180214c0 con 0x55a8b35f7fd0 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.048+0000 7f2531ffa740 10 client.25999 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.048+0000 7f2531ffa740 10 client.25999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.048+0000 7f2531ffa740 20 client.25999 populate_metadata read hostname 'smithi037' 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.048+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a8b361a6c0 0x55a8b361cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.048+0000 7f2531ffa740 4 client.25999 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:36:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.048+0000 7f2531ffa740 1 -- 172.21.15.37:0/1795312615 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8b3569b30 con 0x55a8b361a6c0 2024-03-26T06:36:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.048+0000 7f252d1ae640 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a8b361a6c0 0x55a8b361cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.049+0000 7f252d1ae640 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a8b361a6c0 0x55a8b361cb00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6dfa8194faf733d server_cookie=68d5ff724f94e0bb in_seq=0 out_seq=0 2024-03-26T06:36:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.049+0000 7f2510ff9640 10 client.25999 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2510ff9640 1 -- 172.21.15.37:0/1795312615 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a8b3569b30 con 0x55a8b361a6c0 2024-03-26T06:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2510ff9640 10 client.25999 handle_command_reply: tid=0 2024-03-26T06:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2531ffa740 1 client.25999 shutdown 2024-03-26T06:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2531ffa740 1 -- 172.21.15.37:0/1795312615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 msgr2=0x55a8b35fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35fe570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2518010ea0 tx=0x7f2518009ad0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2531ffa740 1 -- 172.21.15.37:0/1795312615 shutdown_connections 2024-03-26T06:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a8b361a6c0 0x55a8b361cb00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b3601550 0x55a8b35feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f8d40 0x55a8b35feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1795312615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f7fd0 0x55a8b35fe570 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2510ff9640 0 client.25999 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2531ffa740 1 -- 172.21.15.37:0/1795312615 >> 172.21.15.37:0/1795312615 conn(0x55a8b35bf5b0 msgr2=0x55a8b35f4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.051+0000 7f2531ffa740 1 -- 172.21.15.37:0/1795312615 shutdown_connections 2024-03-26T06:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.052+0000 7f2531ffa740 1 -- 172.21.15.37:0/1795312615 wait complete. 2024-03-26T06:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.052+0000 7f2531ffa740 20 client.25999 trim_cache size 0 max 16384 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f2531ffa740 1 Processor -- start 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f2531ffa740 1 -- start start 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 0x55a8b3600ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f8d40 0x55a8b3610f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b3601550 0x55a8b3611470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f2531ffa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25281aaad0 con 0x55a8b3601550 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f2531ffa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25281ac630 con 0x55a8b35f7fd0 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f2531ffa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25281ab130 con 0x55a8b35f8d40 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f252c9ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b3601550 0x55a8b3611470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f252c9ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 0x55a8b3600ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f252d1ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f8d40 0x55a8b3610f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f252c9ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 0x55a8b3600ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:59716/0 (socket says 172.21.15.37:59716) 2024-03-26T06:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f252c9ad640 1 -- 172.21.15.37:0/1307056024 learned_addr learned my addr 172.21.15.37:0/1307056024 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f252d1ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f8d40 0x55a8b3610f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:41842/0 (socket says 172.21.15.37:41842) 2024-03-26T06:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f252c9ad640 1 -- 172.21.15.37:0/1307056024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f8d40 msgr2=0x55a8b3610f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f252c9ad640 1 --2- 172.21.15.37:0/1307056024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f8d40 0x55a8b3610f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f252c9ad640 1 -- 172.21.15.37:0/1307056024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 msgr2=0x55a8b3600ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f252c9ad640 1 --2- 172.21.15.37:0/1307056024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 0x55a8b3600ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.072+0000 7f252c9ad640 1 -- 172.21.15.37:0/1307056024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8b3621cf0 con 0x55a8b3601550 2024-03-26T06:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.073+0000 7f252c9ad640 1 --2- 172.21.15.37:0/1307056024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b3601550 0x55a8b3611470 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2518009fd0 tx=0x7f2518010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59f7b967c5a7415 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.073+0000 7f2512ffd640 1 -- 172.21.15.37:0/1307056024 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518021ad0 con 0x55a8b3601550 2024-03-26T06:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.073+0000 7f2512ffd640 1 -- 172.21.15.37:0/1307056024 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2518014030 con 0x55a8b3601550 2024-03-26T06:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.073+0000 7f2512ffd640 1 -- 172.21.15.37:0/1307056024 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518021690 con 0x55a8b3601550 2024-03-26T06:36:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.074+0000 7f2531ffa740 1 -- 172.21.15.37:0/1307056024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b3601550 msgr2=0x55a8b3611470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.074+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1307056024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b3601550 0x55a8b3611470 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2518009fd0 tx=0x7f2518010f90 comp rx=0 tx=0).stop 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.074+0000 7f2531ffa740 1 -- 172.21.15.37:0/1307056024 shutdown_connections 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.074+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1307056024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b3601550 0x55a8b3611470 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.074+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1307056024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b35f8d40 0x55a8b3610f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.074+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1307056024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 0x55a8b3600ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.074+0000 7f2531ffa740 1 -- 172.21.15.37:0/1307056024 >> 172.21.15.37:0/1307056024 conn(0x55a8b35bf5b0 msgr2=0x55a8b35f27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.074+0000 7f2531ffa740 1 -- 172.21.15.37:0/1307056024 shutdown_connections 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.074+0000 7f2531ffa740 1 -- 172.21.15.37:0/1307056024 wait complete. 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.074+0000 7f2531ffa740 1 -- start start 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.074+0000 7f2531ffa740 1 Processor -- start 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 0x55a8b3611440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b35f8d40 0x55a8b35fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f2531ffa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b3601550 0x55a8b35fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f252c9ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 0x55a8b3611440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f252c9ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 0x55a8b3611440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:59722/0 (socket says 172.21.15.37:59722) 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f252c9ad640 1 -- 172.21.15.37:0/1830175398 learned_addr learned my addr 172.21.15.37:0/1830175398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f252d1ae640 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b35f8d40 0x55a8b35fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f2531ffa740 1 -- 172.21.15.37:0/1830175398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25281abf10 con 0x55a8b35f8d40 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f2531ffa740 1 -- 172.21.15.37:0/1830175398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25281ab850 con 0x55a8b35f7fd0 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f2531ffa740 1 -- 172.21.15.37:0/1830175398 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25281a8d90 con 0x55a8b3601550 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f2531ffa740 10 client.0 _ll_register_callbacks cb 0x7f25240097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.075+0000 7f252c9ad640 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b3601550 0x55a8b35fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.076+0000 7f252c9ad640 1 -- 172.21.15.37:0/1830175398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b3601550 msgr2=0x55a8b35fbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.076+0000 7f252c9ad640 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b3601550 0x55a8b35fbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.076+0000 7f252c9ad640 1 -- 172.21.15.37:0/1830175398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b35f8d40 msgr2=0x55a8b35fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.076+0000 7f252c9ad640 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b35f8d40 0x55a8b35fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.076+0000 7f252c9ad640 1 -- 172.21.15.37:0/1830175398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8b35fa7e0 con 0x55a8b35f7fd0 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.076+0000 7f252d1ae640 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b35f8d40 0x55a8b35fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.076+0000 7f252c9ad640 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 0x55a8b3611440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2518000c00 tx=0x7f25180092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3bce346fceac343 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.076+0000 7f2510ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.077+0000 7f2510ff9640 1 -- 172.21.15.37:0/1830175398 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518019040 con 0x55a8b35f7fd0 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.077+0000 7f2510ff9640 1 -- 172.21.15.37:0/1830175398 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2518005260 con 0x55a8b35f7fd0 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.077+0000 7f2531ffa740 1 -- 172.21.15.37:0/1830175398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2518014030 con 0x55a8b35f7fd0 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.077+0000 7f2510ff9640 1 -- 172.21.15.37:0/1830175398 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25180216e0 con 0x55a8b35f7fd0 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.077+0000 7f2531ffa740 10 client.26138 fetch_fsmap learned FSMap version 54 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.077+0000 7f2531ffa740 1 -- 172.21.15.37:0/1830175398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55a8b3670e80 con 0x55a8b35f7fd0 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.077+0000 7f2510ff9640 1 -- 172.21.15.37:0/1830175398 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2518014030 con 0x55a8b35f7fd0 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.077+0000 7f2510ff9640 1 -- 172.21.15.37:0/1830175398 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f2518036400 con 0x55a8b35f7fd0 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.078+0000 7f2531ffa740 10 client.26138 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.078+0000 7f2531ffa740 10 client.26138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:36:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.078+0000 7f2531ffa740 20 client.26138 populate_metadata read hostname 'smithi037' 2024-03-26T06:36:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.078+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a8b36715b0 0x55a8b36739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.078+0000 7f2531ffa740 4 client.26138 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:36:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.078+0000 7f2531ffa740 1 -- 172.21.15.37:0/1830175398 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a8b3563e80 con 0x55a8b36715b0 2024-03-26T06:36:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.078+0000 7f252d1ae640 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a8b36715b0 0x55a8b36739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.078+0000 7f2510ff9640 10 client.26138 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.079+0000 7f252d1ae640 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a8b36715b0 0x55a8b36739d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1386fdd2b8e8adad server_cookie=40863edad8f94371 in_seq=0 out_seq=0 2024-03-26T06:36:37.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:36 smithi066 ceph-mon[36504]: pgmap v427: 97 pgs: 97 active+clean; 265 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 589 op/s 2024-03-26T06:36:37.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:36 smithi149 ceph-mon[36681]: pgmap v427: 97 pgs: 97 active+clean; 265 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 589 op/s 2024-03-26T06:36:37.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:36 smithi037 ceph-mon[31974]: pgmap v427: 97 pgs: 97 active+clean; 265 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 589 op/s 2024-03-26T06:36:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2510ff9640 1 -- 172.21.15.37:0/1830175398 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a8b3563e80 con 0x55a8b36715b0 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2510ff9640 10 client.26138 handle_command_reply: tid=0 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2531ffa740 1 client.26138 shutdown 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2531ffa740 1 -- 172.21.15.37:0/1830175398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 msgr2=0x55a8b3611440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 0x55a8b3611440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2518000c00 tx=0x7f25180092d0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2531ffa740 1 -- 172.21.15.37:0/1830175398 shutdown_connections 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55a8b36715b0 0x55a8b36739d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8b3601550 0x55a8b35fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a8b35f8d40 0x55a8b35fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2531ffa740 1 --2- 172.21.15.37:0/1830175398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8b35f7fd0 0x55a8b3611440 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2531ffa740 1 -- 172.21.15.37:0/1830175398 >> 172.21.15.37:0/1830175398 conn(0x55a8b35bf5b0 msgr2=0x55a8b35fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.299+0000 7f2510ff9640 0 client.26138 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f2531ffa740 1 -- 172.21.15.37:0/1830175398 shutdown_connections 2024-03-26T06:36:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f2531ffa740 1 -- 172.21.15.37:0/1830175398 wait complete. 2024-03-26T06:36:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f2531ffa740 20 client.26138 trim_cache size 0 max 16384 2024-03-26T06:36:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:36:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:36:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:36:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:36:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f252f412640 1 -- 172.21.15.37:0/2288092175 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25140a2b10 msgr2=0x7f25140a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f252f412640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25140a2b10 0x7f25140a4fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2520009be0 tx=0x7f2520009500 comp rx=0 tx=0).stop 2024-03-26T06:36:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f252f412640 1 -- 172.21.15.37:0/2288092175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2528074070 msgr2=0x7f25281a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f252f412640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2528074070 0x7f25281a5d10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2520033420 tx=0x7f25200336b0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f252f412640 1 -- 172.21.15.37:0/2288092175 shutdown_connections 2024-03-26T06:36:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f252f412640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f25140a2b10 0x7f25140a4fd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f252f412640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2528074070 0x7f25281a5d10 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f252f412640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2528106600 0x7f2528073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f252f412640 1 --2- 172.21.15.37:0/2288092175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2528101530 0x7f25280735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.300+0000 7f252f412640 1 -- 172.21.15.37:0/2288092175 >> 172.21.15.37:0/2288092175 conn(0x7f25280fcdd0 msgr2=0x7f25280ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.301+0000 7f252f412640 1 -- 172.21.15.37:0/2288092175 shutdown_connections 2024-03-26T06:36:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:37.301+0000 7f252f412640 1 -- 172.21.15.37:0/2288092175 wait complete. 2024-03-26T06:36:37.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 11829 67.40 MB/sec warmup 12 sec latency 115.518 ms 2024-03-26T06:36:38.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:37 smithi149 ceph-mon[36681]: pgmap v428: 97 pgs: 97 active+clean; 265 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 505 op/s 2024-03-26T06:36:38.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:37 smithi037 ceph-mon[31974]: pgmap v428: 97 pgs: 97 active+clean; 265 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 505 op/s 2024-03-26T06:36:38.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:37 smithi066 ceph-mon[36504]: pgmap v428: 97 pgs: 97 active+clean; 265 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 505 op/s 2024-03-26T06:36:38.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 12699 65.89 MB/sec warmup 13 sec latency 285.585 ms 2024-03-26T06:36:39.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 13686 64.91 MB/sec warmup 14 sec latency 191.588 ms 2024-03-26T06:36:40.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:40 smithi066 ceph-mon[36504]: pgmap v429: 97 pgs: 97 active+clean; 265 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 505 op/s 2024-03-26T06:36:40.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:40 smithi149 ceph-mon[36681]: pgmap v429: 97 pgs: 97 active+clean; 265 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 505 op/s 2024-03-26T06:36:40.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:40 smithi037 ceph-mon[31974]: pgmap v429: 97 pgs: 97 active+clean; 265 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 505 op/s 2024-03-26T06:36:40.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 14721 64.87 MB/sec warmup 15 sec latency 253.440 ms 2024-03-26T06:36:41.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 15609 63.18 MB/sec warmup 16 sec latency 53.536 ms 2024-03-26T06:36:42.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:42 smithi066 ceph-mon[36504]: pgmap v430: 97 pgs: 97 active+clean; 446 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 24 MiB/s wr, 969 op/s 2024-03-26T06:36:42.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:42 smithi149 ceph-mon[36681]: pgmap v430: 97 pgs: 97 active+clean; 446 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 24 MiB/s wr, 969 op/s 2024-03-26T06:36:42.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:42 smithi037 ceph-mon[31974]: pgmap v430: 97 pgs: 97 active+clean; 446 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 24 MiB/s wr, 969 op/s 2024-03-26T06:36:42.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 16972 63.49 MB/sec warmup 17 sec latency 26.387 ms 2024-03-26T06:36:43.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 18171 64.72 MB/sec warmup 18 sec latency 43.202 ms 2024-03-26T06:36:44.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:44 smithi066 ceph-mon[36504]: pgmap v431: 97 pgs: 97 active+clean; 446 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 22 MiB/s wr, 813 op/s 2024-03-26T06:36:44.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:44 smithi149 ceph-mon[36681]: pgmap v431: 97 pgs: 97 active+clean; 446 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 22 MiB/s wr, 813 op/s 2024-03-26T06:36:44.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:44 smithi037 ceph-mon[31974]: pgmap v431: 97 pgs: 97 active+clean; 446 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 22 MiB/s wr, 813 op/s 2024-03-26T06:36:44.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 18969 62.79 MB/sec warmup 19 sec latency 50.820 ms 2024-03-26T06:36:45.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 20115 62.76 MB/sec warmup 20 sec latency 43.090 ms 2024-03-26T06:36:46.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 21359 62.91 MB/sec warmup 21 sec latency 58.595 ms 2024-03-26T06:36:47.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:46 smithi066 ceph-mon[36504]: pgmap v432: 97 pgs: 97 active+clean; 478 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.14k op/s 2024-03-26T06:36:47.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:46 smithi149 ceph-mon[36681]: pgmap v432: 97 pgs: 97 active+clean; 478 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.14k op/s 2024-03-26T06:36:47.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:46 smithi037 ceph-mon[31974]: pgmap v432: 97 pgs: 97 active+clean; 478 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.14k op/s 2024-03-26T06:36:47.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 22284 62.61 MB/sec warmup 22 sec latency 254.634 ms 2024-03-26T06:36:48.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 23094 61.68 MB/sec warmup 23 sec latency 247.901 ms 2024-03-26T06:36:49.098 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:48 smithi066 ceph-mon[36504]: pgmap v433: 97 pgs: 97 active+clean; 479 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 800 op/s 2024-03-26T06:36:49.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:48 smithi149 ceph-mon[36681]: pgmap v433: 97 pgs: 97 active+clean; 479 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 800 op/s 2024-03-26T06:36:49.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:48 smithi037 ceph-mon[31974]: pgmap v433: 97 pgs: 97 active+clean; 479 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 800 op/s 2024-03-26T06:36:49.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 23299 59.76 MB/sec warmup 24 sec latency 1081.885 ms 2024-03-26T06:36:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.442+0000 7fa4be590640 1 -- 172.21.15.37:0/128206228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8114440 msgr2=0x7fa4b8116830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.442+0000 7fa4be590640 1 --2- 172.21.15.37:0/128206228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8114440 0x7fa4b8116830 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009c10 tx=0x7fa4b002bef0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.442+0000 7fa4be590640 1 -- 172.21.15.37:0/128206228 shutdown_connections 2024-03-26T06:36:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.442+0000 7fa4be590640 1 --2- 172.21.15.37:0/128206228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8114440 0x7fa4b8116830 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.442+0000 7fa4be590640 1 --2- 172.21.15.37:0/128206228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b8108c10 0x7fa4b8113f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.442+0000 7fa4be590640 1 --2- 172.21.15.37:0/128206228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b81082f0 0x7fa4b81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.442+0000 7fa4be590640 1 -- 172.21.15.37:0/128206228 >> 172.21.15.37:0/128206228 conn(0x7fa4b806c820 msgr2=0x7fa4b806cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.442+0000 7fa4be590640 1 -- 172.21.15.37:0/128206228 shutdown_connections 2024-03-26T06:36:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.442+0000 7fa4be590640 1 -- 172.21.15.37:0/128206228 wait complete. 2024-03-26T06:36:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.443+0000 7fa4be590640 1 Processor -- start 2024-03-26T06:36:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.443+0000 7fa4be590640 1 -- start start 2024-03-26T06:36:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.444+0000 7fa4be590640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b81082f0 0x7fa4b810bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.444+0000 7fa4be590640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b8108c10 0x7fa4b810c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.444+0000 7fa4be590640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8114440 0x7fa4b810c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.444+0000 7fa4be590640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b8119eb0 con 0x7fa4b8114440 2024-03-26T06:36:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.444+0000 7fa4be590640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b8119d30 con 0x7fa4b8108c10 2024-03-26T06:36:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.444+0000 7fa4be590640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b811a030 con 0x7fa4b81082f0 2024-03-26T06:36:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.444+0000 7fa4b7fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b81082f0 0x7fa4b810bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.444+0000 7fa4bcb2d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8114440 0x7fa4b810c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.444+0000 7fa4b7fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b81082f0 0x7fa4b810bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:42158/0 (socket says 172.21.15.37:42158) 2024-03-26T06:36:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.444+0000 7fa4bcb2d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8114440 0x7fa4b810c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38106/0 (socket says 172.21.15.37:38106) 2024-03-26T06:36:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.444+0000 7fa4b7fff640 1 -- 172.21.15.37:0/1367679473 learned_addr learned my addr 172.21.15.37:0/1367679473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.445+0000 7fa4b77fe640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b8108c10 0x7fa4b810c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.445+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/1367679473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b81082f0 msgr2=0x7fa4b810bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.445+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b81082f0 0x7fa4b810bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.445+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/1367679473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b8108c10 msgr2=0x7fa4b810c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.445+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b8108c10 0x7fa4b810c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.445+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/1367679473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b81b2b70 con 0x7fa4b8114440 2024-03-26T06:36:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.445+0000 7fa4b77fe640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b8108c10 0x7fa4b810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:36:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.445+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8114440 0x7fa4b810c9f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0036040 tx=0x7fa4b00336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.446+0000 7fa4b57fa640 1 -- 172.21.15.37:0/1367679473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0043020 con 0x7fa4b8114440 2024-03-26T06:36:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.446+0000 7fa4b57fa640 1 -- 172.21.15.37:0/1367679473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4b0003e50 con 0x7fa4b8114440 2024-03-26T06:36:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.446+0000 7fa4b57fa640 1 -- 172.21.15.37:0/1367679473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b003e6b0 con 0x7fa4b8114440 2024-03-26T06:36:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.446+0000 7fa4be590640 1 -- 172.21.15.37:0/1367679473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b81b2e00 con 0x7fa4b8114440 2024-03-26T06:36:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.446+0000 7fa4be590640 1 -- 172.21.15.37:0/1367679473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b81b3270 con 0x7fa4b8114440 2024-03-26T06:36:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.450+0000 7fa4b57fa640 1 -- 172.21.15.37:0/1367679473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa4b0046020 con 0x7fa4b8114440 2024-03-26T06:36:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4c1172740 1 Processor -- start 2024-03-26T06:36:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4c1172740 1 -- start start 2024-03-26T06:36:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4b57fa640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa4a00a2ae0 0x7fa4a00a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f8197da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197dfd0 0x555f8197e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 0x555f81989940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4c1172740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f818e25f0 con 0x555f81987550 2024-03-26T06:36:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4c1172740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8188b930 con 0x555f8197ed40 2024-03-26T06:36:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4c1172740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f818de690 con 0x555f8197dfd0 2024-03-26T06:36:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4b77fe640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa4a00a2ae0 0x7fa4a00a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f8197da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4bcb2d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 0x555f81989940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4b7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f8197da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:50462/0 (socket says 172.21.15.37:50462) 2024-03-26T06:36:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4b7fff640 1 -- 172.21.15.37:0/3972173262 learned_addr learned my addr 172.21.15.37:0/3972173262 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.451+0000 7fa4b77fe640 1 --2- 172.21.15.37:0/3972173262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197dfd0 0x555f8197e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa4b57fa640 1 -- 172.21.15.37:0/1367679473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa4b00f0fb0 con 0x7fa4b8114440 2024-03-26T06:36:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/3972173262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197dfd0 msgr2=0x555f8197e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/3972173262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197dfd0 0x555f8197e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/3972173262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 msgr2=0x555f8197da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/3972173262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f8197da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/3972173262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f815f7210 con 0x555f81987550 2024-03-26T06:36:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa4b7fff640 1 --2- 172.21.15.37:0/3972173262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f8197da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:36:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/3972173262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 0x555f81989940 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00ee7b0 tx=0x7fa4b008f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5a12b6699f8a3c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa49e7fc640 1 -- 172.21.15.37:0/3972173262 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00d6960 con 0x555f81987550 2024-03-26T06:36:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa4b77fe640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa4a00a2ae0 0x7fa4a00a4fa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac006fd0 tx=0x7fa4ac007870 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa49e7fc640 1 -- 172.21.15.37:0/3972173262 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4b00bddc0 con 0x555f81987550 2024-03-26T06:36:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.452+0000 7fa49e7fc640 1 -- 172.21.15.37:0/3972173262 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00d6780 con 0x555f81987550 2024-03-26T06:36:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.454+0000 7fa4c1172740 1 -- 172.21.15.37:0/3972173262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 msgr2=0x555f81989940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.454+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3972173262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 0x555f81989940 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00ee7b0 tx=0x7fa4b008f2f0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.454+0000 7fa4c1172740 1 -- 172.21.15.37:0/3972173262 shutdown_connections 2024-03-26T06:36:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.454+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3972173262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 0x555f81989940 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.454+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3972173262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197dfd0 0x555f8197e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.454+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3972173262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f8197da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.454+0000 7fa4c1172740 1 -- 172.21.15.37:0/3972173262 >> 172.21.15.37:0/3972173262 conn(0x555f819455b0 msgr2=0x555f81978cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.454+0000 7fa4c1172740 1 -- 172.21.15.37:0/3972173262 shutdown_connections 2024-03-26T06:36:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.454+0000 7fa4c1172740 1 -- 172.21.15.37:0/3972173262 wait complete. 2024-03-26T06:36:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.454+0000 7fa4c1172740 1 -- start start 2024-03-26T06:36:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.455+0000 7fa4c1172740 1 Processor -- start 2024-03-26T06:36:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.455+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f8197dfd0 0x555f81984610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.455+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f81984b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.455+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f81987550 0x555f81985090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.455+0000 7fa4c1172740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8198a040 con 0x555f8197dfd0 2024-03-26T06:36:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.455+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f81984b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.455+0000 7fa4bcb2d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f81987550 0x555f81985090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.455+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f81984b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:50476/0 (socket says 172.21.15.37:50476) 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.455+0000 7fa4bcb2d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f81987550 0x555f81985090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:42166/0 (socket says 172.21.15.37:42166) 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.455+0000 7fa4b77fe640 1 -- 172.21.15.37:0/3061420154 learned_addr learned my addr 172.21.15.37:0/3061420154 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.456+0000 7fa4c1172740 1 -- 172.21.15.37:0/3061420154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8189ed90 con 0x555f8197ed40 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.456+0000 7fa4c1172740 1 -- 172.21.15.37:0/3061420154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8198a1c0 con 0x555f81987550 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.456+0000 7fa4c1172740 10 client.0 _ll_register_callbacks cb 0x555f815f6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.456+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/3061420154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 msgr2=0x555f81984b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.456+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f81984b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.456+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/3061420154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f8197dfd0 msgr2=0x555f81984610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.456+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f8197dfd0 0x555f81984610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.456+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/3061420154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f816c3620 con 0x555f81987550 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.456+0000 7fa4b77fe640 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f81984b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.456+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f81987550 0x555f81985090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00ee7b0 tx=0x7fa4b0004670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=966d6197f49163c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.456+0000 7fa4977fe640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:36:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.457+0000 7fa4977fe640 1 -- 172.21.15.37:0/3061420154 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00bd6c0 con 0x555f81987550 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.457+0000 7fa4c1172740 1 -- 172.21.15.37:0/3061420154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4b00bddc0 con 0x555f81987550 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.457+0000 7fa4977fe640 1 -- 172.21.15.37:0/3061420154 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa4b00bb9c0 con 0x555f81987550 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.457+0000 7fa4977fe640 1 -- 172.21.15.37:0/3061420154 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4b00db050 con 0x555f81987550 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.457+0000 7fa4c1172740 10 client.26014 fetch_fsmap learned FSMap version 54 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.457+0000 7fa4c1172740 1 -- 172.21.15.37:0/3061420154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x555f8198a040 con 0x555f81987550 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.457+0000 7fa4977fe640 1 -- 172.21.15.37:0/3061420154 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4b00c5e60 con 0x555f81987550 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.458+0000 7fa4977fe640 1 -- 172.21.15.37:0/3061420154 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fa4b00ee890 con 0x555f81987550 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.458+0000 7fa4c1172740 10 client.26014 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.458+0000 7fa4c1172740 10 client.26014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.458+0000 7fa4c1172740 20 client.26014 populate_metadata read hostname 'smithi037' 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.458+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555f819975c0 0x555f81999a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.458+0000 7fa4c1172740 4 client.26014 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:36:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.458+0000 7fa4c1172740 1 -- 172.21.15.37:0/3061420154 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4b00bddc0 con 0x555f819975c0 2024-03-26T06:36:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.459+0000 7fa4b7fff640 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555f819975c0 0x555f81999a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.460+0000 7fa4977fe640 10 client.26014 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.460+0000 7fa4b7fff640 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555f819975c0 0x555f81999a00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9219ae7adfb7c1f6 server_cookie=3468351c794da8cb in_seq=0 out_seq=0 2024-03-26T06:36:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.462+0000 7fa4977fe640 1 -- 172.21.15.37:0/3061420154 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fa4b00bddc0 con 0x555f819975c0 2024-03-26T06:36:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.462+0000 7fa4977fe640 10 client.26014 handle_command_reply: tid=0 2024-03-26T06:36:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.462+0000 7fa4c1172740 1 client.26014 shutdown 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4c1172740 1 -- 172.21.15.37:0/3061420154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f81987550 msgr2=0x555f81985090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f81987550 0x555f81985090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00ee7b0 tx=0x7fa4b0004670 comp rx=0 tx=0).stop 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4c1172740 1 -- 172.21.15.37:0/3061420154 shutdown_connections 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555f819975c0 0x555f81999a00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f81987550 0x555f81985090 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197ed40 0x555f81984b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4977fe640 0 client.26014 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3061420154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f8197dfd0 0x555f81984610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4c1172740 1 -- 172.21.15.37:0/3061420154 >> 172.21.15.37:0/3061420154 conn(0x555f819455b0 msgr2=0x555f8197aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4c1172740 1 -- 172.21.15.37:0/3061420154 shutdown_connections 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4c1172740 1 -- 172.21.15.37:0/3061420154 wait complete. 2024-03-26T06:36:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.463+0000 7fa4c1172740 20 client.26014 trim_cache size 0 max 16384 2024-03-26T06:36:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.487+0000 7fa4c1172740 1 Processor -- start 2024-03-26T06:36:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.487+0000 7fa4c1172740 1 -- start start 2024-03-26T06:36:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.487+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197dfd0 0x555f81986d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.487+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f8197ed40 0x555f81984eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.487+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f81987550 0x555f8198e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.487+0000 7fa4c1172740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b81bb9e0 con 0x555f8197ed40 2024-03-26T06:36:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.487+0000 7fa4c1172740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b81bd540 con 0x555f81987550 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.487+0000 7fa4c1172740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b81bc040 con 0x555f8197dfd0 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f8197ed40 0x555f81984eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f8197ed40 0x555f81984eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38138/0 (socket says 172.21.15.37:38138) 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4b7fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197dfd0 0x555f81986d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4b77fe640 1 -- 172.21.15.37:0/3276801705 learned_addr learned my addr 172.21.15.37:0/3276801705 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/3276801705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f81987550 0x555f8198e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4b77fe640 1 -- 172.21.15.37:0/3276801705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197dfd0 msgr2=0x555f81986d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4b77fe640 1 --2- 172.21.15.37:0/3276801705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197dfd0 0x555f81986d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4b77fe640 1 -- 172.21.15.37:0/3276801705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f81987550 msgr2=0x555f8198e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4b77fe640 1 --2- 172.21.15.37:0/3276801705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f81987550 0x555f8198e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4b77fe640 1 -- 172.21.15.37:0/3276801705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f81974350 con 0x555f8197ed40 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/3276801705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f81987550 0x555f8198e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.488+0000 7fa4b77fe640 1 --2- 172.21.15.37:0/3276801705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f8197ed40 0x555f81984eb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac007c60 tx=0x7fa4ac01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af46d1abe7d6ca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.489+0000 7fa49dffb640 1 -- 172.21.15.37:0/3276801705 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac00f040 con 0x555f8197ed40 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.489+0000 7fa49dffb640 1 -- 172.21.15.37:0/3276801705 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac028070 con 0x555f8197ed40 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.489+0000 7fa49dffb640 1 -- 172.21.15.37:0/3276801705 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0239d0 con 0x555f8197ed40 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.490+0000 7fa4c1172740 1 -- 172.21.15.37:0/3276801705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f8197ed40 msgr2=0x555f81984eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.490+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3276801705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f8197ed40 0x555f81984eb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac007c60 tx=0x7fa4ac01b000 comp rx=0 tx=0).stop 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.490+0000 7fa4c1172740 1 -- 172.21.15.37:0/3276801705 shutdown_connections 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.490+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3276801705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f81987550 0x555f8198e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.490+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3276801705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f8197ed40 0x555f81984eb0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.490+0000 7fa4c1172740 1 --2- 172.21.15.37:0/3276801705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197dfd0 0x555f81986d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.490+0000 7fa4c1172740 1 -- 172.21.15.37:0/3276801705 >> 172.21.15.37:0/3276801705 conn(0x555f819455b0 msgr2=0x555f81982ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.491+0000 7fa4c1172740 1 -- 172.21.15.37:0/3276801705 shutdown_connections 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.491+0000 7fa4c1172740 1 -- 172.21.15.37:0/3276801705 wait complete. 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.491+0000 7fa4c1172740 1 -- start start 2024-03-26T06:36:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.491+0000 7fa4c1172740 1 Processor -- start 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.492+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197dfd0 0x555f81981620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.492+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197ed40 0x555f81981b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.492+0000 7fa4c1172740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 0x555f819820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.492+0000 7fa4c1172740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b81bce20 con 0x555f81987550 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.492+0000 7fa4c1172740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b81bc760 con 0x555f8197dfd0 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.492+0000 7fa4c1172740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b81b9ca0 con 0x555f8197ed40 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.492+0000 7fa4bcb2d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 0x555f819820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.492+0000 7fa4bcb2d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 0x555f819820c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38144/0 (socket says 172.21.15.37:38144) 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.492+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/1116060137 learned_addr learned my addr 172.21.15.37:0/1116060137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.492+0000 7fa4b77fe640 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197ed40 0x555f81981b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.492+0000 7fa4c1172740 10 client.0 _ll_register_callbacks cb 0x7fa4b00b9960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:36:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.493+0000 7fa4b7fff640 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197dfd0 0x555f81981620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.493+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/1116060137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197ed40 msgr2=0x555f81981b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.493+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197ed40 0x555f81981b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.493+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/1116060137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197dfd0 msgr2=0x555f81981620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:36:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.493+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197dfd0 0x555f81981620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.493+0000 7fa4bcb2d640 1 -- 172.21.15.37:0/1116060137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f819eddb0 con 0x555f81987550 2024-03-26T06:36:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.493+0000 7fa4bcb2d640 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 0x555f819820c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00eec30 tx=0x7fa4b00bbbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d55e0f82cf2f38c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:36:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.493+0000 7fa4977fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:36:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.494+0000 7fa4977fe640 1 -- 172.21.15.37:0/1116060137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00bddc0 con 0x555f81987550 2024-03-26T06:36:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.494+0000 7fa4977fe640 1 -- 172.21.15.37:0/1116060137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa4b00bcb10 con 0x555f81987550 2024-03-26T06:36:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.494+0000 7fa4c1172740 1 -- 172.21.15.37:0/1116060137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4ac028070 con 0x555f81987550 2024-03-26T06:36:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.494+0000 7fa4977fe640 1 -- 172.21.15.37:0/1116060137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4b00c5e60 con 0x555f81987550 2024-03-26T06:36:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.494+0000 7fa4c1172740 10 client.16635 fetch_fsmap learned FSMap version 54 2024-03-26T06:36:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.494+0000 7fa4c1172740 1 -- 172.21.15.37:0/1116060137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x555f819eddb0 con 0x555f81987550 2024-03-26T06:36:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.495+0000 7fa4977fe640 1 -- 172.21.15.37:0/1116060137 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac028070 con 0x555f81987550 2024-03-26T06:36:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.495+0000 7fa4977fe640 1 -- 172.21.15.37:0/1116060137 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fa4b00b9410 con 0x555f81987550 2024-03-26T06:36:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.495+0000 7fa4c1172740 10 client.16635 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:36:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.495+0000 7fa4c1172740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:36:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.495+0000 7fa4c1172740 20 client.16635 populate_metadata read hostname 'smithi037' 2024-03-26T06:36:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.495+0000 7fa4c1172740 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555f819ef350 0x555f819f1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:36:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.495+0000 7fa4c1172740 4 client.16635 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:36:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.495+0000 7fa4c1172740 1 -- 172.21.15.37:0/1116060137 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555f818efb30 con 0x555f819ef350 2024-03-26T06:36:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.497+0000 7fa4b7fff640 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555f819ef350 0x555f819f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:36:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.497+0000 7fa4b7fff640 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555f819ef350 0x555f819f1770 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f595a1b10f021e35 server_cookie=777446b607cd5562 in_seq=0 out_seq=0 2024-03-26T06:36:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.497+0000 7fa4977fe640 10 client.16635 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.661+0000 7fa4977fe640 1 -- 172.21.15.37:0/1116060137 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555f818efb30 con 0x555f819ef350 2024-03-26T06:36:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.661+0000 7fa4977fe640 10 client.16635 handle_command_reply: tid=0 2024-03-26T06:36:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.661+0000 7fa4c1172740 1 client.16635 shutdown 2024-03-26T06:36:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.662+0000 7fa4c1172740 1 -- 172.21.15.37:0/1116060137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 msgr2=0x555f819820c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:50.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.662+0000 7fa4c1172740 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 0x555f819820c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00eec30 tx=0x7fa4b00bbbd0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.662+0000 7fa4c1172740 1 -- 172.21.15.37:0/1116060137 shutdown_connections 2024-03-26T06:36:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.662+0000 7fa4c1172740 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x555f819ef350 0x555f819f1770 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.662+0000 7fa4977fe640 0 client.16635 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:36:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.662+0000 7fa4c1172740 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f81987550 0x555f819820c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.662+0000 7fa4c1172740 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f8197ed40 0x555f81981b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.662+0000 7fa4c1172740 1 --2- 172.21.15.37:0/1116060137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f8197dfd0 0x555f81981620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.663+0000 7fa4c1172740 1 -- 172.21.15.37:0/1116060137 >> 172.21.15.37:0/1116060137 conn(0x555f819455b0 msgr2=0x555f819775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.663+0000 7fa4c1172740 1 -- 172.21.15.37:0/1116060137 shutdown_connections 2024-03-26T06:36:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.663+0000 7fa4c1172740 1 -- 172.21.15.37:0/1116060137 wait complete. 2024-03-26T06:36:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.663+0000 7fa4c1172740 20 client.16635 trim_cache size 0 max 16384 2024-03-26T06:36:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:36:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:36:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:36:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:36:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.664+0000 7fa4be590640 1 -- 172.21.15.37:0/1367679473 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa4a00a2ae0 msgr2=0x7fa4a00a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.664+0000 7fa4be590640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa4a00a2ae0 0x7fa4a00a4fa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac006fd0 tx=0x7fa4ac007870 comp rx=0 tx=0).stop 2024-03-26T06:36:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.664+0000 7fa4be590640 1 -- 172.21.15.37:0/1367679473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8114440 msgr2=0x7fa4b810c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:36:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.664+0000 7fa4be590640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8114440 0x7fa4b810c9f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0036040 tx=0x7fa4b00336b0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.664+0000 7fa4be590640 1 -- 172.21.15.37:0/1367679473 shutdown_connections 2024-03-26T06:36:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.665+0000 7fa4be590640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa4a00a2ae0 0x7fa4a00a4fa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.665+0000 7fa4be590640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8114440 0x7fa4b810c9f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.665+0000 7fa4be590640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b8108c10 0x7fa4b810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.665+0000 7fa4be590640 1 --2- 172.21.15.37:0/1367679473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b81082f0 0x7fa4b810bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:36:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.665+0000 7fa4be590640 1 -- 172.21.15.37:0/1367679473 >> 172.21.15.37:0/1367679473 conn(0x7fa4b806c820 msgr2=0x7fa4b8109c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:36:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.665+0000 7fa4be590640 1 -- 172.21.15.37:0/1367679473 shutdown_connections 2024-03-26T06:36:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:36:50.665+0000 7fa4be590640 1 -- 172.21.15.37:0/1367679473 wait complete. 2024-03-26T06:36:50.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 23939 58.84 MB/sec warmup 25 sec latency 109.045 ms 2024-03-26T06:36:51.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:50 smithi066 ceph-mon[36504]: pgmap v434: 97 pgs: 97 active+clean; 479 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 800 op/s 2024-03-26T06:36:51.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:50 smithi149 ceph-mon[36681]: pgmap v434: 97 pgs: 97 active+clean; 479 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 800 op/s 2024-03-26T06:36:51.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:50 smithi037 ceph-mon[31974]: pgmap v434: 97 pgs: 97 active+clean; 479 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 800 op/s 2024-03-26T06:36:51.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 24963 58.28 MB/sec warmup 26 sec latency 51.576 ms 2024-03-26T06:36:52.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 26142 58.75 MB/sec warmup 27 sec latency 56.182 ms 2024-03-26T06:36:53.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:52 smithi066 ceph-mon[36504]: pgmap v435: 97 pgs: 97 active+clean; 672 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 39 MiB/s wr, 1.32k op/s 2024-03-26T06:36:53.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:52 smithi149 ceph-mon[36681]: pgmap v435: 97 pgs: 97 active+clean; 672 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 39 MiB/s wr, 1.32k op/s 2024-03-26T06:36:53.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:52 smithi037 ceph-mon[31974]: pgmap v435: 97 pgs: 97 active+clean; 672 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 39 MiB/s wr, 1.32k op/s 2024-03-26T06:36:53.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 27020 58.11 MB/sec warmup 28 sec latency 173.347 ms 2024-03-26T06:36:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 28126 57.89 MB/sec warmup 29 sec latency 54.178 ms 2024-03-26T06:36:55.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:54 smithi066 ceph-mon[36504]: pgmap v436: 97 pgs: 97 active+clean; 672 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 857 op/s 2024-03-26T06:36:55.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:54 smithi149 ceph-mon[36681]: pgmap v436: 97 pgs: 97 active+clean; 672 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 857 op/s 2024-03-26T06:36:55.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:54 smithi037 ceph-mon[31974]: pgmap v436: 97 pgs: 97 active+clean; 672 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 857 op/s 2024-03-26T06:36:55.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 29336 58.68 MB/sec warmup 30 sec latency 123.004 ms 2024-03-26T06:36:56.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 30219 58.27 MB/sec warmup 31 sec latency 44.292 ms 2024-03-26T06:36:57.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:56 smithi066 ceph-mon[36504]: pgmap v437: 97 pgs: 97 active+clean; 557 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 24 MiB/s wr, 1.38k op/s 2024-03-26T06:36:57.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:56 smithi149 ceph-mon[36681]: pgmap v437: 97 pgs: 97 active+clean; 557 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 24 MiB/s wr, 1.38k op/s 2024-03-26T06:36:57.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:56 smithi037 ceph-mon[31974]: pgmap v437: 97 pgs: 97 active+clean; 557 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 24 MiB/s wr, 1.38k op/s 2024-03-26T06:36:57.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 31150 57.84 MB/sec warmup 32 sec latency 71.597 ms 2024-03-26T06:36:58.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:36:57 smithi066 ceph-mon[36504]: pgmap v438: 97 pgs: 97 active+clean; 541 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 22 MiB/s wr, 1.11k op/s 2024-03-26T06:36:58.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:36:57 smithi149 ceph-mon[36681]: pgmap v438: 97 pgs: 97 active+clean; 541 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 22 MiB/s wr, 1.11k op/s 2024-03-26T06:36:58.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:36:57 smithi037 ceph-mon[31974]: pgmap v438: 97 pgs: 97 active+clean; 541 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 22 MiB/s wr, 1.11k op/s 2024-03-26T06:36:58.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 32342 58.00 MB/sec warmup 33 sec latency 43.629 ms 2024-03-26T06:36:59.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 33436 58.28 MB/sec warmup 34 sec latency 42.435 ms 2024-03-26T06:37:00.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:00 smithi066 ceph-mon[36504]: pgmap v439: 97 pgs: 97 active+clean; 541 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 1.10k op/s 2024-03-26T06:37:00.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:00 smithi149 ceph-mon[36681]: pgmap v439: 97 pgs: 97 active+clean; 541 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 1.10k op/s 2024-03-26T06:37:00.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:00 smithi037 ceph-mon[31974]: pgmap v439: 97 pgs: 97 active+clean; 541 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 21 MiB/s wr, 1.10k op/s 2024-03-26T06:37:00.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 33918 57.17 MB/sec warmup 35 sec latency 239.431 ms 2024-03-26T06:37:01.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 34384 56.17 MB/sec warmup 36 sec latency 258.484 ms 2024-03-26T06:37:02.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:02 smithi066 ceph-mon[36504]: pgmap v440: 97 pgs: 97 active+clean; 570 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 24 MiB/s wr, 1.42k op/s 2024-03-26T06:37:02.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:02 smithi149 ceph-mon[36681]: pgmap v440: 97 pgs: 97 active+clean; 570 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 24 MiB/s wr, 1.42k op/s 2024-03-26T06:37:02.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:02 smithi037 ceph-mon[31974]: pgmap v440: 97 pgs: 97 active+clean; 570 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 24 MiB/s wr, 1.42k op/s 2024-03-26T06:37:02.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 35380 56.17 MB/sec warmup 37 sec latency 64.173 ms 2024-03-26T06:37:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.794+0000 7f19dcb1b640 1 -- 172.21.15.37:0/2577139686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19d81073e0 msgr2=0x7f19d81077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.794+0000 7f19dcb1b640 1 --2- 172.21.15.37:0/2577139686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19d81073e0 0x7f19d81077c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009900 tx=0x7f19c802bcd0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.794+0000 7f19dcb1b640 1 -- 172.21.15.37:0/2577139686 shutdown_connections 2024-03-26T06:37:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.794+0000 7f19dcb1b640 1 --2- 172.21.15.37:0/2577139686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d8069310 0x7f19d810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.794+0000 7f19dcb1b640 1 --2- 172.21.15.37:0/2577139686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d8068970 0x7f19d8068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.794+0000 7f19dcb1b640 1 --2- 172.21.15.37:0/2577139686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19d81073e0 0x7f19d81077c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.795+0000 7f19dcb1b640 1 -- 172.21.15.37:0/2577139686 >> 172.21.15.37:0/2577139686 conn(0x7f19d8077040 msgr2=0x7f19d8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.795+0000 7f19dcb1b640 1 -- 172.21.15.37:0/2577139686 shutdown_connections 2024-03-26T06:37:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.795+0000 7f19dcb1b640 1 -- 172.21.15.37:0/2577139686 wait complete. 2024-03-26T06:37:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.795+0000 7f19dcb1b640 1 Processor -- start 2024-03-26T06:37:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.795+0000 7f19dcb1b640 1 -- start start 2024-03-26T06:37:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.795+0000 7f19dcb1b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d8068970 0x7f19d8100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.795+0000 7f19dcb1b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d8069310 0x7f19d8100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19dcb1b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19d81073e0 0x7f19d8100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19dcb1b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d8111660 con 0x7f19d81073e0 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19dcb1b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81114e0 con 0x7f19d8069310 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19dcb1b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81117e0 con 0x7f19d8068970 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19d81073e0 0x7f19d8100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19d81073e0 0x7f19d8100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56144/0 (socket says 172.21.15.37:56144) 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d6d9d640 1 -- 172.21.15.37:0/484436978 learned_addr learned my addr 172.21.15.37:0/484436978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d659c640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d8068970 0x7f19d8100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d5d9b640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d8069310 0x7f19d8100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d659c640 1 -- 172.21.15.37:0/484436978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d8069310 msgr2=0x7f19d8100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d659c640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d8069310 0x7f19d8100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d659c640 1 -- 172.21.15.37:0/484436978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19d81073e0 msgr2=0x7f19d8100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d659c640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19d81073e0 0x7f19d8100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d659c640 1 -- 172.21.15.37:0/484436978 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d81a5f60 con 0x7f19d8068970 2024-03-26T06:37:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d6d9d640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19d81073e0 0x7f19d8100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:37:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.796+0000 7f19d659c640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d8068970 0x7f19d8100510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19c8000c00 tx=0x7f19c802cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.797+0000 7f19c77fe640 1 -- 172.21.15.37:0/484436978 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8041070 con 0x7f19d8068970 2024-03-26T06:37:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.797+0000 7f19dcb1b640 1 -- 172.21.15.37:0/484436978 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d81a61f0 con 0x7f19d8068970 2024-03-26T06:37:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.797+0000 7f19c77fe640 1 -- 172.21.15.37:0/484436978 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19c80054e0 con 0x7f19d8068970 2024-03-26T06:37:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.797+0000 7f19c77fe640 1 -- 172.21.15.37:0/484436978 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8003d00 con 0x7f19d8068970 2024-03-26T06:37:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.797+0000 7f19dcb1b640 1 -- 172.21.15.37:0/484436978 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d81a6700 con 0x7f19d8068970 2024-03-26T06:37:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.799+0000 7f19df7d9740 1 Processor -- start 2024-03-26T06:37:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19df7d9740 1 -- start start 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 0x556d54214a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19c77fe640 1 -- 172.21.15.37:0/484436978 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f19c8044070 con 0x7f19d8068970 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54214fd0 0x556d54215450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d5421e550 0x556d54220940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19df7d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d541795f0 con 0x556d5421e550 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19df7d9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d54122930 con 0x556d54215d40 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19df7d9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d54175690 con 0x556d54214fd0 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d5421e550 0x556d54220940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d659c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54214fd0 0x556d54215450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d5421e550 0x556d54220940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56160/0 (socket says 172.21.15.37:56160) 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d659c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54214fd0 0x556d54215450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:35126/0 (socket says 172.21.15.37:35126) 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d5d9b640 1 -- 172.21.15.37:0/218401988 learned_addr learned my addr 172.21.15.37:0/218401988 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d6d9d640 1 --2- 172.21.15.37:0/218401988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 0x556d54214a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d5d9b640 1 -- 172.21.15.37:0/218401988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54214fd0 msgr2=0x556d54215450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d5d9b640 1 --2- 172.21.15.37:0/218401988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54214fd0 0x556d54215450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d5d9b640 1 -- 172.21.15.37:0/218401988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 msgr2=0x556d54214a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d5d9b640 1 --2- 172.21.15.37:0/218401988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 0x556d54214a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d5d9b640 1 -- 172.21.15.37:0/218401988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d53e8e210 con 0x556d5421e550 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.800+0000 7f19d6d9d640 1 --2- 172.21.15.37:0/218401988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 0x556d54214a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.801+0000 7f19d5d9b640 1 --2- 172.21.15.37:0/218401988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d5421e550 0x556d54220940 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009c90 tx=0x7f19cc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf13664ec9226c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.801+0000 7f19c77fe640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f19c00a2a10 0x7f19c00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.801+0000 7f19c4ff9640 1 -- 172.21.15.37:0/218401988 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc01e070 con 0x556d5421e550 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.801+0000 7f19d6d9d640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f19c00a2a10 0x7f19c00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.801+0000 7f19c77fe640 1 -- 172.21.15.37:0/484436978 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f19c80f0600 con 0x7f19d8068970 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.801+0000 7f19c4ff9640 1 -- 172.21.15.37:0/218401988 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19cc009e40 con 0x556d5421e550 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19c4ff9640 1 -- 172.21.15.37:0/218401988 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00ea90 con 0x556d5421e550 2024-03-26T06:37:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19df7d9740 1 -- 172.21.15.37:0/218401988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d5421e550 msgr2=0x556d54220940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19df7d9740 1 --2- 172.21.15.37:0/218401988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d5421e550 0x556d54220940 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009c90 tx=0x7f19cc011810 comp rx=0 tx=0).stop 2024-03-26T06:37:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19d6d9d640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f19c00a2a10 0x7f19c00a4ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009fd0 tx=0x7f19d0009500 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19df7d9740 1 -- 172.21.15.37:0/218401988 shutdown_connections 2024-03-26T06:37:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19df7d9740 1 --2- 172.21.15.37:0/218401988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d5421e550 0x556d54220940 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19df7d9740 1 --2- 172.21.15.37:0/218401988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54214fd0 0x556d54215450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19df7d9740 1 --2- 172.21.15.37:0/218401988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 0x556d54214a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19df7d9740 1 -- 172.21.15.37:0/218401988 >> 172.21.15.37:0/218401988 conn(0x556d541dc5b0 msgr2=0x556d5420fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19df7d9740 1 -- 172.21.15.37:0/218401988 shutdown_connections 2024-03-26T06:37:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19df7d9740 1 -- 172.21.15.37:0/218401988 wait complete. 2024-03-26T06:37:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19df7d9740 1 -- start start 2024-03-26T06:37:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.802+0000 7f19df7d9740 1 Processor -- start 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54215d40 0x556d5421b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d5421e550 0x556d5421bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19df7d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d54221320 con 0x556d54214fd0 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19df7d9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d54135d90 con 0x556d5421e550 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19df7d9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d542214a0 con 0x556d54215d40 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19df7d9740 10 client.0 _ll_register_callbacks cb 0x556d53e8da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19d659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19d659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56164/0 (socket says 172.21.15.37:56164) 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19d659c640 1 -- 172.21.15.37:0/2050251089 learned_addr learned my addr 172.21.15.37:0/2050251089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19d5d9b640 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54215d40 0x556d5421b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19d6d9d640 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d5421e550 0x556d5421bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19d659c640 1 -- 172.21.15.37:0/2050251089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54215d40 msgr2=0x556d5421b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19d659c640 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54215d40 0x556d5421b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19d659c640 1 -- 172.21.15.37:0/2050251089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d5421e550 msgr2=0x556d5421bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19d659c640 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d5421e550 0x556d5421bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19d659c640 1 -- 172.21.15.37:0/2050251089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d53f5a620 con 0x556d54214fd0 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19d659c640 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421b420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f19c80b9280 tx=0x7f19c8003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f875ed402c0b45 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19bdffb640 1 -- 172.21.15.37:0/2050251089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c804b070 con 0x556d54214fd0 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19bdffb640 1 -- 172.21.15.37:0/2050251089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f19c80b9490 con 0x556d54214fd0 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.803+0000 7f19df7d9740 1 -- 172.21.15.37:0/2050251089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d54186b30 con 0x556d54214fd0 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19bdffb640 1 -- 172.21.15.37:0/2050251089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19c80bcbf0 con 0x556d54214fd0 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19df7d9740 10 client.16647 fetch_fsmap learned FSMap version 54 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19df7d9740 1 -- 172.21.15.37:0/2050251089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x556d5422d540 con 0x556d54214fd0 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19bdffb640 1 -- 172.21.15.37:0/2050251089 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556d54186b30 con 0x556d54214fd0 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19bdffb640 1 -- 172.21.15.37:0/2050251089 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f19c80c5080 con 0x556d54214fd0 2024-03-26T06:37:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19df7d9740 10 client.16647 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19df7d9740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19df7d9740 20 client.16647 populate_metadata read hostname 'smithi037' 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19df7d9740 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556d5422e980 0x556d54230de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19df7d9740 4 client.16647 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19df7d9740 1 -- 172.21.15.37:0/2050251089 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d54180e80 con 0x556d5422e980 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.804+0000 7f19d5d9b640 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556d5422e980 0x556d54230de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.805+0000 7f19d5d9b640 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556d5422e980 0x556d54230de0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed1bf2cf575823d5 server_cookie=393a06f9bc58b044 in_seq=0 out_seq=0 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.805+0000 7f19bdffb640 10 client.16647 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.805+0000 7f19bdffb640 1 -- 172.21.15.37:0/2050251089 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d54180e80 con 0x556d5422e980 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.805+0000 7f19bdffb640 10 client.16647 handle_command_reply: tid=0 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.805+0000 7f19df7d9740 1 client.16647 shutdown 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.805+0000 7f19df7d9740 1 -- 172.21.15.37:0/2050251089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 msgr2=0x556d5421b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.805+0000 7f19df7d9740 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421b420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f19c80b9280 tx=0x7f19c8003ea0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.805+0000 7f19df7d9740 1 -- 172.21.15.37:0/2050251089 shutdown_connections 2024-03-26T06:37:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.806+0000 7f19df7d9740 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556d5422e980 0x556d54230de0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.806+0000 7f19bdffb640 0 client.16647 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.806+0000 7f19df7d9740 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d5421e550 0x556d5421bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.806+0000 7f19df7d9740 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54215d40 0x556d5421b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.806+0000 7f19df7d9740 1 --2- 172.21.15.37:0/2050251089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421b420 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.806+0000 7f19df7d9740 1 -- 172.21.15.37:0/2050251089 >> 172.21.15.37:0/2050251089 conn(0x556d541dc5b0 msgr2=0x556d54211a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.806+0000 7f19df7d9740 1 -- 172.21.15.37:0/2050251089 shutdown_connections 2024-03-26T06:37:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.806+0000 7f19df7d9740 1 -- 172.21.15.37:0/2050251089 wait complete. 2024-03-26T06:37:03.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.806+0000 7f19df7d9740 20 client.16647 trim_cache size 0 max 16384 2024-03-26T06:37:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19df7d9740 1 Processor -- start 2024-03-26T06:37:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19df7d9740 1 -- start start 2024-03-26T06:37:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54215d40 0x556d54219cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d5421e550 0x556d5421a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19df7d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81aede0 con 0x556d54214fd0 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19df7d9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81b0940 con 0x556d5421e550 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19df7d9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81af440 con 0x556d54215d40 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19d659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19d659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56166/0 (socket says 172.21.15.37:56166) 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19d659c640 1 -- 172.21.15.37:0/848955025 learned_addr learned my addr 172.21.15.37:0/848955025 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.830+0000 7f19d5d9b640 1 --2- 172.21.15.37:0/848955025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54215d40 0x556d54219cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.831+0000 7f19d659c640 1 -- 172.21.15.37:0/848955025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54215d40 msgr2=0x556d54219cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.831+0000 7f19d659c640 1 --2- 172.21.15.37:0/848955025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54215d40 0x556d54219cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.831+0000 7f19d659c640 1 -- 172.21.15.37:0/848955025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d5421e550 msgr2=0x556d5421a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.831+0000 7f19d659c640 1 --2- 172.21.15.37:0/848955025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d5421e550 0x556d5421a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.831+0000 7f19d659c640 1 -- 172.21.15.37:0/848955025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d54220cb0 con 0x556d54214fd0 2024-03-26T06:37:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.831+0000 7f19d5d9b640 1 --2- 172.21.15.37:0/848955025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54215d40 0x556d54219cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:37:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.831+0000 7f19d659c640 1 --2- 172.21.15.37:0/848955025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421db50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f19c802e040 tx=0x7f19c802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d2bcb63d7ca6a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.832+0000 7f19bffff640 1 -- 172.21.15.37:0/848955025 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c804b070 con 0x556d54214fd0 2024-03-26T06:37:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.832+0000 7f19bffff640 1 -- 172.21.15.37:0/848955025 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19c8004c80 con 0x556d54214fd0 2024-03-26T06:37:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.832+0000 7f19bffff640 1 -- 172.21.15.37:0/848955025 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c80bc780 con 0x556d54214fd0 2024-03-26T06:37:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.833+0000 7f19df7d9740 1 -- 172.21.15.37:0/848955025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 msgr2=0x556d5421db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.833+0000 7f19df7d9740 1 --2- 172.21.15.37:0/848955025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421db50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f19c802e040 tx=0x7f19c802bcd0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.833+0000 7f19df7d9740 1 -- 172.21.15.37:0/848955025 shutdown_connections 2024-03-26T06:37:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.833+0000 7f19df7d9740 1 --2- 172.21.15.37:0/848955025 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d5421e550 0x556d5421a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.833+0000 7f19df7d9740 1 --2- 172.21.15.37:0/848955025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d54215d40 0x556d54219cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.833+0000 7f19df7d9740 1 --2- 172.21.15.37:0/848955025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d5421db50 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.833+0000 7f19df7d9740 1 -- 172.21.15.37:0/848955025 >> 172.21.15.37:0/848955025 conn(0x556d541dc5b0 msgr2=0x556d54235780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.833+0000 7f19df7d9740 1 -- 172.21.15.37:0/848955025 shutdown_connections 2024-03-26T06:37:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.833+0000 7f19df7d9740 1 -- 172.21.15.37:0/848955025 wait complete. 2024-03-26T06:37:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.833+0000 7f19df7d9740 1 -- start start 2024-03-26T06:37:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19df7d9740 1 Processor -- start 2024-03-26T06:37:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d54225580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 0x556d54225ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19df7d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d5421e550 0x556d54223be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19df7d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81b0220 con 0x556d54214fd0 2024-03-26T06:37:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19df7d9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81afb60 con 0x556d54215d40 2024-03-26T06:37:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19df7d9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81ad0a0 con 0x556d5421e550 2024-03-26T06:37:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19df7d9740 10 client.0 _ll_register_callbacks cb 0x7f19cc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:37:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19d5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 0x556d54225ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19d5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 0x556d54225ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:35458/0 (socket says 172.21.15.37:35458) 2024-03-26T06:37:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19d5d9b640 1 -- 172.21.15.37:0/1466746029 learned_addr learned my addr 172.21.15.37:0/1466746029 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19d6d9d640 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d5421e550 0x556d54223be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.834+0000 7f19d659c640 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d54225580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19d6d9d640 1 -- 172.21.15.37:0/1466746029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 msgr2=0x556d54225ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19d6d9d640 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 0x556d54225ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19d6d9d640 1 -- 172.21.15.37:0/1466746029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 msgr2=0x556d54225580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19d6d9d640 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d54225580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19d6d9d640 1 -- 172.21.15.37:0/1466746029 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d54224410 con 0x556d5421e550 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19d5d9b640 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 0x556d54225ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19d6d9d640 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d5421e550 0x556d54223be0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x556d54210ce0 tx=0x7f19d0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ded30bf668472fb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19bdffb640 1 -- 172.21.15.37:0/1466746029 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d002f060 con 0x556d5421e550 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19bdffb640 1 -- 172.21.15.37:0/1466746029 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f19d0011ca0 con 0x556d5421e550 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19df7d9740 1 -- 172.21.15.37:0/1466746029 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19c8004c80 con 0x556d5421e550 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19bdffb640 1 -- 172.21.15.37:0/1466746029 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19d0011ea0 con 0x556d5421e550 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19df7d9740 10 client.26038 fetch_fsmap learned FSMap version 54 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.835+0000 7f19df7d9740 1 -- 172.21.15.37:0/1466746029 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x556d54285a90 con 0x556d5421e550 2024-03-26T06:37:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.836+0000 7f19bdffb640 1 -- 172.21.15.37:0/1466746029 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19c8004c80 con 0x556d5421e550 2024-03-26T06:37:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.836+0000 7f19bdffb640 1 -- 172.21.15.37:0/1466746029 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 54) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f19d001f6f0 con 0x556d5421e550 2024-03-26T06:37:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.836+0000 7f19df7d9740 10 client.26038 fetch_fsmap finished waiting for FSMap version 54 2024-03-26T06:37:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.836+0000 7f19df7d9740 10 client.26038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:37:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.836+0000 7f19df7d9740 20 client.26038 populate_metadata read hostname 'smithi037' 2024-03-26T06:37:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.836+0000 7f19df7d9740 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556d5422e980 0x556d54288260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.836+0000 7f19df7d9740 4 client.26038 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:37:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.836+0000 7f19df7d9740 1 -- 172.21.15.37:0/1466746029 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556d540ff710 con 0x556d5422e980 2024-03-26T06:37:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.836+0000 7f19d659c640 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556d5422e980 0x556d54288260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.837+0000 7f19bdffb640 10 client.26038 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:03.837+0000 7f19d659c640 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556d5422e980 0x556d54288260 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea8d76a6b09f425b server_cookie=478b3239e8f182c0 in_seq=0 out_seq=0 2024-03-26T06:37:03.845 INFO:tasks.workunit.client.0.smithi037.stdout: 10 36428 56.31 MB/sec warmup 38 sec latency 88.111 ms 2024-03-26T06:37:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19bdffb640 1 -- 172.21.15.37:0/1466746029 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556d540ff710 con 0x556d5422e980 2024-03-26T06:37:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19bdffb640 10 client.26038 handle_command_reply: tid=0 2024-03-26T06:37:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 1 client.26038 shutdown 2024-03-26T06:37:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 1 -- 172.21.15.37:0/1466746029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d5421e550 msgr2=0x556d54223be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d5421e550 0x556d54223be0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x556d54210ce0 tx=0x7f19d0004ab0 comp rx=0 tx=0).stop 2024-03-26T06:37:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 1 -- 172.21.15.37:0/1466746029 shutdown_connections 2024-03-26T06:37:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556d5422e980 0x556d54288260 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19bdffb640 0 client.26038 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d5421e550 0x556d54223be0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556d54215d40 0x556d54225ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 1 --2- 172.21.15.37:0/1466746029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d54214fd0 0x556d54225580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 1 -- 172.21.15.37:0/1466746029 >> 172.21.15.37:0/1466746029 conn(0x556d541dc5b0 msgr2=0x556d5421b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 1 -- 172.21.15.37:0/1466746029 shutdown_connections 2024-03-26T06:37:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 1 -- 172.21.15.37:0/1466746029 wait complete. 2024-03-26T06:37:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.042+0000 7f19df7d9740 20 client.26038 trim_cache size 0 max 16384 2024-03-26T06:37:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:37:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:37:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:37:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.043+0000 7f19dcb1b640 1 -- 172.21.15.37:0/484436978 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f19c00a2a10 msgr2=0x7f19c00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.043+0000 7f19dcb1b640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f19c00a2a10 0x7f19c00a4ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009fd0 tx=0x7f19d0009500 comp rx=0 tx=0).stop 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.043+0000 7f19dcb1b640 1 -- 172.21.15.37:0/484436978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d8068970 msgr2=0x7f19d8100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.043+0000 7f19dcb1b640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d8068970 0x7f19d8100510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19c8000c00 tx=0x7f19c802cb50 comp rx=0 tx=0).stop 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.043+0000 7f19dcb1b640 1 -- 172.21.15.37:0/484436978 shutdown_connections 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.043+0000 7f19dcb1b640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f19c00a2a10 0x7f19c00a4ed0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.043+0000 7f19dcb1b640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19d81073e0 0x7f19d8100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.043+0000 7f19dcb1b640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d8069310 0x7f19d8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.043+0000 7f19dcb1b640 1 --2- 172.21.15.37:0/484436978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d8068970 0x7f19d8100510 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.043+0000 7f19dcb1b640 1 -- 172.21.15.37:0/484436978 >> 172.21.15.37:0/484436978 conn(0x7f19d8077040 msgr2=0x7f19d80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.044+0000 7f19dcb1b640 1 -- 172.21.15.37:0/484436978 shutdown_connections 2024-03-26T06:37:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:04.044+0000 7f19dcb1b640 1 -- 172.21.15.37:0/484436978 wait complete. 2024-03-26T06:37:04.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:04 smithi066 ceph-mon[36504]: pgmap v441: 97 pgs: 97 active+clean; 570 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.3 MiB/s wr, 903 op/s 2024-03-26T06:37:04.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:04 smithi149 ceph-mon[36681]: pgmap v441: 97 pgs: 97 active+clean; 570 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.3 MiB/s wr, 903 op/s 2024-03-26T06:37:04.411 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:37:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: ignoring --setuser ceph since I am not root 2024-03-26T06:37:04.411 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:37:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: ignoring --setgroup ceph since I am not root 2024-03-26T06:37:04.412 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:37:04 smithi149 ceph-mds[67761]: -- 172.21.15.149:0/3859193927 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56483f8ba1a0 con 0x56483f890c00 2024-03-26T06:37:04.412 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:37:04 smithi149 ceph-mds[67761]: -- 172.21.15.149:0/3859193927 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56483f8bda40 con 0x56483f890c00 2024-03-26T06:37:04.412 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:37:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: starting mds.c at 2024-03-26T06:37:04.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:04 smithi037 ceph-mon[31974]: pgmap v441: 97 pgs: 97 active+clean; 570 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.3 MiB/s wr, 903 op/s 2024-03-26T06:37:04.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 37203 56.11 MB/sec warmup 39 sec latency 98.936 ms 2024-03-26T06:37:05.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:05 smithi066 ceph-mon[36504]: daemon mds.c restarted 2024-03-26T06:37:05.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:05 smithi066 ceph-mon[36504]: osdmap e105: 12 total, 12 up, 12 in 2024-03-26T06:37:05.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:05 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6836/1243624880,v1:172.21.15.149:6837/1243624880] up:boot 2024-03-26T06:37:05.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:05 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:37:05.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:05 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:37:05.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:05 smithi149 ceph-mon[36681]: daemon mds.c restarted 2024-03-26T06:37:05.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:05 smithi149 ceph-mon[36681]: osdmap e105: 12 total, 12 up, 12 in 2024-03-26T06:37:05.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:05 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6836/1243624880,v1:172.21.15.149:6837/1243624880] up:boot 2024-03-26T06:37:05.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:05 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:37:05.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:05 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:37:05.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:05 smithi037 ceph-mon[31974]: daemon mds.c restarted 2024-03-26T06:37:05.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:05 smithi037 ceph-mon[31974]: osdmap e105: 12 total, 12 up, 12 in 2024-03-26T06:37:05.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:05 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6836/1243624880,v1:172.21.15.149:6837/1243624880] up:boot 2024-03-26T06:37:05.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:05 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:37:05.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:05 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:37:05.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 38022 55.53 MB/sec warmup 40 sec latency 122.265 ms 2024-03-26T06:37:06.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 38924 55.28 MB/sec warmup 41 sec latency 143.582 ms 2024-03-26T06:37:07.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:06 smithi066 ceph-mon[36504]: pgmap v443: 97 pgs: 97 active+clean; 583 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 7.6 MiB/s wr, 1.03k op/s 2024-03-26T06:37:07.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:06 smithi066 ceph-mon[36504]: osdmap e106: 12 total, 12 up, 12 in 2024-03-26T06:37:07.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:06 smithi149 ceph-mon[36681]: pgmap v443: 97 pgs: 97 active+clean; 583 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 7.6 MiB/s wr, 1.03k op/s 2024-03-26T06:37:07.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:06 smithi149 ceph-mon[36681]: osdmap e106: 12 total, 12 up, 12 in 2024-03-26T06:37:07.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:06 smithi037 ceph-mon[31974]: pgmap v443: 97 pgs: 97 active+clean; 583 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 7.6 MiB/s wr, 1.03k op/s 2024-03-26T06:37:07.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:06 smithi037 ceph-mon[31974]: osdmap e106: 12 total, 12 up, 12 in 2024-03-26T06:37:07.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 40257 55.81 MB/sec warmup 42 sec latency 47.410 ms 2024-03-26T06:37:08.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 41212 55.93 MB/sec warmup 43 sec latency 42.676 ms 2024-03-26T06:37:09.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:08 smithi066 ceph-mon[36504]: pgmap v445: 97 pgs: 97 active+clean; 587 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 9.9 MiB/s wr, 1.33k op/s 2024-03-26T06:37:09.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:08 smithi149 ceph-mon[36681]: pgmap v445: 97 pgs: 97 active+clean; 587 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 9.9 MiB/s wr, 1.33k op/s 2024-03-26T06:37:09.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:08 smithi037 ceph-mon[31974]: pgmap v445: 97 pgs: 97 active+clean; 587 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 9.9 MiB/s wr, 1.33k op/s 2024-03-26T06:37:09.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 41412 54.78 MB/sec warmup 44 sec latency 740.214 ms 2024-03-26T06:37:10.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:09 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:37:10.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:09 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:37:10.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:09 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:37:10.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 41615 53.75 MB/sec warmup 45 sec latency 847.306 ms 2024-03-26T06:37:11.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:10 smithi066 ceph-mon[36504]: pgmap v446: 97 pgs: 97 active+clean; 587 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.4 MiB/s wr, 846 op/s 2024-03-26T06:37:11.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:10 smithi149 ceph-mon[36681]: pgmap v446: 97 pgs: 97 active+clean; 587 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.4 MiB/s wr, 846 op/s 2024-03-26T06:37:11.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:10 smithi037 ceph-mon[31974]: pgmap v446: 97 pgs: 97 active+clean; 587 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 5.4 MiB/s wr, 846 op/s 2024-03-26T06:37:11.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 42139 53.11 MB/sec warmup 46 sec latency 85.044 ms 2024-03-26T06:37:12.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 43287 53.14 MB/sec warmup 47 sec latency 67.664 ms 2024-03-26T06:37:13.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:12 smithi066 ceph-mon[36504]: pgmap v447: 97 pgs: 97 active+clean; 776 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 30 MiB/s wr, 1.63k op/s 2024-03-26T06:37:13.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:12 smithi149 ceph-mon[36681]: pgmap v447: 97 pgs: 97 active+clean; 776 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 30 MiB/s wr, 1.63k op/s 2024-03-26T06:37:13.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:12 smithi037 ceph-mon[31974]: pgmap v447: 97 pgs: 97 active+clean; 776 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 30 MiB/s wr, 1.63k op/s 2024-03-26T06:37:13.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 44422 53.72 MB/sec warmup 48 sec latency 36.752 ms 2024-03-26T06:37:14.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:13 smithi149 ceph-mon[36681]: pgmap v448: 97 pgs: 97 active+clean; 776 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 30 MiB/s wr, 1.59k op/s 2024-03-26T06:37:14.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:13 smithi037 ceph-mon[31974]: pgmap v448: 97 pgs: 97 active+clean; 776 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 30 MiB/s wr, 1.59k op/s 2024-03-26T06:37:14.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:13 smithi066 ceph-mon[36504]: pgmap v448: 97 pgs: 97 active+clean; 776 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 30 MiB/s wr, 1.59k op/s 2024-03-26T06:37:14.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 45228 53.31 MB/sec warmup 49 sec latency 102.087 ms 2024-03-26T06:37:15.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 46135 53.08 MB/sec warmup 50 sec latency 85.044 ms 2024-03-26T06:37:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:15 smithi066 ceph-mon[36504]: osdmap e107: 12 total, 12 up, 12 in 2024-03-26T06:37:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:15 smithi149 ceph-mon[36681]: osdmap e107: 12 total, 12 up, 12 in 2024-03-26T06:37:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:15 smithi037 ceph-mon[31974]: osdmap e107: 12 total, 12 up, 12 in 2024-03-26T06:37:16.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 47348 53.41 MB/sec warmup 51 sec latency 86.922 ms 2024-03-26T06:37:17.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:16 smithi066 ceph-mon[36504]: pgmap v450: 97 pgs: 97 active+clean; 665 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.22k op/s 2024-03-26T06:37:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:16 smithi149 ceph-mon[36681]: pgmap v450: 97 pgs: 97 active+clean; 665 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.22k op/s 2024-03-26T06:37:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.171+0000 7fe34486c640 1 -- 172.21.15.37:0/812335609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34006a710 msgr2=0x7fe340105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.171+0000 7fe34486c640 1 --2- 172.21.15.37:0/812335609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34006a710 0x7fe340105f80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe334009900 tx=0x7fe33402bc30 comp rx=0 tx=0).stop 2024-03-26T06:37:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.171+0000 7fe34486c640 1 -- 172.21.15.37:0/812335609 shutdown_connections 2024-03-26T06:37:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.171+0000 7fe34486c640 1 --2- 172.21.15.37:0/812335609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3401066b0 0x7fe34010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.171+0000 7fe34486c640 1 --2- 172.21.15.37:0/812335609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe34006a710 0x7fe340105f80 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.171+0000 7fe34486c640 1 --2- 172.21.15.37:0/812335609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe340069d60 0x7fe34006a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.171+0000 7fe34486c640 1 -- 172.21.15.37:0/812335609 >> 172.21.15.37:0/812335609 conn(0x7fe340076fc0 msgr2=0x7fe3400773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.171+0000 7fe34486c640 1 -- 172.21.15.37:0/812335609 shutdown_connections 2024-03-26T06:37:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.171+0000 7fe34486c640 1 -- 172.21.15.37:0/812335609 wait complete. 2024-03-26T06:37:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe34486c640 1 Processor -- start 2024-03-26T06:37:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe34486c640 1 -- start start 2024-03-26T06:37:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe34486c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe340069d60 0x7fe34019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe34486c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3401066b0 0x7fe34019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe34486c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe340199b00 0x7fe340199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe34486c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401138c0 con 0x7fe340069d60 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe34486c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe340113740 con 0x7fe340199b00 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe34486c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe340113a40 con 0x7fe3401066b0 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe33e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe340069d60 0x7fe34019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe33e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe340069d60 0x7fe34019f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34412/0 (socket says 172.21.15.37:34412) 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe33e59c640 1 -- 172.21.15.37:0/123980441 learned_addr learned my addr 172.21.15.37:0/123980441 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe33ed9d640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe340199b00 0x7fe340199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.172+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3401066b0 0x7fe34019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.173+0000 7fe33e59c640 1 -- 172.21.15.37:0/123980441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3401066b0 msgr2=0x7fe34019fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.173+0000 7fe33e59c640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3401066b0 0x7fe34019fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.173+0000 7fe33e59c640 1 -- 172.21.15.37:0/123980441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe340199b00 msgr2=0x7fe340199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.173+0000 7fe33e59c640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe340199b00 0x7fe340199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.173+0000 7fe33e59c640 1 -- 172.21.15.37:0/123980441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34019a620 con 0x7fe340069d60 2024-03-26T06:37:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.173+0000 7fe33e59c640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe340069d60 0x7fe34019f900 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe33000d950 tx=0x7fe33000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.174+0000 7fe32f7fe640 1 -- 172.21.15.37:0/123980441 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330017070 con 0x7fe340069d60 2024-03-26T06:37:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.174+0000 7fe32f7fe640 1 -- 172.21.15.37:0/123980441 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe330009c30 con 0x7fe340069d60 2024-03-26T06:37:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.174+0000 7fe34486c640 1 -- 172.21.15.37:0/123980441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34019a910 con 0x7fe340069d60 2024-03-26T06:37:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.175+0000 7fe32f7fe640 1 -- 172.21.15.37:0/123980441 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33000f6b0 con 0x7fe340069d60 2024-03-26T06:37:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.175+0000 7fe34486c640 1 -- 172.21.15.37:0/123980441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3401a6b10 con 0x7fe340069d60 2024-03-26T06:37:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.178+0000 7fe32f7fe640 1 -- 172.21.15.37:0/123980441 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe3300109d0 con 0x7fe340069d60 2024-03-26T06:37:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.178+0000 7fe34744e740 1 Processor -- start 2024-03-26T06:37:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.179+0000 7fe34744e740 1 -- start start 2024-03-26T06:37:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.179+0000 7fe34744e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 0x556041709a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.179+0000 7fe34744e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x55604170a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe34744e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041713550 0x556041715940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe32f7fe640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3280a2c40 0x7fe3280a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe34744e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604166e5f0 con 0x55604170ad40 2024-03-26T06:37:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe34744e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556041617930 con 0x556041709fd0 2024-03-26T06:37:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe34744e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604166a690 con 0x556041713550 2024-03-26T06:37:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe33ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 0x556041709a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe33ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 0x556041709a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34420/0 (socket says 172.21.15.37:34420) 2024-03-26T06:37:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/938702764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041713550 0x556041715940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe33ed9d640 1 -- 172.21.15.37:0/938702764 learned_addr learned my addr 172.21.15.37:0/938702764 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe33ed9d640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3280a2c40 0x7fe3280a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe33dd9b640 1 -- 172.21.15.37:0/938702764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 msgr2=0x55604170a450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T06:37:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/938702764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x55604170a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe33dd9b640 1 -- 172.21.15.37:0/938702764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 msgr2=0x556041709a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/938702764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 0x556041709a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.180+0000 7fe33dd9b640 1 -- 172.21.15.37:0/938702764 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556041383210 con 0x556041713550 2024-03-26T06:37:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.181+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/938702764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041713550 0x556041715940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe340069900 tx=0x7fe334005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ec1779ba449ea5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.181+0000 7fe32f7fe640 1 -- 172.21.15.37:0/123980441 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe3300ce970 con 0x7fe340069d60 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.181+0000 7fe32cff9640 1 -- 172.21.15.37:0/938702764 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334045020 con 0x556041713550 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.181+0000 7fe33ed9d640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3280a2c40 0x7fe3280a5100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe3380099e0 tx=0x7fe338009500 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.181+0000 7fe32cff9640 1 -- 172.21.15.37:0/938702764 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe33402c880 con 0x556041713550 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.181+0000 7fe32cff9640 1 -- 172.21.15.37:0/938702764 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334005640 con 0x556041713550 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.182+0000 7fe34744e740 1 -- 172.21.15.37:0/938702764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041713550 msgr2=0x556041715940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.183+0000 7fe34744e740 1 --2- 172.21.15.37:0/938702764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041713550 0x556041715940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe340069900 tx=0x7fe334005fd0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.183+0000 7fe34744e740 1 -- 172.21.15.37:0/938702764 shutdown_connections 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.183+0000 7fe34744e740 1 --2- 172.21.15.37:0/938702764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041713550 0x556041715940 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe340069900 tx=0x7fe334005fd0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.183+0000 7fe34744e740 1 --2- 172.21.15.37:0/938702764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x55604170a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.183+0000 7fe34744e740 1 --2- 172.21.15.37:0/938702764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 0x556041709a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.183+0000 7fe34744e740 1 -- 172.21.15.37:0/938702764 >> 172.21.15.37:0/938702764 conn(0x5560416d15b0 msgr2=0x556041704cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.184+0000 7fe34744e740 1 -- 172.21.15.37:0/938702764 shutdown_connections 2024-03-26T06:37:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.184+0000 7fe34744e740 1 -- 172.21.15.37:0/938702764 wait complete. 2024-03-26T06:37:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.184+0000 7fe34744e740 1 -- start start 2024-03-26T06:37:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.184+0000 7fe34744e740 1 Processor -- start 2024-03-26T06:37:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe34744e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x556041710740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe34744e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604170ad40 0x556041710c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe34744e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 0x556041727ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe34744e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556041716220 con 0x556041711210 2024-03-26T06:37:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe34744e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604162ad90 con 0x556041709fd0 2024-03-26T06:37:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe34744e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560417163a0 con 0x55604170ad40 2024-03-26T06:37:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe34744e740 10 client.0 _ll_register_callbacks cb 0x556041382a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:37:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe33e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x556041710740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe33dd9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604170ad40 0x556041710c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe33e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x556041710740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:38092/0 (socket says 172.21.15.37:38092) 2024-03-26T06:37:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe33e59c640 1 -- 172.21.15.37:0/3515732106 learned_addr learned my addr 172.21.15.37:0/3515732106 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe33ed9d640 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 0x556041727ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe33e59c640 1 -- 172.21.15.37:0/3515732106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604170ad40 msgr2=0x556041710c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe33e59c640 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604170ad40 0x556041710c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe33e59c640 1 -- 172.21.15.37:0/3515732106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 msgr2=0x556041727ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe33e59c640 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 0x556041727ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.185+0000 7fe33e59c640 1 -- 172.21.15.37:0/3515732106 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55604144f620 con 0x556041709fd0 2024-03-26T06:37:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.186+0000 7fe33e59c640 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x556041710740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe33009afb0 tx=0x7fe330009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f85ed16ad0961ad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.186+0000 7fe325ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:37:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.187+0000 7fe325ffb640 1 -- 172.21.15.37:0/3515732106 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330020040 con 0x556041709fd0 2024-03-26T06:37:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.187+0000 7fe34744e740 1 -- 172.21.15.37:0/3515732106 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe33402c880 con 0x556041709fd0 2024-03-26T06:37:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.187+0000 7fe325ffb640 1 -- 172.21.15.37:0/3515732106 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe330095b90 con 0x556041709fd0 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.188+0000 7fe325ffb640 1 -- 172.21.15.37:0/3515732106 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3300cc4e0 con 0x556041709fd0 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.188+0000 7fe34744e740 10 client.26180 fetch_fsmap learned FSMap version 56 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.188+0000 7fe34744e740 1 -- 172.21.15.37:0/3515732106 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55604172b3c0 con 0x556041709fd0 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.189+0000 7fe325ffb640 1 -- 172.21.15.37:0/3515732106 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe33402c880 con 0x556041709fd0 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.189+0000 7fe325ffb640 1 -- 172.21.15.37:0/3515732106 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fe330099ee0 con 0x556041709fd0 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.189+0000 7fe34744e740 10 client.26180 fetch_fsmap finished waiting for FSMap version 56 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.189+0000 7fe34744e740 10 client.26180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.189+0000 7fe34744e740 20 client.26180 populate_metadata read hostname 'smithi037' 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.189+0000 7fe34744e740 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55604172c880 0x55604172ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.189+0000 7fe34744e740 4 client.26180 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.189+0000 7fe34744e740 1 -- 172.21.15.37:0/3515732106 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55604167bb30 con 0x55604172c880 2024-03-26T06:37:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.190+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55604172c880 0x55604172ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.191+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55604172c880 0x55604172ec90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7469cdad4ad6fc4f server_cookie=c9b0f420da16d3ea in_seq=0 out_seq=0 2024-03-26T06:37:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.191+0000 7fe325ffb640 10 client.26180 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe325ffb640 1 -- 172.21.15.37:0/3515732106 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55604167bb30 con 0x55604172c880 2024-03-26T06:37:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe325ffb640 10 client.26180 handle_command_reply: tid=0 2024-03-26T06:37:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe34744e740 1 client.26180 shutdown 2024-03-26T06:37:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe34744e740 1 -- 172.21.15.37:0/3515732106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 msgr2=0x556041710740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe34744e740 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x556041710740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe33009afb0 tx=0x7fe330009de0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe34744e740 1 -- 172.21.15.37:0/3515732106 shutdown_connections 2024-03-26T06:37:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe34744e740 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55604172c880 0x55604172ec90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe34744e740 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 0x556041727ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe34744e740 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604170ad40 0x556041710c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe34744e740 1 --2- 172.21.15.37:0/3515732106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x556041710740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe34744e740 1 -- 172.21.15.37:0/3515732106 >> 172.21.15.37:0/3515732106 conn(0x5560416d15b0 msgr2=0x556041706a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe325ffb640 0 client.26180 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.196+0000 7fe34744e740 1 -- 172.21.15.37:0/3515732106 shutdown_connections 2024-03-26T06:37:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.197+0000 7fe34744e740 1 -- 172.21.15.37:0/3515732106 wait complete. 2024-03-26T06:37:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.197+0000 7fe34744e740 20 client.26180 trim_cache size 0 max 16384 2024-03-26T06:37:17.201 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:16 smithi037 ceph-mon[31974]: pgmap v450: 97 pgs: 97 active+clean; 665 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.22k op/s 2024-03-26T06:37:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.216+0000 7fe34744e740 1 Processor -- start 2024-03-26T06:37:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.216+0000 7fe34744e740 1 -- start start 2024-03-26T06:37:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.216+0000 7fe34744e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x556041722ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.216+0000 7fe34744e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 0x5560417233e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.216+0000 7fe34744e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041711210 0x556041714940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.216+0000 7fe34744e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401aa800 con 0x55604170ad40 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.216+0000 7fe34744e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401ac330 con 0x556041709fd0 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.216+0000 7fe34744e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401aaf20 con 0x556041711210 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 0x5560417233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x556041722ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33ed9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041711210 0x556041714940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 0x5560417233e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34448/0 (socket says 172.21.15.37:34448) 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x556041722ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:38104/0 (socket says 172.21.15.37:38104) 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33dd9b640 1 -- 172.21.15.37:0/3089663646 learned_addr learned my addr 172.21.15.37:0/3089663646 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33dd9b640 1 -- 172.21.15.37:0/3089663646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041711210 msgr2=0x556041714940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/3089663646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041711210 0x556041714940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33dd9b640 1 -- 172.21.15.37:0/3089663646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 msgr2=0x556041722ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/3089663646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x556041722ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33dd9b640 1 -- 172.21.15.37:0/3089663646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560417092a0 con 0x55604170ad40 2024-03-26T06:37:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.217+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/3089663646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 0x5560417233e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe33402c240 tx=0x7fe33402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76df268c8d696b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.218+0000 7fe327fff640 1 -- 172.21.15.37:0/3089663646 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334050070 con 0x55604170ad40 2024-03-26T06:37:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.218+0000 7fe327fff640 1 -- 172.21.15.37:0/3089663646 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe334005640 con 0x55604170ad40 2024-03-26T06:37:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.218+0000 7fe327fff640 1 -- 172.21.15.37:0/3089663646 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334045040 con 0x55604170ad40 2024-03-26T06:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.218+0000 7fe34744e740 1 -- 172.21.15.37:0/3089663646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 msgr2=0x5560417233e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.218+0000 7fe34744e740 1 --2- 172.21.15.37:0/3089663646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 0x5560417233e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe33402c240 tx=0x7fe33402c650 comp rx=0 tx=0).stop 2024-03-26T06:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.218+0000 7fe34744e740 1 -- 172.21.15.37:0/3089663646 shutdown_connections 2024-03-26T06:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.218+0000 7fe34744e740 1 --2- 172.21.15.37:0/3089663646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041711210 0x556041714940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.218+0000 7fe34744e740 1 --2- 172.21.15.37:0/3089663646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55604170ad40 0x5560417233e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.218+0000 7fe34744e740 1 --2- 172.21.15.37:0/3089663646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556041709fd0 0x556041722ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.219+0000 7fe34744e740 1 -- 172.21.15.37:0/3089663646 >> 172.21.15.37:0/3089663646 conn(0x5560416d15b0 msgr2=0x556041727960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.219+0000 7fe34744e740 1 -- 172.21.15.37:0/3089663646 shutdown_connections 2024-03-26T06:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.219+0000 7fe34744e740 1 -- 172.21.15.37:0/3089663646 wait complete. 2024-03-26T06:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.219+0000 7fe34744e740 1 -- start start 2024-03-26T06:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.219+0000 7fe34744e740 1 Processor -- start 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.219+0000 7fe34744e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041709fd0 0x55604171f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.219+0000 7fe34744e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 0x55604171f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe34744e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55604177fd90 0x556041782180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe34744e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401abcd0 con 0x556041711210 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe34744e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401ab5b0 con 0x55604177fd90 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe34744e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401a8ac0 con 0x556041709fd0 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041709fd0 0x55604171f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 0x55604171f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe34744e740 10 client.0 _ll_register_callbacks cb 0x7fe334009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55604177fd90 0x556041782180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 0x55604171f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34458/0 (socket says 172.21.15.37:34458) 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041709fd0 0x55604171f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:34064/0 (socket says 172.21.15.37:34064) 2024-03-26T06:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33dd9b640 1 -- 172.21.15.37:0/1332354538 learned_addr learned my addr 172.21.15.37:0/1332354538 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33dd9b640 1 -- 172.21.15.37:0/1332354538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041709fd0 msgr2=0x55604171f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/1332354538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041709fd0 0x55604171f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33dd9b640 1 -- 172.21.15.37:0/1332354538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55604177fd90 msgr2=0x556041782180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/1332354538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55604177fd90 0x556041782180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33dd9b640 1 -- 172.21.15.37:0/1332354538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560417826c0 con 0x556041711210 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe33dd9b640 1 --2- 172.21.15.37:0/1332354538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 0x55604171f5b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe334009850 tx=0x7fe334004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a3750b0825d18b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.220+0000 7fe325ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.221+0000 7fe325ffb640 1 -- 172.21.15.37:0/1332354538 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334045070 con 0x556041711210 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.221+0000 7fe325ffb640 1 -- 172.21.15.37:0/1332354538 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe33402cc90 con 0x556041711210 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.221+0000 7fe34744e740 1 -- 172.21.15.37:0/1332354538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe334005640 con 0x556041711210 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.221+0000 7fe325ffb640 1 -- 172.21.15.37:0/1332354538 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe334050040 con 0x556041711210 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.221+0000 7fe34744e740 10 client.16677 fetch_fsmap learned FSMap version 56 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.221+0000 7fe34744e740 1 -- 172.21.15.37:0/1332354538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5560417839b0 con 0x556041711210 2024-03-26T06:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.221+0000 7fe325ffb640 1 -- 172.21.15.37:0/1332354538 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe334005640 con 0x556041711210 2024-03-26T06:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.222+0000 7fe325ffb640 1 -- 172.21.15.37:0/1332354538 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fe3340555f0 con 0x556041711210 2024-03-26T06:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.222+0000 7fe34744e740 10 client.16677 fetch_fsmap finished waiting for FSMap version 56 2024-03-26T06:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.222+0000 7fe34744e740 10 client.16677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.222+0000 7fe34744e740 20 client.16677 populate_metadata read hostname 'smithi037' 2024-03-26T06:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.222+0000 7fe34744e740 1 --2- 172.21.15.37:0/1332354538 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556041783b80 0x556041785f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.222+0000 7fe34744e740 4 client.16677 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.222+0000 7fe34744e740 1 -- 172.21.15.37:0/1332354538 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556041675e80 con 0x556041783b80 2024-03-26T06:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.222+0000 7fe33e59c640 1 --2- 172.21.15.37:0/1332354538 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556041783b80 0x556041785f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.224+0000 7fe33e59c640 1 --2- 172.21.15.37:0/1332354538 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556041783b80 0x556041785f70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac58c5d73f0214b server_cookie=3a989eab09501dd5 in_seq=0 out_seq=0 2024-03-26T06:37:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.224+0000 7fe325ffb640 10 client.16677 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.392+0000 7fe325ffb640 1 -- 172.21.15.37:0/1332354538 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556041675e80 con 0x556041783b80 2024-03-26T06:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.392+0000 7fe325ffb640 10 client.16677 handle_command_reply: tid=0 2024-03-26T06:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.392+0000 7fe34744e740 1 client.16677 shutdown 2024-03-26T06:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.392+0000 7fe34744e740 1 -- 172.21.15.37:0/1332354538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 msgr2=0x55604171f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.392+0000 7fe34744e740 1 --2- 172.21.15.37:0/1332354538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 0x55604171f5b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe334009850 tx=0x7fe334004270 comp rx=0 tx=0).stop 2024-03-26T06:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.392+0000 7fe34744e740 1 -- 172.21.15.37:0/1332354538 shutdown_connections 2024-03-26T06:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.392+0000 7fe34744e740 1 --2- 172.21.15.37:0/1332354538 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556041783b80 0x556041785f70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.393+0000 7fe34744e740 1 --2- 172.21.15.37:0/1332354538 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55604177fd90 0x556041782180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.393+0000 7fe34744e740 1 --2- 172.21.15.37:0/1332354538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556041711210 0x55604171f5b0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.393+0000 7fe34744e740 1 --2- 172.21.15.37:0/1332354538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556041709fd0 0x55604171f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.393+0000 7fe34744e740 1 -- 172.21.15.37:0/1332354538 >> 172.21.15.37:0/1332354538 conn(0x5560416d15b0 msgr2=0x5560417035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.393+0000 7fe325ffb640 0 client.16677 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.393+0000 7fe34744e740 1 -- 172.21.15.37:0/1332354538 shutdown_connections 2024-03-26T06:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.393+0000 7fe34744e740 1 -- 172.21.15.37:0/1332354538 wait complete. 2024-03-26T06:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.393+0000 7fe34744e740 20 client.16677 trim_cache size 0 max 16384 2024-03-26T06:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 -- 172.21.15.37:0/123980441 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3280a2c40 msgr2=0x7fe3280a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3280a2c40 0x7fe3280a5100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe3380099e0 tx=0x7fe338009500 comp rx=0 tx=0).stop 2024-03-26T06:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 -- 172.21.15.37:0/123980441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe340069d60 msgr2=0x7fe34019f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe340069d60 0x7fe34019f900 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe33000d950 tx=0x7fe33000de10 comp rx=0 tx=0).stop 2024-03-26T06:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 -- 172.21.15.37:0/123980441 shutdown_connections 2024-03-26T06:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe3280a2c40 0x7fe3280a5100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe340199b00 0x7fe340199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3401066b0 0x7fe34019fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 --2- 172.21.15.37:0/123980441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe340069d60 0x7fe34019f900 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 -- 172.21.15.37:0/123980441 >> 172.21.15.37:0/123980441 conn(0x7fe340076fc0 msgr2=0x7fe3400fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 -- 172.21.15.37:0/123980441 shutdown_connections 2024-03-26T06:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:17.394+0000 7fe34486c640 1 -- 172.21.15.37:0/123980441 wait complete. 2024-03-26T06:37:17.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 48322 53.55 MB/sec warmup 52 sec latency 76.064 ms 2024-03-26T06:37:18.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 49109 53.28 MB/sec warmup 53 sec latency 45.903 ms 2024-03-26T06:37:19.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:18 smithi066 ceph-mon[36504]: pgmap v451: 97 pgs: 97 active+clean; 656 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.25k op/s 2024-03-26T06:37:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:18 smithi149 ceph-mon[36681]: pgmap v451: 97 pgs: 97 active+clean; 656 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.25k op/s 2024-03-26T06:37:19.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:18 smithi037 ceph-mon[31974]: pgmap v451: 97 pgs: 97 active+clean; 656 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.25k op/s 2024-03-26T06:37:19.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 50211 53.21 MB/sec warmup 54 sec latency 43.804 ms 2024-03-26T06:37:20.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:19 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:37:20.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:19 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:37:20.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:19 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:37:20.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 51256 53.38 MB/sec warmup 55 sec latency 85.533 ms 2024-03-26T06:37:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:20 smithi066 ceph-mon[36504]: pgmap v452: 97 pgs: 97 active+clean; 656 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.25k op/s 2024-03-26T06:37:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:20 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:37:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:20 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:37:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:20 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:37:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:20 smithi149 ceph-mon[36681]: pgmap v452: 97 pgs: 97 active+clean; 656 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.25k op/s 2024-03-26T06:37:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:20 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:37:21.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:20 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:37:21.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:20 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:37:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:20 smithi037 ceph-mon[31974]: pgmap v452: 97 pgs: 97 active+clean; 656 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.25k op/s 2024-03-26T06:37:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:20 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:37:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:20 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:37:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:20 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:37:21.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 52186 53.43 MB/sec warmup 56 sec latency 53.483 ms 2024-03-26T06:37:22.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 53133 53.23 MB/sec warmup 57 sec latency 40.920 ms 2024-03-26T06:37:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:22 smithi066 ceph-mon[36504]: pgmap v453: 97 pgs: 97 active+clean; 677 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-03-26T06:37:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:22 smithi149 ceph-mon[36681]: pgmap v453: 97 pgs: 97 active+clean; 677 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-03-26T06:37:23.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:22 smithi037 ceph-mon[31974]: pgmap v453: 97 pgs: 97 active+clean; 677 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-03-26T06:37:23.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 54253 53.45 MB/sec warmup 58 sec latency 63.215 ms 2024-03-26T06:37:24.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 55193 53.31 MB/sec warmup 59 sec latency 116.636 ms 2024-03-26T06:37:25.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:24 smithi066 ceph-mon[36504]: pgmap v454: 97 pgs: 97 active+clean; 677 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-03-26T06:37:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:24 smithi149 ceph-mon[36681]: pgmap v454: 97 pgs: 97 active+clean; 677 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-03-26T06:37:25.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:24 smithi037 ceph-mon[31974]: pgmap v454: 97 pgs: 97 active+clean; 677 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-03-26T06:37:25.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 56109 53.39 MB/sec warmup 60 sec latency 47.802 ms 2024-03-26T06:37:26.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 57058 53.12 MB/sec warmup 61 sec latency 43.663 ms 2024-03-26T06:37:27.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:26 smithi066 ceph-mon[36504]: pgmap v455: 97 pgs: 97 active+clean; 710 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-03-26T06:37:27.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:26 smithi149 ceph-mon[36681]: pgmap v455: 97 pgs: 97 active+clean; 710 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-03-26T06:37:27.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:26 smithi037 ceph-mon[31974]: pgmap v455: 97 pgs: 97 active+clean; 710 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-03-26T06:37:27.846 INFO:tasks.workunit.client.0.smithi037.stdout: 10 58279 53.46 MB/sec warmup 62 sec latency 51.747 ms 2024-03-26T06:37:28.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 59257 53.57 MB/sec warmup 63 sec latency 50.711 ms 2024-03-26T06:37:29.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:28 smithi066 ceph-mon[36504]: pgmap v456: 97 pgs: 97 active+clean; 711 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.8 MiB/s wr, 825 op/s 2024-03-26T06:37:29.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:28 smithi149 ceph-mon[36681]: pgmap v456: 97 pgs: 97 active+clean; 711 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.8 MiB/s wr, 825 op/s 2024-03-26T06:37:29.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:28 smithi037 ceph-mon[31974]: pgmap v456: 97 pgs: 97 active+clean; 711 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.8 MiB/s wr, 825 op/s 2024-03-26T06:37:29.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 60123 53.38 MB/sec warmup 64 sec latency 49.083 ms 2024-03-26T06:37:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.538+0000 7f8344812640 1 -- 172.21.15.37:0/3433978065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83401055b0 msgr2=0x7f8340105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.538+0000 7f8344812640 1 --2- 172.21.15.37:0/3433978065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83401055b0 0x7f8340105990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8330009cd0 tx=0x7f833002d1f0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.538+0000 7f8344812640 1 -- 172.21.15.37:0/3433978065 shutdown_connections 2024-03-26T06:37:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.538+0000 7f8344812640 1 --2- 172.21.15.37:0/3433978065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83400ffeb0 0x7f834010c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.538+0000 7f8344812640 1 --2- 172.21.15.37:0/3433978065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83400ff4f0 0x7f83400ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.538+0000 7f8344812640 1 --2- 172.21.15.37:0/3433978065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83401055b0 0x7f8340105990 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8330009cd0 tx=0x7f833002d1f0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.538+0000 7f8344812640 1 -- 172.21.15.37:0/3433978065 >> 172.21.15.37:0/3433978065 conn(0x7f8340076380 msgr2=0x7f83400fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.539+0000 7f8344812640 1 -- 172.21.15.37:0/3433978065 shutdown_connections 2024-03-26T06:37:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.539+0000 7f8344812640 1 -- 172.21.15.37:0/3433978065 wait complete. 2024-03-26T06:37:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.540+0000 7f8344812640 1 Processor -- start 2024-03-26T06:37:30.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.540+0000 7f8344812640 1 -- start start 2024-03-26T06:37:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.540+0000 7f8344812640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83400ff4f0 0x7f834019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.540+0000 7f8344812640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83400ffeb0 0x7f834019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.540+0000 7f8344812640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8340197af0 0x7f8340197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.541+0000 7f8344812640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834010f4a0 con 0x7f83400ffeb0 2024-03-26T06:37:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.541+0000 7f8344812640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834010f320 con 0x7f83400ff4f0 2024-03-26T06:37:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.541+0000 7f8344812640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834010f620 con 0x7f8340197af0 2024-03-26T06:37:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.541+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83400ffeb0 0x7f834019de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.541+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83400ffeb0 0x7f834019de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48812/0 (socket says 172.21.15.37:48812) 2024-03-26T06:37:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.541+0000 7f833ed9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8340197af0 0x7f8340197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.541+0000 7f833dd9b640 1 -- 172.21.15.37:0/413563410 learned_addr learned my addr 172.21.15.37:0/413563410 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.541+0000 7f833e59c640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83400ff4f0 0x7f834019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.541+0000 7f833ed9d640 1 -- 172.21.15.37:0/413563410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83400ff4f0 msgr2=0x7f834019d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.541+0000 7f833ed9d640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83400ff4f0 0x7f834019d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.542+0000 7f833ed9d640 1 -- 172.21.15.37:0/413563410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83400ffeb0 msgr2=0x7f834019de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.542+0000 7f833ed9d640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83400ffeb0 0x7f834019de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.542+0000 7f833ed9d640 1 -- 172.21.15.37:0/413563410 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8340198610 con 0x7f8340197af0 2024-03-26T06:37:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.542+0000 7f833dd9b640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83400ffeb0 0x7f834019de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:37:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.542+0000 7f833ed9d640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8340197af0 0x7f8340197fa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8328009e40 tx=0x7f832800ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.542+0000 7f83377fe640 1 -- 172.21.15.37:0/413563410 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832800ccc0 con 0x7f8340197af0 2024-03-26T06:37:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.543+0000 7f83377fe640 1 -- 172.21.15.37:0/413563410 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8328005020 con 0x7f8340197af0 2024-03-26T06:37:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.543+0000 7f83377fe640 1 -- 172.21.15.37:0/413563410 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328013a30 con 0x7f8340197af0 2024-03-26T06:37:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.543+0000 7f8344812640 1 -- 172.21.15.37:0/413563410 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8340198900 con 0x7f8340197af0 2024-03-26T06:37:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.543+0000 7f8344812640 1 -- 172.21.15.37:0/413563410 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8340195af0 con 0x7f8340197af0 2024-03-26T06:37:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.546+0000 7f83377fe640 1 -- 172.21.15.37:0/413563410 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f8328022020 con 0x7f8340197af0 2024-03-26T06:37:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.547+0000 7f83473f8740 1 Processor -- start 2024-03-26T06:37:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f83473f8740 1 -- start start 2024-03-26T06:37:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f83473f8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959fad40 0x55f5959f9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f83473f8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959f9fd0 0x55f5959fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f83473f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f595a05940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f83473f8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59595e5f0 con 0x55f5959fad40 2024-03-26T06:37:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f83473f8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f595907930 con 0x55f5959f9fd0 2024-03-26T06:37:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f83473f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59595a690 con 0x55f595a03550 2024-03-26T06:37:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959fad40 0x55f5959f9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f83377fe640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f832c0a2bf0 0x7f832c0a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f833e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959f9fd0 0x55f5959fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f595a05940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f833e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959f9fd0 0x55f5959fa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:37754/0 (socket says 172.21.15.37:37754) 2024-03-26T06:37:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.548+0000 7f833e59c640 1 -- 172.21.15.37:0/1300585181 learned_addr learned my addr 172.21.15.37:0/1300585181 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.549+0000 7f83377fe640 1 -- 172.21.15.37:0/413563410 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f83280d2e20 con 0x7f8340197af0 2024-03-26T06:37:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.549+0000 7f833ed9d640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f832c0a2bf0 0x7f832c0a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.549+0000 7f833e59c640 1 -- 172.21.15.37:0/1300585181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 msgr2=0x55f595a05940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.549+0000 7f833e59c640 1 --2- 172.21.15.37:0/1300585181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f595a05940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.549+0000 7f833e59c640 1 -- 172.21.15.37:0/1300585181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959fad40 msgr2=0x55f5959f9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.549+0000 7f833e59c640 1 --2- 172.21.15.37:0/1300585181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959fad40 0x55f5959f9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.549+0000 7f833e59c640 1 -- 172.21.15.37:0/1300585181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f595673210 con 0x55f5959f9fd0 2024-03-26T06:37:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.550+0000 7f833ed9d640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f832c0a2bf0 0x7f832c0a50b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f83280a77a0 tx=0x7f8328015040 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.550+0000 7f833e59c640 1 --2- 172.21.15.37:0/1300585181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959f9fd0 0x55f5959fa450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8330009bf0 tx=0x7f8330005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc4a6d8483a7939 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.551+0000 7f8334ff9640 1 -- 172.21.15.37:0/1300585181 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833003a050 con 0x55f5959f9fd0 2024-03-26T06:37:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.551+0000 7f8334ff9640 1 -- 172.21.15.37:0/1300585181 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f833003b040 con 0x55f5959f9fd0 2024-03-26T06:37:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.551+0000 7f8334ff9640 1 -- 172.21.15.37:0/1300585181 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330003d90 con 0x55f5959f9fd0 2024-03-26T06:37:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.552+0000 7f83473f8740 1 -- 172.21.15.37:0/1300585181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959f9fd0 msgr2=0x55f5959fa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.552+0000 7f83473f8740 1 --2- 172.21.15.37:0/1300585181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959f9fd0 0x55f5959fa450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8330009bf0 tx=0x7f8330005f50 comp rx=0 tx=0).stop 2024-03-26T06:37:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.552+0000 7f83473f8740 1 -- 172.21.15.37:0/1300585181 shutdown_connections 2024-03-26T06:37:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.552+0000 7f83473f8740 1 --2- 172.21.15.37:0/1300585181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f595a05940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.552+0000 7f83473f8740 1 --2- 172.21.15.37:0/1300585181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959f9fd0 0x55f5959fa450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.552+0000 7f83473f8740 1 --2- 172.21.15.37:0/1300585181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959fad40 0x55f5959f9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.552+0000 7f83473f8740 1 -- 172.21.15.37:0/1300585181 >> 172.21.15.37:0/1300585181 conn(0x55f5959c15b0 msgr2=0x55f5959f4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.552+0000 7f83473f8740 1 -- 172.21.15.37:0/1300585181 shutdown_connections 2024-03-26T06:37:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.553+0000 7f83473f8740 1 -- 172.21.15.37:0/1300585181 wait complete. 2024-03-26T06:37:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.553+0000 7f83473f8740 1 -- start start 2024-03-26T06:37:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.553+0000 7f83473f8740 1 Processor -- start 2024-03-26T06:37:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.554+0000 7f83473f8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a00680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.554+0000 7f83473f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5959fad40 0x55f595a00bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.554+0000 7f83473f8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f595a03550 0x55f595a01100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.554+0000 7f83473f8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f595a06220 con 0x55f5959f9fd0 2024-03-26T06:37:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.555+0000 7f833e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5959fad40 0x55f595a00bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.555+0000 7f833e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5959fad40 0x55f595a00bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:60516/0 (socket says 172.21.15.37:60516) 2024-03-26T06:37:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.555+0000 7f833e59c640 1 -- 172.21.15.37:0/949946590 learned_addr learned my addr 172.21.15.37:0/949946590 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.555+0000 7f83473f8740 1 -- 172.21.15.37:0/949946590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f59591ad90 con 0x55f595a03550 2024-03-26T06:37:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.555+0000 7f83473f8740 1 -- 172.21.15.37:0/949946590 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f595a063a0 con 0x55f5959fad40 2024-03-26T06:37:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.555+0000 7f83473f8740 10 client.0 _ll_register_callbacks cb 0x55f595672a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:37:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.556+0000 7f833dd9b640 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f595a03550 0x55f595a01100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.556+0000 7f833dd9b640 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.556+0000 7f833e59c640 1 -- 172.21.15.37:0/949946590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f595a03550 msgr2=0x55f595a01100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.556+0000 7f833e59c640 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f595a03550 0x55f595a01100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.556+0000 7f833e59c640 1 -- 172.21.15.37:0/949946590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 msgr2=0x55f595a00680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.556+0000 7f833e59c640 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a00680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.556+0000 7f833e59c640 1 -- 172.21.15.37:0/949946590 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f59573f620 con 0x55f5959fad40 2024-03-26T06:37:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.556+0000 7f833dd9b640 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a00680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:37:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.557+0000 7f833e59c640 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5959fad40 0x55f595a00bc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8330004af0 tx=0x7f8330005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e70359e97dfebc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.557+0000 7f8325ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:37:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.557+0000 7f8325ffb640 1 -- 172.21.15.37:0/949946590 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833004f020 con 0x55f5959fad40 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.557+0000 7f83473f8740 1 -- 172.21.15.37:0/949946590 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f833003b040 con 0x55f5959fad40 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.557+0000 7f8325ffb640 1 -- 172.21.15.37:0/949946590 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8330009800 con 0x55f5959fad40 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.558+0000 7f8325ffb640 1 -- 172.21.15.37:0/949946590 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f833005a050 con 0x55f5959fad40 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.558+0000 7f83473f8740 10 client.26074 fetch_fsmap learned FSMap version 56 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.558+0000 7f83473f8740 1 -- 172.21.15.37:0/949946590 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55f595a06220 con 0x55f5959fad40 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.558+0000 7f8325ffb640 1 -- 172.21.15.37:0/949946590 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f833003b040 con 0x55f5959fad40 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.558+0000 7f8325ffb640 1 -- 172.21.15.37:0/949946590 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f8330035540 con 0x55f5959fad40 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.558+0000 7f83473f8740 10 client.26074 fetch_fsmap finished waiting for FSMap version 56 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.558+0000 7f83473f8740 10 client.26074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.558+0000 7f83473f8740 20 client.26074 populate_metadata read hostname 'smithi037' 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.558+0000 7f83473f8740 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f595a13790 0x55f595a15b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.559+0000 7f83473f8740 4 client.26074 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:37:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.559+0000 7f83473f8740 1 -- 172.21.15.37:0/949946590 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f59596bb30 con 0x55f595a13790 2024-03-26T06:37:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.560+0000 7f833dd9b640 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f595a13790 0x55f595a15b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.562+0000 7f833dd9b640 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f595a13790 0x55f595a15b80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26697773129d0af1 server_cookie=cf349c19d119d28c in_seq=0 out_seq=0 2024-03-26T06:37:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.562+0000 7f8325ffb640 10 client.26074 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.563+0000 7f8325ffb640 1 -- 172.21.15.37:0/949946590 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f59596bb30 con 0x55f595a13790 2024-03-26T06:37:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.563+0000 7f8325ffb640 10 client.26074 handle_command_reply: tid=0 2024-03-26T06:37:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.563+0000 7f83473f8740 1 client.26074 shutdown 2024-03-26T06:37:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f83473f8740 1 -- 172.21.15.37:0/949946590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5959fad40 msgr2=0x55f595a00bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f83473f8740 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5959fad40 0x55f595a00bc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8330004af0 tx=0x7f8330005fd0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f83473f8740 1 -- 172.21.15.37:0/949946590 shutdown_connections 2024-03-26T06:37:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f83473f8740 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f595a13790 0x55f595a15b80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f83473f8740 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f595a03550 0x55f595a01100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f83473f8740 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5959fad40 0x55f595a00bc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f83473f8740 1 --2- 172.21.15.37:0/949946590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a00680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f83473f8740 1 -- 172.21.15.37:0/949946590 >> 172.21.15.37:0/949946590 conn(0x55f5959c15b0 msgr2=0x55f5959f6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f8325ffb640 0 client.26074 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f83473f8740 1 -- 172.21.15.37:0/949946590 shutdown_connections 2024-03-26T06:37:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f83473f8740 1 -- 172.21.15.37:0/949946590 wait complete. 2024-03-26T06:37:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.564+0000 7f83473f8740 20 client.26074 trim_cache size 0 max 16384 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.587+0000 7f83473f8740 1 Processor -- start 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.587+0000 7f83473f8740 1 -- start start 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.587+0000 7f83473f8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a02db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.587+0000 7f83473f8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959fad40 0x55f595a0a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f83473f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f595a0ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f83473f8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83401aaf40 con 0x55f5959f9fd0 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f83473f8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83401aca70 con 0x55f5959fad40 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f83473f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83401ab660 con 0x55f595a03550 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a02db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a02db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48836/0 (socket says 172.21.15.37:48836) 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f833dd9b640 1 -- 172.21.15.37:0/3112601050 learned_addr learned my addr 172.21.15.37:0/3112601050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f833dd9b640 1 --2- 172.21.15.37:0/3112601050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f595a0ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f833dd9b640 1 -- 172.21.15.37:0/3112601050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 msgr2=0x55f595a0ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f833dd9b640 1 --2- 172.21.15.37:0/3112601050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f595a0ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f833dd9b640 1 -- 172.21.15.37:0/3112601050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959fad40 msgr2=0x55f595a0a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f833dd9b640 1 --2- 172.21.15.37:0/3112601050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959fad40 0x55f595a0a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f833dd9b640 1 -- 172.21.15.37:0/3112601050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5959f92a0 con 0x55f5959f9fd0 2024-03-26T06:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.588+0000 7f833dd9b640 1 --2- 172.21.15.37:0/3112601050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a02db0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8338012010 tx=0x7f8338009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343ef484515334b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.589+0000 7f8327fff640 1 -- 172.21.15.37:0/3112601050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338019040 con 0x55f5959f9fd0 2024-03-26T06:37:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.589+0000 7f8327fff640 1 -- 172.21.15.37:0/3112601050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83380054d0 con 0x55f5959f9fd0 2024-03-26T06:37:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.589+0000 7f8327fff640 1 -- 172.21.15.37:0/3112601050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338019040 con 0x55f5959f9fd0 2024-03-26T06:37:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.589+0000 7f83473f8740 1 -- 172.21.15.37:0/3112601050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 msgr2=0x55f595a02db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.589+0000 7f83473f8740 1 --2- 172.21.15.37:0/3112601050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a02db0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8338012010 tx=0x7f8338009530 comp rx=0 tx=0).stop 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.590+0000 7f83473f8740 1 -- 172.21.15.37:0/3112601050 shutdown_connections 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.590+0000 7f83473f8740 1 --2- 172.21.15.37:0/3112601050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f595a0ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.590+0000 7f83473f8740 1 --2- 172.21.15.37:0/3112601050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959fad40 0x55f595a0a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.590+0000 7f83473f8740 1 --2- 172.21.15.37:0/3112601050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a02db0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.590+0000 7f83473f8740 1 -- 172.21.15.37:0/3112601050 >> 172.21.15.37:0/3112601050 conn(0x55f5959c15b0 msgr2=0x55f5959f47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.590+0000 7f83473f8740 1 -- 172.21.15.37:0/3112601050 shutdown_connections 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.590+0000 7f83473f8740 1 -- 172.21.15.37:0/3112601050 wait complete. 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.590+0000 7f83473f8740 1 -- start start 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.590+0000 7f83473f8740 1 Processor -- start 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f83473f8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a0a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f83473f8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959fad40 0x55f5959fc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f83473f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f5959fce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f83473f8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83401ac410 con 0x55f5959f9fd0 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f83473f8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83401abcf0 con 0x55f5959fad40 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f83473f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83401a9200 con 0x55f595a03550 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a0a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f833e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959fad40 0x55f5959fc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f833e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959fad40 0x55f5959fc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:37774/0 (socket says 172.21.15.37:37774) 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f833dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a0a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48842/0 (socket says 172.21.15.37:48842) 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f833e59c640 1 -- 172.21.15.37:0/2539581653 learned_addr learned my addr 172.21.15.37:0/2539581653 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f833dd9b640 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f5959fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.591+0000 7f83473f8740 10 client.0 _ll_register_callbacks cb 0x7f8330038070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.592+0000 7f833dd9b640 1 -- 172.21.15.37:0/2539581653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 msgr2=0x55f5959fce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.592+0000 7f833dd9b640 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f5959fce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.592+0000 7f833dd9b640 1 -- 172.21.15.37:0/2539581653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959fad40 msgr2=0x55f5959fc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.592+0000 7f833dd9b640 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959fad40 0x55f5959fc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.592+0000 7f833dd9b640 1 -- 172.21.15.37:0/2539581653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f595a0ab60 con 0x55f5959f9fd0 2024-03-26T06:37:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.592+0000 7f833e59c640 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959fad40 0x55f5959fc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:37:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.592+0000 7f833dd9b640 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a0a620 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8338006010 tx=0x7f8338009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4688bbce5ba2027e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.592+0000 7f8325ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:37:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.592+0000 7f8325ffb640 1 -- 172.21.15.37:0/2539581653 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338019040 con 0x55f5959f9fd0 2024-03-26T06:37:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.592+0000 7f8325ffb640 1 -- 172.21.15.37:0/2539581653 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f833802b030 con 0x55f5959f9fd0 2024-03-26T06:37:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f83473f8740 1 -- 172.21.15.37:0/2539581653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83380054d0 con 0x55f5959f9fd0 2024-03-26T06:37:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f8325ffb640 1 -- 172.21.15.37:0/2539581653 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f833801f680 con 0x55f5959f9fd0 2024-03-26T06:37:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f83473f8740 10 client.16692 fetch_fsmap learned FSMap version 56 2024-03-26T06:37:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f83473f8740 1 -- 172.21.15.37:0/2539581653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55f595a6a310 con 0x55f5959f9fd0 2024-03-26T06:37:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f8325ffb640 1 -- 172.21.15.37:0/2539581653 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83380054d0 con 0x55f5959f9fd0 2024-03-26T06:37:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f8325ffb640 1 -- 172.21.15.37:0/2539581653 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f8338004290 con 0x55f5959f9fd0 2024-03-26T06:37:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f83473f8740 10 client.16692 fetch_fsmap finished waiting for FSMap version 56 2024-03-26T06:37:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f83473f8740 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:37:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f83473f8740 20 client.16692 populate_metadata read hostname 'smithi037' 2024-03-26T06:37:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f83473f8740 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f595a6a9e0 0x55f595a6ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f83473f8740 4 client.16692 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:37:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.593+0000 7f83473f8740 1 -- 172.21.15.37:0/2539581653 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f595965e80 con 0x55f595a6a9e0 2024-03-26T06:37:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.594+0000 7f833e59c640 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f595a6a9e0 0x55f595a6ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.594+0000 7f833e59c640 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f595a6a9e0 0x55f595a6ce00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b627e92b146934 server_cookie=1336f4642cf22644 in_seq=0 out_seq=0 2024-03-26T06:37:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.594+0000 7f8325ffb640 10 client.16692 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.800+0000 7f8325ffb640 1 -- 172.21.15.37:0/2539581653 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f595965e80 con 0x55f595a6a9e0 2024-03-26T06:37:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.800+0000 7f8325ffb640 10 client.16692 handle_command_reply: tid=0 2024-03-26T06:37:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.800+0000 7f83473f8740 1 client.16692 shutdown 2024-03-26T06:37:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f83473f8740 1 -- 172.21.15.37:0/2539581653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 msgr2=0x55f595a0a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f83473f8740 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a0a620 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8338006010 tx=0x7f8338009fd0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f83473f8740 1 -- 172.21.15.37:0/2539581653 shutdown_connections 2024-03-26T06:37:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f83473f8740 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55f595a6a9e0 0x55f595a6ce00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f83473f8740 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f595a03550 0x55f5959fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f83473f8740 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f5959fad40 0x55f5959fc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f83473f8740 1 --2- 172.21.15.37:0/2539581653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5959f9fd0 0x55f595a0a620 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f83473f8740 1 -- 172.21.15.37:0/2539581653 >> 172.21.15.37:0/2539581653 conn(0x55f5959c15b0 msgr2=0x55f5959fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f8325ffb640 0 client.16692 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f83473f8740 1 -- 172.21.15.37:0/2539581653 shutdown_connections 2024-03-26T06:37:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f83473f8740 1 -- 172.21.15.37:0/2539581653 wait complete. 2024-03-26T06:37:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.801+0000 7f83473f8740 20 client.16692 trim_cache size 0 max 16384 2024-03-26T06:37:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:37:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:37:30.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:37:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:37:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 -- 172.21.15.37:0/413563410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f832c0a2bf0 msgr2=0x7f832c0a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f832c0a2bf0 0x7f832c0a50b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f83280a77a0 tx=0x7f8328015040 comp rx=0 tx=0).stop 2024-03-26T06:37:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 -- 172.21.15.37:0/413563410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8340197af0 msgr2=0x7f8340197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8340197af0 0x7f8340197fa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8328009e40 tx=0x7f832800ed40 comp rx=0 tx=0).stop 2024-03-26T06:37:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 -- 172.21.15.37:0/413563410 shutdown_connections 2024-03-26T06:37:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f832c0a2bf0 0x7f832c0a50b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8340197af0 0x7f8340197fa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83400ffeb0 0x7f834019de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 --2- 172.21.15.37:0/413563410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83400ff4f0 0x7f834019d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 -- 172.21.15.37:0/413563410 >> 172.21.15.37:0/413563410 conn(0x7f8340076380 msgr2=0x7f83400fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 -- 172.21.15.37:0/413563410 shutdown_connections 2024-03-26T06:37:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:30.802+0000 7f8344812640 1 -- 172.21.15.37:0/413563410 wait complete. 2024-03-26T06:37:30.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 61101 53.32 MB/sec warmup 65 sec latency 192.407 ms 2024-03-26T06:37:31.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:30 smithi066 ceph-mon[36504]: pgmap v457: 97 pgs: 97 active+clean; 711 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.7 MiB/s wr, 719 op/s 2024-03-26T06:37:31.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:30 smithi149 ceph-mon[36681]: pgmap v457: 97 pgs: 97 active+clean; 711 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.7 MiB/s wr, 719 op/s 2024-03-26T06:37:31.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:30 smithi037 ceph-mon[31974]: pgmap v457: 97 pgs: 97 active+clean; 711 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.7 MiB/s wr, 719 op/s 2024-03-26T06:37:31.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 62068 53.40 MB/sec warmup 66 sec latency 41.459 ms 2024-03-26T06:37:32.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:31 smithi149 ceph-mon[36681]: pgmap v458: 97 pgs: 97 active+clean; 745 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-03-26T06:37:32.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:31 smithi037 ceph-mon[31974]: pgmap v458: 97 pgs: 97 active+clean; 745 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-03-26T06:37:32.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:31 smithi066 ceph-mon[36504]: pgmap v458: 97 pgs: 97 active+clean; 745 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-03-26T06:37:32.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 63078 53.31 MB/sec warmup 67 sec latency 73.523 ms 2024-03-26T06:37:33.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 64167 53.44 MB/sec warmup 68 sec latency 57.404 ms 2024-03-26T06:37:34.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:34 smithi066 ceph-mon[36504]: pgmap v459: 97 pgs: 97 active+clean; 745 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.5 MiB/s wr, 660 op/s 2024-03-26T06:37:34.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:34 smithi149 ceph-mon[36681]: pgmap v459: 97 pgs: 97 active+clean; 745 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.5 MiB/s wr, 660 op/s 2024-03-26T06:37:34.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:34 smithi037 ceph-mon[31974]: pgmap v459: 97 pgs: 97 active+clean; 745 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.5 MiB/s wr, 660 op/s 2024-03-26T06:37:34.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 64913 53.29 MB/sec warmup 69 sec latency 165.481 ms 2024-03-26T06:37:35.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 64933 52.53 MB/sec warmup 70 sec latency 954.425 ms 2024-03-26T06:37:36.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:36 smithi066 ceph-mon[36504]: pgmap v460: 97 pgs: 97 active+clean; 766 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.6 MiB/s wr, 770 op/s 2024-03-26T06:37:36.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:36 smithi149 ceph-mon[36681]: pgmap v460: 97 pgs: 97 active+clean; 766 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.6 MiB/s wr, 770 op/s 2024-03-26T06:37:36.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:36 smithi037 ceph-mon[31974]: pgmap v460: 97 pgs: 97 active+clean; 766 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.6 MiB/s wr, 770 op/s 2024-03-26T06:37:36.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 65445 52.20 MB/sec warmup 71 sec latency 979.268 ms 2024-03-26T06:37:37.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 66204 51.92 MB/sec warmup 72 sec latency 79.360 ms 2024-03-26T06:37:38.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:38 smithi066 ceph-mon[36504]: pgmap v461: 97 pgs: 97 active+clean; 821 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 835 op/s 2024-03-26T06:37:38.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:38 smithi149 ceph-mon[36681]: pgmap v461: 97 pgs: 97 active+clean; 821 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 835 op/s 2024-03-26T06:37:38.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:38 smithi037 ceph-mon[31974]: pgmap v461: 97 pgs: 97 active+clean; 821 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 835 op/s 2024-03-26T06:37:38.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 67335 52.09 MB/sec warmup 73 sec latency 44.428 ms 2024-03-26T06:37:39.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 68204 52.11 MB/sec warmup 74 sec latency 88.380 ms 2024-03-26T06:37:40.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:40 smithi066 ceph-mon[36504]: pgmap v462: 97 pgs: 97 active+clean; 821 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 822 op/s 2024-03-26T06:37:40.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:40 smithi149 ceph-mon[36681]: pgmap v462: 97 pgs: 97 active+clean; 821 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 822 op/s 2024-03-26T06:37:40.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:40 smithi037 ceph-mon[31974]: pgmap v462: 97 pgs: 97 active+clean; 821 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 822 op/s 2024-03-26T06:37:40.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 68974 51.90 MB/sec warmup 75 sec latency 227.790 ms 2024-03-26T06:37:41.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 69996 51.86 MB/sec warmup 76 sec latency 68.583 ms 2024-03-26T06:37:42.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:42 smithi066 ceph-mon[36504]: pgmap v463: 97 pgs: 97 active+clean; 770 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 25 MiB/s wr, 1.47k op/s 2024-03-26T06:37:42.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:42 smithi149 ceph-mon[36681]: pgmap v463: 97 pgs: 97 active+clean; 770 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 25 MiB/s wr, 1.47k op/s 2024-03-26T06:37:42.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:42 smithi037 ceph-mon[31974]: pgmap v463: 97 pgs: 97 active+clean; 770 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 25 MiB/s wr, 1.47k op/s 2024-03-26T06:37:42.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 71173 52.08 MB/sec warmup 77 sec latency 163.899 ms 2024-03-26T06:37:43.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 72091 52.10 MB/sec warmup 78 sec latency 32.272 ms 2024-03-26T06:37:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.932+0000 7f1e084d1640 1 -- 172.21.15.37:0/4209426004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00100e00 msgr2=0x7f1e0010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.932+0000 7f1e084d1640 1 --2- 172.21.15.37:0/4209426004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00100e00 0x7f1e0010d020 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009ca0 tx=0x7f1df802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.932+0000 7f1e084d1640 1 -- 172.21.15.37:0/4209426004 shutdown_connections 2024-03-26T06:37:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.932+0000 7f1e084d1640 1 --2- 172.21.15.37:0/4209426004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00100e00 0x7f1e0010d020 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.932+0000 7f1e084d1640 1 --2- 172.21.15.37:0/4209426004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00100440 0x7f1e001008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.932+0000 7f1e084d1640 1 --2- 172.21.15.37:0/4209426004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e00105190 0x7f1e00105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.932+0000 7f1e084d1640 1 -- 172.21.15.37:0/4209426004 >> 172.21.15.37:0/4209426004 conn(0x7f1e00077040 msgr2=0x7f1e00077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.932+0000 7f1e084d1640 1 -- 172.21.15.37:0/4209426004 shutdown_connections 2024-03-26T06:37:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.933+0000 7f1e084d1640 1 -- 172.21.15.37:0/4209426004 wait complete. 2024-03-26T06:37:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.933+0000 7f1e084d1640 1 Processor -- start 2024-03-26T06:37:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.933+0000 7f1e084d1640 1 -- start start 2024-03-26T06:37:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e084d1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e00100440 0x7f1e0019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e084d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00100e00 0x7f1e0019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e084d1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00105190 0x7f1e0019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e084d1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0010f540 con 0x7f1e00100e00 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e084d1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0010f3c0 con 0x7f1e00105190 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e084d1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0010f6c0 con 0x7f1e00100440 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e0626d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e00100440 0x7f1e0019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e0626d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e00100440 0x7f1e0019a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:54024/0 (socket says 172.21.15.37:54024) 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e05a6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00100e00 0x7f1e0019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e0626d640 1 -- 172.21.15.37:0/1448459244 learned_addr learned my addr 172.21.15.37:0/1448459244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e06a6e640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00105190 0x7f1e0019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e05a6c640 1 -- 172.21.15.37:0/1448459244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e00100440 msgr2=0x7f1e0019a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e05a6c640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e00100440 0x7f1e0019a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e05a6c640 1 -- 172.21.15.37:0/1448459244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00105190 msgr2=0x7f1e0019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e05a6c640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00105190 0x7f1e0019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.934+0000 7f1e05a6c640 1 -- 172.21.15.37:0/1448459244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0019ed80 con 0x7f1e00100e00 2024-03-26T06:37:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.935+0000 7f1e05a6c640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00100e00 0x7f1e0019a760 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00e9f0 tx=0x7f1dfc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.935+0000 7f1df77fe640 1 -- 172.21.15.37:0/1448459244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc00ccc0 con 0x7f1e00100e00 2024-03-26T06:37:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.935+0000 7f1df77fe640 1 -- 172.21.15.37:0/1448459244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dfc010910 con 0x7f1e00100e00 2024-03-26T06:37:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.935+0000 7f1e084d1640 1 -- 172.21.15.37:0/1448459244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0019f070 con 0x7f1e00100e00 2024-03-26T06:37:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.935+0000 7f1e084d1640 1 -- 172.21.15.37:0/1448459244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e00110f20 con 0x7f1e00100e00 2024-03-26T06:37:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.936+0000 7f1df77fe640 1 -- 172.21.15.37:0/1448459244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc018960 con 0x7f1e00100e00 2024-03-26T06:37:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.939+0000 7f1e0b194740 1 Processor -- start 2024-03-26T06:37:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.939+0000 7f1e0b194740 1 -- start start 2024-03-26T06:37:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.939+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333bcd40 0x55bf333bba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.939+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333bc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.939+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333c5550 0x55bf333c7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.939+0000 7f1e0b194740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf333205f0 con 0x55bf333bcd40 2024-03-26T06:37:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.939+0000 7f1e0b194740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf332c9930 con 0x55bf333bbfd0 2024-03-26T06:37:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.939+0000 7f1e0b194740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf3331c690 con 0x55bf333c5550 2024-03-26T06:37:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.940+0000 7f1e0626d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.940+0000 7f1e0626d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333bc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:51246/0 (socket says 172.21.15.37:51246) 2024-03-26T06:37:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.940+0000 7f1e0626d640 1 -- 172.21.15.37:0/1789245088 learned_addr learned my addr 172.21.15.37:0/1789245088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.940+0000 7f1e06a6e640 1 --2- 172.21.15.37:0/1789245088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333bcd40 0x55bf333bba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.940+0000 7f1e0626d640 1 -- 172.21.15.37:0/1789245088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333c5550 msgr2=0x55bf333c7940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T06:37:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.940+0000 7f1e0626d640 1 --2- 172.21.15.37:0/1789245088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333c5550 0x55bf333c7940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.940+0000 7f1e0626d640 1 -- 172.21.15.37:0/1789245088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333bcd40 msgr2=0x55bf333bba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.940+0000 7f1e0626d640 1 --2- 172.21.15.37:0/1789245088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333bcd40 0x55bf333bba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.940+0000 7f1e0626d640 1 -- 172.21.15.37:0/1789245088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf33035210 con 0x55bf333bbfd0 2024-03-26T06:37:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.940+0000 7f1e0626d640 1 --2- 172.21.15.37:0/1789245088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333bc450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009850 tx=0x7f1df0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71232f513563dcb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.941+0000 7f1df4bfe640 1 -- 172.21.15.37:0/1789245088 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0022020 con 0x55bf333bbfd0 2024-03-26T06:37:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.941+0000 7f1df4bfe640 1 -- 172.21.15.37:0/1789245088 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df001e400 con 0x55bf333bbfd0 2024-03-26T06:37:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.941+0000 7f1df77fe640 1 -- 172.21.15.37:0/1448459244 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1dfc010ac0 con 0x7f1e00100e00 2024-03-26T06:37:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.941+0000 7f1df4bfe640 1 -- 172.21.15.37:0/1789245088 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df001d5a0 con 0x55bf333bbfd0 2024-03-26T06:37:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1df77fe640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1dec0a2c10 0x7f1dec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1e0b194740 1 -- 172.21.15.37:0/1789245088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 msgr2=0x55bf333bc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1e0b194740 1 --2- 172.21.15.37:0/1789245088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333bc450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009850 tx=0x7f1df0005f90 comp rx=0 tx=0).stop 2024-03-26T06:37:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1df77fe640 1 -- 172.21.15.37:0/1448459244 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1dfc018700 con 0x7f1e00100e00 2024-03-26T06:37:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1e0b194740 1 -- 172.21.15.37:0/1789245088 shutdown_connections 2024-03-26T06:37:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1e0b194740 1 --2- 172.21.15.37:0/1789245088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333c5550 0x55bf333c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1e0b194740 1 --2- 172.21.15.37:0/1789245088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333bc450 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1e0b194740 1 --2- 172.21.15.37:0/1789245088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333bcd40 0x55bf333bba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1e0b194740 1 -- 172.21.15.37:0/1789245088 >> 172.21.15.37:0/1789245088 conn(0x55bf333835b0 msgr2=0x55bf333b6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1e06a6e640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1dec0a2c10 0x7f1dec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1e0b194740 1 -- 172.21.15.37:0/1789245088 shutdown_connections 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.942+0000 7f1e0b194740 1 -- 172.21.15.37:0/1789245088 wait complete. 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e0b194740 1 -- start start 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e06a6e640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1dec0a2c10 0x7f1dec0a50d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1df8005e30 tx=0x7f1df8005d80 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e0b194740 1 Processor -- start 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333bbfd0 0x55bf333c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333c5550 0x55bf333c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e0b194740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf333c8320 con 0x55bf333bbfd0 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e0b194740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf332dcd90 con 0x55bf333c5550 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e0b194740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf333c84a0 con 0x55bf333bcd40 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e0b194740 10 client.0 _ll_register_callbacks cb 0x55bf33034a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e06a6e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333c5550 0x55bf333c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e05a6c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e05a6c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333c2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:54034/0 (socket says 172.21.15.37:54034) 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e0626d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333bbfd0 0x55bf333c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.943+0000 7f1e05a6c640 1 -- 172.21.15.37:0/3635496405 learned_addr learned my addr 172.21.15.37:0/3635496405 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.944+0000 7f1e05a6c640 1 -- 172.21.15.37:0/3635496405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333c5550 msgr2=0x55bf333c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.944+0000 7f1e05a6c640 1 --2- 172.21.15.37:0/3635496405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333c5550 0x55bf333c2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.944+0000 7f1e05a6c640 1 -- 172.21.15.37:0/3635496405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333bbfd0 msgr2=0x55bf333c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.944+0000 7f1e05a6c640 1 --2- 172.21.15.37:0/3635496405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333bbfd0 0x55bf333c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.944+0000 7f1e05a6c640 1 -- 172.21.15.37:0/3635496405 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf33101620 con 0x55bf333bcd40 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.944+0000 7f1e05a6c640 1 --2- 172.21.15.37:0/3635496405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333c2a60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0ccda0 tx=0x7f1dfc09b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d52b2277311e8831 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.944+0000 7f1de9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.944+0000 7f1de9ffb640 1 -- 172.21.15.37:0/3635496405 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0a3e50 con 0x55bf333bcd40 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.944+0000 7f1de9ffb640 1 -- 172.21.15.37:0/3635496405 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1dfc09abb0 con 0x55bf333bcd40 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.944+0000 7f1e0b194740 1 -- 172.21.15.37:0/3635496405 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1df001e400 con 0x55bf333bcd40 2024-03-26T06:37:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.944+0000 7f1de9ffb640 1 -- 172.21.15.37:0/3635496405 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dfc09adb0 con 0x55bf333bcd40 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.945+0000 7f1e0b194740 10 client.26089 fetch_fsmap learned FSMap version 56 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.945+0000 7f1e0b194740 1 -- 172.21.15.37:0/3635496405 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55bf333d45a0 con 0x55bf333bcd40 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.945+0000 7f1de9ffb640 1 -- 172.21.15.37:0/3635496405 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df001e400 con 0x55bf333bcd40 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.945+0000 7f1de9ffb640 1 -- 172.21.15.37:0/3635496405 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f1dfc097640 con 0x55bf333bcd40 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.945+0000 7f1e0b194740 10 client.26089 fetch_fsmap finished waiting for FSMap version 56 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.945+0000 7f1e0b194740 10 client.26089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.945+0000 7f1e0b194740 20 client.26089 populate_metadata read hostname 'smithi037' 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.945+0000 7f1e0b194740 1 --2- 172.21.15.37:0/3635496405 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bf333d5a60 0x55bf333d7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.945+0000 7f1e0b194740 4 client.26089 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.945+0000 7f1e0b194740 1 -- 172.21.15.37:0/3635496405 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf3332db30 con 0x55bf333d5a60 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.945+0000 7f1e0626d640 1 --2- 172.21.15.37:0/3635496405 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bf333d5a60 0x55bf333d7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.946+0000 7f1e0626d640 1 --2- 172.21.15.37:0/3635496405 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bf333d5a60 0x55bf333d7e80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7084d599ffb5482d server_cookie=a824f8fa861ae193 in_seq=0 out_seq=0 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.946+0000 7f1de9ffb640 10 client.26089 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1de9ffb640 1 -- 172.21.15.37:0/3635496405 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf3332db30 con 0x55bf333d5a60 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1de9ffb640 10 client.26089 handle_command_reply: tid=0 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 1 client.26089 shutdown 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 1 -- 172.21.15.37:0/3635496405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 msgr2=0x55bf333c2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 1 --2- 172.21.15.37:0/3635496405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333c2a60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0ccda0 tx=0x7f1dfc09b390 comp rx=0 tx=0).stop 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 1 -- 172.21.15.37:0/3635496405 shutdown_connections 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 1 --2- 172.21.15.37:0/3635496405 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bf333d5a60 0x55bf333d7e80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 1 --2- 172.21.15.37:0/3635496405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333c5550 0x55bf333c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1de9ffb640 0 client.26089 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 1 --2- 172.21.15.37:0/3635496405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333c2a60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 1 --2- 172.21.15.37:0/3635496405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333bbfd0 0x55bf333c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 1 -- 172.21.15.37:0/3635496405 >> 172.21.15.37:0/3635496405 conn(0x55bf333835b0 msgr2=0x55bf333b8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 1 -- 172.21.15.37:0/3635496405 shutdown_connections 2024-03-26T06:37:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 1 -- 172.21.15.37:0/3635496405 wait complete. 2024-03-26T06:37:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.947+0000 7f1e0b194740 20 client.26089 trim_cache size 0 max 16384 2024-03-26T06:37:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.969+0000 7f1e0b194740 1 Processor -- start 2024-03-26T06:37:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.969+0000 7f1e0b194740 1 -- start start 2024-03-26T06:37:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 0x55bf333bf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e0b194740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0006eeb0 con 0x55bf333c5550 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e0b194740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e000709e0 con 0x55bf333bbfd0 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e0b194740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0006f5d0 con 0x55bf333bcd40 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e06a6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 0x55bf333bf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e0626d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e0626d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333c4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:51252/0 (socket says 172.21.15.37:51252) 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e0626d640 1 -- 172.21.15.37:0/2517052793 learned_addr learned my addr 172.21.15.37:0/2517052793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e05a6c640 1 --2- 172.21.15.37:0/2517052793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e06a6e640 1 -- 172.21.15.37:0/2517052793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 msgr2=0x55bf333c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e06a6e640 1 --2- 172.21.15.37:0/2517052793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e06a6e640 1 -- 172.21.15.37:0/2517052793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 msgr2=0x55bf333c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e06a6e640 1 --2- 172.21.15.37:0/2517052793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e06a6e640 1 -- 172.21.15.37:0/2517052793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf333b2590 con 0x55bf333c5550 2024-03-26T06:37:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e0626d640 1 --2- 172.21.15.37:0/2517052793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:37:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.970+0000 7f1e06a6e640 1 --2- 172.21.15.37:0/2517052793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 0x55bf333bf7a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009bf0 tx=0x7f1df8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad19c76c3a82a7f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.971+0000 7f1debfff640 1 -- 172.21.15.37:0/2517052793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8039040 con 0x55bf333c5550 2024-03-26T06:37:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.971+0000 7f1debfff640 1 -- 172.21.15.37:0/2517052793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df8003e50 con 0x55bf333c5550 2024-03-26T06:37:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.971+0000 7f1debfff640 1 -- 172.21.15.37:0/2517052793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df804e6c0 con 0x55bf333c5550 2024-03-26T06:37:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.972+0000 7f1e0b194740 1 -- 172.21.15.37:0/2517052793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 msgr2=0x55bf333bf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.972+0000 7f1e0b194740 1 --2- 172.21.15.37:0/2517052793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 0x55bf333bf7a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009bf0 tx=0x7f1df8041000 comp rx=0 tx=0).stop 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.972+0000 7f1e0b194740 1 -- 172.21.15.37:0/2517052793 shutdown_connections 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.972+0000 7f1e0b194740 1 --2- 172.21.15.37:0/2517052793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 0x55bf333bf7a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.972+0000 7f1e0b194740 1 --2- 172.21.15.37:0/2517052793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.972+0000 7f1e0b194740 1 --2- 172.21.15.37:0/2517052793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.972+0000 7f1e0b194740 1 -- 172.21.15.37:0/2517052793 >> 172.21.15.37:0/2517052793 conn(0x55bf333835b0 msgr2=0x55bf333bd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.972+0000 7f1e0b194740 1 -- 172.21.15.37:0/2517052793 shutdown_connections 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.972+0000 7f1e0b194740 1 -- 172.21.15.37:0/2517052793 wait complete. 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.973+0000 7f1e0b194740 1 -- start start 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.973+0000 7f1e0b194740 1 Processor -- start 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.973+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.973+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.973+0000 7f1e0b194740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 0x55bf333cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.973+0000 7f1e0b194740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00070380 con 0x55bf333c5550 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.973+0000 7f1e0b194740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0006fc60 con 0x55bf333bbfd0 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.973+0000 7f1e0b194740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0006d1b0 con 0x55bf333bcd40 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.973+0000 7f1e0b194740 10 client.0 _ll_register_callbacks cb 0x7f1df0014dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1e0626d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1e06a6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 0x55bf333cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1e06a6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 0x55bf333cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33508/0 (socket says 172.21.15.37:33508) 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1e06a6e640 1 -- 172.21.15.37:0/173130247 learned_addr learned my addr 172.21.15.37:0/173130247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1e05a6c640 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1e05a6c640 1 -- 172.21.15.37:0/173130247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 msgr2=0x55bf333cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1e05a6c640 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1e05a6c640 1 -- 172.21.15.37:0/173130247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 msgr2=0x55bf333cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1e05a6c640 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 0x55bf333cb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1e05a6c640 1 -- 172.21.15.37:0/173130247 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf333cb880 con 0x55bf333bcd40 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1e05a6c640 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333cad00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0cc850 tx=0x7f1dfc009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=159a5bda3b9fb479 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1de9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1de9ffb640 1 -- 172.21.15.37:0/173130247 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0a3e50 con 0x55bf333bcd40 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.974+0000 7f1de9ffb640 1 -- 172.21.15.37:0/173130247 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1dfc09abb0 con 0x55bf333bcd40 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1e0b194740 1 -- 172.21.15.37:0/173130247 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1df8003e50 con 0x55bf333bcd40 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1de9ffb640 1 -- 172.21.15.37:0/173130247 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dfc09a9d0 con 0x55bf333bcd40 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1e0b194740 10 client.26095 fetch_fsmap learned FSMap version 56 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1e0b194740 1 -- 172.21.15.37:0/173130247 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55bf3342ce10 con 0x55bf333bcd40 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1de9ffb640 1 -- 172.21.15.37:0/173130247 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df8003e50 con 0x55bf333bcd40 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1de9ffb640 1 -- 172.21.15.37:0/173130247 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f1dfc097540 con 0x55bf333bcd40 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1e0b194740 10 client.26095 fetch_fsmap finished waiting for FSMap version 56 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1e0b194740 10 client.26095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1e0b194740 20 client.26095 populate_metadata read hostname 'smithi037' 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1e0b194740 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bf3342d5a0 0x55bf3342f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1e0b194740 4 client.26095 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:37:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.975+0000 7f1e0b194740 1 -- 172.21.15.37:0/173130247 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bf33327e80 con 0x55bf3342d5a0 2024-03-26T06:37:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.977+0000 7f1e0626d640 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bf3342d5a0 0x55bf3342f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.978+0000 7f1de9ffb640 10 client.26095 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:43.978+0000 7f1e0626d640 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bf3342d5a0 0x55bf3342f9c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38f828b2f4901a7a server_cookie=4b9c906279257821 in_seq=0 out_seq=0 2024-03-26T06:37:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.171+0000 7f1de9ffb640 1 -- 172.21.15.37:0/173130247 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bf33327e80 con 0x55bf3342d5a0 2024-03-26T06:37:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.171+0000 7f1de9ffb640 10 client.26095 handle_command_reply: tid=0 2024-03-26T06:37:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.171+0000 7f1e0b194740 1 client.26095 shutdown 2024-03-26T06:37:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.171+0000 7f1e0b194740 1 -- 172.21.15.37:0/173130247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 msgr2=0x55bf333cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.172+0000 7f1e0b194740 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333cad00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0cc850 tx=0x7f1dfc009700 comp rx=0 tx=0).stop 2024-03-26T06:37:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.172+0000 7f1e0b194740 1 -- 172.21.15.37:0/173130247 shutdown_connections 2024-03-26T06:37:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.172+0000 7f1e0b194740 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bf3342d5a0 0x55bf3342f9c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.172+0000 7f1de9ffb640 0 client.26095 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.172+0000 7f1e0b194740 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf333c5550 0x55bf333cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.172+0000 7f1e0b194740 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf333bcd40 0x55bf333cad00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.172+0000 7f1e0b194740 1 --2- 172.21.15.37:0/173130247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bf333bbfd0 0x55bf333cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.172+0000 7f1e0b194740 1 -- 172.21.15.37:0/173130247 >> 172.21.15.37:0/173130247 conn(0x55bf333835b0 msgr2=0x55bf333c0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.172+0000 7f1e0b194740 1 -- 172.21.15.37:0/173130247 shutdown_connections 2024-03-26T06:37:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.172+0000 7f1e0b194740 1 -- 172.21.15.37:0/173130247 wait complete. 2024-03-26T06:37:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.173+0000 7f1e0b194740 20 client.26095 trim_cache size 0 max 16384 2024-03-26T06:37:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:37:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:37:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:37:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:37:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.173+0000 7f1e084d1640 1 -- 172.21.15.37:0/1448459244 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1dec0a2c10 msgr2=0x7f1dec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.173+0000 7f1e084d1640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1dec0a2c10 0x7f1dec0a50d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1df8005e30 tx=0x7f1df8005d80 comp rx=0 tx=0).stop 2024-03-26T06:37:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.173+0000 7f1e084d1640 1 -- 172.21.15.37:0/1448459244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00100e00 msgr2=0x7f1e0019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.173+0000 7f1e084d1640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00100e00 0x7f1e0019a760 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00e9f0 tx=0x7f1dfc00eeb0 comp rx=0 tx=0).stop 2024-03-26T06:37:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.174+0000 7f1e084d1640 1 -- 172.21.15.37:0/1448459244 shutdown_connections 2024-03-26T06:37:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.174+0000 7f1e084d1640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1dec0a2c10 0x7f1dec0a50d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.174+0000 7f1e084d1640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e00105190 0x7f1e0019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.174+0000 7f1e084d1640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e00100e00 0x7f1e0019a760 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.174+0000 7f1e084d1640 1 --2- 172.21.15.37:0/1448459244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e00100440 0x7f1e0019a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.174+0000 7f1e084d1640 1 -- 172.21.15.37:0/1448459244 >> 172.21.15.37:0/1448459244 conn(0x7f1e00077040 msgr2=0x7f1e000fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.174+0000 7f1e084d1640 1 -- 172.21.15.37:0/1448459244 shutdown_connections 2024-03-26T06:37:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:44.174+0000 7f1e084d1640 1 -- 172.21.15.37:0/1448459244 wait complete. 2024-03-26T06:37:44.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:44 smithi066 ceph-mon[36504]: pgmap v464: 97 pgs: 97 active+clean; 770 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2024-03-26T06:37:44.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:44 smithi149 ceph-mon[36681]: pgmap v464: 97 pgs: 97 active+clean; 770 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2024-03-26T06:37:44.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:44 smithi037 ceph-mon[31974]: pgmap v464: 97 pgs: 97 active+clean; 770 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2024-03-26T06:37:44.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 72925 51.88 MB/sec warmup 79 sec latency 115.206 ms 2024-03-26T06:37:45.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 74092 52.04 MB/sec warmup 80 sec latency 56.089 ms 2024-03-26T06:37:46.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:46 smithi066 ceph-mon[36504]: pgmap v465: 97 pgs: 97 active+clean; 778 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.29k op/s 2024-03-26T06:37:46.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:46 smithi149 ceph-mon[36681]: pgmap v465: 97 pgs: 97 active+clean; 778 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.29k op/s 2024-03-26T06:37:46.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:46 smithi037 ceph-mon[31974]: pgmap v465: 97 pgs: 97 active+clean; 778 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.29k op/s 2024-03-26T06:37:46.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 75114 52.21 MB/sec warmup 81 sec latency 39.455 ms 2024-03-26T06:37:47.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 75930 51.97 MB/sec warmup 82 sec latency 72.345 ms 2024-03-26T06:37:48.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:48 smithi066 ceph-mon[36504]: pgmap v466: 97 pgs: 97 active+clean; 797 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 23 MiB/s wr, 1.44k op/s 2024-03-26T06:37:48.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:48 smithi149 ceph-mon[36681]: pgmap v466: 97 pgs: 97 active+clean; 797 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 23 MiB/s wr, 1.44k op/s 2024-03-26T06:37:48.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:48 smithi037 ceph-mon[31974]: pgmap v466: 97 pgs: 97 active+clean; 797 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 23 MiB/s wr, 1.44k op/s 2024-03-26T06:37:48.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 76980 52.05 MB/sec warmup 83 sec latency 59.471 ms 2024-03-26T06:37:49.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 77948 52.12 MB/sec warmup 84 sec latency 52.732 ms 2024-03-26T06:37:50.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:50 smithi066 ceph-mon[36504]: pgmap v467: 97 pgs: 97 active+clean; 797 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2024-03-26T06:37:50.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:50 smithi149 ceph-mon[36681]: pgmap v467: 97 pgs: 97 active+clean; 797 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2024-03-26T06:37:50.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:50 smithi037 ceph-mon[31974]: pgmap v467: 97 pgs: 97 active+clean; 797 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2024-03-26T06:37:50.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 78899 52.07 MB/sec warmup 85 sec latency 48.026 ms 2024-03-26T06:37:51.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 79748 52.00 MB/sec warmup 86 sec latency 66.138 ms 2024-03-26T06:37:52.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:52 smithi066 ceph-mon[36504]: pgmap v468: 97 pgs: 97 active+clean; 829 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-03-26T06:37:52.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:52 smithi149 ceph-mon[36681]: pgmap v468: 97 pgs: 97 active+clean; 829 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-03-26T06:37:52.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:52 smithi037 ceph-mon[31974]: pgmap v468: 97 pgs: 97 active+clean; 829 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-03-26T06:37:52.847 INFO:tasks.workunit.client.0.smithi037.stdout: 10 80704 51.96 MB/sec warmup 87 sec latency 166.097 ms 2024-03-26T06:37:53.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 81880 52.06 MB/sec warmup 88 sec latency 45.438 ms 2024-03-26T06:37:54.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:54 smithi066 ceph-mon[36504]: pgmap v469: 97 pgs: 97 active+clean; 829 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 707 op/s 2024-03-26T06:37:54.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:54 smithi149 ceph-mon[36681]: pgmap v469: 97 pgs: 97 active+clean; 829 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 707 op/s 2024-03-26T06:37:54.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:54 smithi037 ceph-mon[31974]: pgmap v469: 97 pgs: 97 active+clean; 829 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 707 op/s 2024-03-26T06:37:54.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 82874 52.24 MB/sec warmup 89 sec latency 37.120 ms 2024-03-26T06:37:55.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 83657 52.00 MB/sec warmup 90 sec latency 116.043 ms 2024-03-26T06:37:56.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:56 smithi066 ceph-mon[36504]: pgmap v470: 97 pgs: 97 active+clean; 840 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.8 MiB/s wr, 822 op/s 2024-03-26T06:37:56.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:56 smithi149 ceph-mon[36681]: pgmap v470: 97 pgs: 97 active+clean; 840 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.8 MiB/s wr, 822 op/s 2024-03-26T06:37:56.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:56 smithi037 ceph-mon[31974]: pgmap v470: 97 pgs: 97 active+clean; 840 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.8 MiB/s wr, 822 op/s 2024-03-26T06:37:56.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 84782 52.14 MB/sec warmup 91 sec latency 54.175 ms 2024-03-26T06:37:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.283+0000 7f686cf5b640 1 -- 172.21.15.37:0/3232882480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f686810d8f0 msgr2=0x7f686810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.283+0000 7f686cf5b640 1 --2- 172.21.15.37:0/3232882480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f686810d8f0 0x7f686810dcd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6858009930 tx=0x7f685802bc30 comp rx=0 tx=0).stop 2024-03-26T06:37:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.283+0000 7f686cf5b640 1 -- 172.21.15.37:0/3232882480 shutdown_connections 2024-03-26T06:37:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.283+0000 7f686cf5b640 1 --2- 172.21.15.37:0/3232882480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68681082a0 0x7f68681147b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.283+0000 7f686cf5b640 1 --2- 172.21.15.37:0/3232882480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681078e0 0x7f6868107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.283+0000 7f686cf5b640 1 --2- 172.21.15.37:0/3232882480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f686810d8f0 0x7f686810dcd0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.283+0000 7f686cf5b640 1 -- 172.21.15.37:0/3232882480 >> 172.21.15.37:0/3232882480 conn(0x7f6868077020 msgr2=0x7f6868106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.284+0000 7f686cf5b640 1 -- 172.21.15.37:0/3232882480 shutdown_connections 2024-03-26T06:37:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.284+0000 7f686cf5b640 1 -- 172.21.15.37:0/3232882480 wait complete. 2024-03-26T06:37:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.284+0000 7f686cf5b640 1 Processor -- start 2024-03-26T06:37:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.284+0000 7f686cf5b640 1 -- start start 2024-03-26T06:37:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f686cf5b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681078e0 0x7f686819e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f686cf5b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68681082a0 0x7f686819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f686cf5b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f686810d8f0 0x7f68681a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f686cf5b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6868117cc0 con 0x7f686810d8f0 2024-03-26T06:37:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f686cf5b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6868117b40 con 0x7f68681078e0 2024-03-26T06:37:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f686cf5b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6868117e40 con 0x7f68681082a0 2024-03-26T06:37:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f6865d9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68681082a0 0x7f686819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f6866d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f686810d8f0 0x7f68681a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f6865d9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68681082a0 0x7f686819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:49174/0 (socket says 172.21.15.37:49174) 2024-03-26T06:37:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f6865d9b640 1 -- 172.21.15.37:0/1304423499 learned_addr learned my addr 172.21.15.37:0/1304423499 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f6865d9b640 1 -- 172.21.15.37:0/1304423499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681078e0 msgr2=0x7f686819e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f6865d9b640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681078e0 0x7f686819e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f6865d9b640 1 -- 172.21.15.37:0/1304423499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f686810d8f0 msgr2=0x7f68681a29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f6865d9b640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f686810d8f0 0x7f68681a29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f6865d9b640 1 -- 172.21.15.37:0/1304423499 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68681a30d0 con 0x7f68681082a0 2024-03-26T06:37:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.285+0000 7f6866d9d640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f686810d8f0 0x7f68681a29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:37:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.286+0000 7f6865d9b640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68681082a0 0x7f686819e620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f685c00e9f0 tx=0x7f685c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.286+0000 7f68577fe640 1 -- 172.21.15.37:0/1304423499 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c00cd80 con 0x7f68681082a0 2024-03-26T06:37:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.286+0000 7f68577fe640 1 -- 172.21.15.37:0/1304423499 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f685c010910 con 0x7f68681082a0 2024-03-26T06:37:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.286+0000 7f686cf5b640 1 -- 172.21.15.37:0/1304423499 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68681a33c0 con 0x7f68681082a0 2024-03-26T06:37:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.286+0000 7f68577fe640 1 -- 172.21.15.37:0/1304423499 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c0188f0 con 0x7f68681082a0 2024-03-26T06:37:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.286+0000 7f686cf5b640 1 -- 172.21.15.37:0/1304423499 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68681aac50 con 0x7f68681082a0 2024-03-26T06:37:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.289+0000 7f68577fe640 1 -- 172.21.15.37:0/1304423499 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f685c010ac0 con 0x7f68681082a0 2024-03-26T06:37:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.290+0000 7f68577fe640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f68500a2a90 0x7f68500a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.290+0000 7f686fc18740 1 Processor -- start 2024-03-26T06:37:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.290+0000 7f68577fe640 1 -- 172.21.15.37:0/1304423499 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f685c0cf340 con 0x7f68681082a0 2024-03-26T06:37:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.290+0000 7f686fc18740 1 -- start start 2024-03-26T06:37:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.290+0000 7f686659c640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f68500a2a90 0x7f68500a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.290+0000 7f686fc18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb65d40 0x558b8fb64a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.290+0000 7f686fc18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb64fd0 0x558b8fb65450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.290+0000 7f686fc18740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb6e550 0x558b8fb70940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.290+0000 7f686fc18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8fac95f0 con 0x558b8fb64fd0 2024-03-26T06:37:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.290+0000 7f686fc18740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8fa72930 con 0x558b8fb6e550 2024-03-26T06:37:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.290+0000 7f686fc18740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8fac5690 con 0x558b8fb65d40 2024-03-26T06:37:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.291+0000 7f686659c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb65d40 0x558b8fb64a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.291+0000 7f6866d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb6e550 0x558b8fb70940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.291+0000 7f6866d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb6e550 0x558b8fb70940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:42468/0 (socket says 172.21.15.37:42468) 2024-03-26T06:37:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.291+0000 7f6866d9d640 1 -- 172.21.15.37:0/827163079 learned_addr learned my addr 172.21.15.37:0/827163079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.291+0000 7f6865d9b640 1 --2- 172.21.15.37:0/827163079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb64fd0 0x558b8fb65450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.291+0000 7f686659c640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f68500a2a90 0x7f68500a4f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6858005ec0 tx=0x7f685802c5d0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.291+0000 7f6865d9b640 1 -- 172.21.15.37:0/827163079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb65d40 msgr2=0x558b8fb64a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.291+0000 7f6865d9b640 1 --2- 172.21.15.37:0/827163079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb65d40 0x558b8fb64a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.291+0000 7f6865d9b640 1 -- 172.21.15.37:0/827163079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb6e550 msgr2=0x558b8fb70940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.291+0000 7f6865d9b640 1 --2- 172.21.15.37:0/827163079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb6e550 0x558b8fb70940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.291+0000 7f6865d9b640 1 -- 172.21.15.37:0/827163079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b8f7de210 con 0x558b8fb64fd0 2024-03-26T06:37:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.292+0000 7f6865d9b640 1 --2- 172.21.15.37:0/827163079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb64fd0 0x558b8fb65450 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f685c0cca30 tx=0x7f685c09c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2664d764b3879e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.292+0000 7f6854ff9640 1 -- 172.21.15.37:0/827163079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c0a40e0 con 0x558b8fb64fd0 2024-03-26T06:37:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.292+0000 7f6854ff9640 1 -- 172.21.15.37:0/827163079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f685c0ce070 con 0x558b8fb64fd0 2024-03-26T06:37:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.292+0000 7f6854ff9640 1 -- 172.21.15.37:0/827163079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c09ad90 con 0x558b8fb64fd0 2024-03-26T06:37:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.293+0000 7f686fc18740 1 -- 172.21.15.37:0/827163079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb64fd0 msgr2=0x558b8fb65450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.293+0000 7f686fc18740 1 --2- 172.21.15.37:0/827163079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb64fd0 0x558b8fb65450 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f685c0cca30 tx=0x7f685c09c100 comp rx=0 tx=0).stop 2024-03-26T06:37:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.293+0000 7f686fc18740 1 -- 172.21.15.37:0/827163079 shutdown_connections 2024-03-26T06:37:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.293+0000 7f686fc18740 1 --2- 172.21.15.37:0/827163079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb6e550 0x558b8fb70940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.293+0000 7f686fc18740 1 --2- 172.21.15.37:0/827163079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb64fd0 0x558b8fb65450 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.293+0000 7f686fc18740 1 --2- 172.21.15.37:0/827163079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb65d40 0x558b8fb64a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.293+0000 7f686fc18740 1 -- 172.21.15.37:0/827163079 >> 172.21.15.37:0/827163079 conn(0x558b8fb2c5b0 msgr2=0x558b8fb5fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.293+0000 7f686fc18740 1 -- 172.21.15.37:0/827163079 shutdown_connections 2024-03-26T06:37:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.293+0000 7f686fc18740 1 -- 172.21.15.37:0/827163079 wait complete. 2024-03-26T06:37:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.294+0000 7f686fc18740 1 -- start start 2024-03-26T06:37:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.294+0000 7f686fc18740 1 Processor -- start 2024-03-26T06:37:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.294+0000 7f686fc18740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb64fd0 0x558b8fb6b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.294+0000 7f686fc18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb65d40 0x558b8fb6bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.294+0000 7f686fc18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb6e550 0x558b8fb6c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.294+0000 7f686fc18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8fb71210 con 0x558b8fb65d40 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.294+0000 7f686fc18740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8fa85d90 con 0x558b8fb64fd0 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.294+0000 7f686fc18740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8fb71390 con 0x558b8fb6e550 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.294+0000 7f686fc18740 10 client.0 _ll_register_callbacks cb 0x558b8f7dda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.294+0000 7f6865d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb65d40 0x558b8fb6bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.294+0000 7f6865d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb65d40 0x558b8fb6bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52546/0 (socket says 172.21.15.37:52546) 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f6865d9b640 1 -- 172.21.15.37:0/2018039867 learned_addr learned my addr 172.21.15.37:0/2018039867 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f6866d9d640 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb6e550 0x558b8fb6c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f686659c640 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb64fd0 0x558b8fb6b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f6865d9b640 1 -- 172.21.15.37:0/2018039867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb6e550 msgr2=0x558b8fb6c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f6865d9b640 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb6e550 0x558b8fb6c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f6865d9b640 1 -- 172.21.15.37:0/2018039867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb64fd0 msgr2=0x558b8fb6b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f6865d9b640 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb64fd0 0x558b8fb6b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f6865d9b640 1 -- 172.21.15.37:0/2018039867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b8f8aa620 con 0x558b8fb65d40 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f6866d9d640 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb6e550 0x558b8fb6c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f6865d9b640 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb65d40 0x558b8fb6bb60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f685c09b8d0 tx=0x7f685c09b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca517e9c2f5bcd1c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f684dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f684dffb640 1 -- 172.21.15.37:0/2018039867 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c0b3020 con 0x558b8fb65d40 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f684dffb640 1 -- 172.21.15.37:0/2018039867 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f685c0862c0 con 0x558b8fb65d40 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.295+0000 7f686fc18740 1 -- 172.21.15.37:0/2018039867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f685c0ce070 con 0x558b8fb65d40 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f684dffb640 1 -- 172.21.15.37:0/2018039867 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f685c0a40e0 con 0x558b8fb65d40 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f686fc18740 10 client.16722 fetch_fsmap learned FSMap version 56 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f686fc18740 1 -- 172.21.15.37:0/2018039867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x558b8fb71210 con 0x558b8fb65d40 2024-03-26T06:37:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f684dffb640 1 -- 172.21.15.37:0/2018039867 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f685c0ce070 con 0x558b8fb65d40 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f684dffb640 1 -- 172.21.15.37:0/2018039867 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f685c018a90 con 0x558b8fb65d40 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f686fc18740 10 client.16722 fetch_fsmap finished waiting for FSMap version 56 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f686fc18740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f686fc18740 20 client.16722 populate_metadata read hostname 'smithi037' 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f686fc18740 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8fb7e4f0 0x558b8fb808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f686fc18740 4 client.16722 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f686fc18740 1 -- 172.21.15.37:0/2018039867 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b8fad6b30 con 0x558b8fb7e4f0 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.296+0000 7f686659c640 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8fb7e4f0 0x558b8fb808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.297+0000 7f684dffb640 10 client.16722 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.297+0000 7f686659c640 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8fb7e4f0 0x558b8fb808e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f22b75cc3e006a9 server_cookie=e9b3576653943756 in_seq=0 out_seq=0 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.299+0000 7f684dffb640 1 -- 172.21.15.37:0/2018039867 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b8fad6b30 con 0x558b8fb7e4f0 2024-03-26T06:37:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.299+0000 7f684dffb640 10 client.16722 handle_command_reply: tid=0 2024-03-26T06:37:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.299+0000 7f686fc18740 1 client.16722 shutdown 2024-03-26T06:37:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.299+0000 7f686fc18740 1 -- 172.21.15.37:0/2018039867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb65d40 msgr2=0x558b8fb6bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.299+0000 7f686fc18740 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb65d40 0x558b8fb6bb60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f685c09b8d0 tx=0x7f685c09b040 comp rx=0 tx=0).stop 2024-03-26T06:37:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.299+0000 7f686fc18740 1 -- 172.21.15.37:0/2018039867 shutdown_connections 2024-03-26T06:37:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.299+0000 7f686fc18740 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8fb7e4f0 0x558b8fb808e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.299+0000 7f686fc18740 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb6e550 0x558b8fb6c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.299+0000 7f686fc18740 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb65d40 0x558b8fb6bb60 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.299+0000 7f686fc18740 1 --2- 172.21.15.37:0/2018039867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb64fd0 0x558b8fb6b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.299+0000 7f686fc18740 1 -- 172.21.15.37:0/2018039867 >> 172.21.15.37:0/2018039867 conn(0x558b8fb2c5b0 msgr2=0x558b8fb61a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.300+0000 7f684dffb640 0 client.16722 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.300+0000 7f686fc18740 1 -- 172.21.15.37:0/2018039867 shutdown_connections 2024-03-26T06:37:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.300+0000 7f686fc18740 1 -- 172.21.15.37:0/2018039867 wait complete. 2024-03-26T06:37:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.300+0000 7f686fc18740 20 client.16722 trim_cache size 0 max 16384 2024-03-26T06:37:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.322+0000 7f686fc18740 1 Processor -- start 2024-03-26T06:37:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.323+0000 7f686fc18740 1 -- start start 2024-03-26T06:37:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.323+0000 7f686fc18740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb64fd0 0x558b8fb6dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.323+0000 7f686fc18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb65d40 0x558b8fb75600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.323+0000 7f686fc18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 0x558b8fb75b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.323+0000 7f686fc18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681b3280 con 0x558b8fb6e550 2024-03-26T06:37:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.323+0000 7f686659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb64fd0 0x558b8fb6dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.323+0000 7f686659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb64fd0 0x558b8fb6dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:42496/0 (socket says 172.21.15.37:42496) 2024-03-26T06:37:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.323+0000 7f686659c640 1 -- 172.21.15.37:0/1374692411 learned_addr learned my addr 172.21.15.37:0/1374692411 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.323+0000 7f6866d9d640 1 --2- 172.21.15.37:0/1374692411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 0x558b8fb75b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.323+0000 7f686fc18740 1 -- 172.21.15.37:0/1374692411 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681b4db0 con 0x558b8fb64fd0 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.323+0000 7f686fc18740 1 -- 172.21.15.37:0/1374692411 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681b39a0 con 0x558b8fb65d40 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.324+0000 7f6865d9b640 1 --2- 172.21.15.37:0/1374692411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb65d40 0x558b8fb75600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.324+0000 7f6866d9d640 1 -- 172.21.15.37:0/1374692411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb65d40 msgr2=0x558b8fb75600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.324+0000 7f6866d9d640 1 --2- 172.21.15.37:0/1374692411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb65d40 0x558b8fb75600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.324+0000 7f6866d9d640 1 -- 172.21.15.37:0/1374692411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb64fd0 msgr2=0x558b8fb6dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.324+0000 7f6866d9d640 1 --2- 172.21.15.37:0/1374692411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb64fd0 0x558b8fb6dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.324+0000 7f6866d9d640 1 -- 172.21.15.37:0/1374692411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b8fb642a0 con 0x558b8fb6e550 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.324+0000 7f6866d9d640 1 --2- 172.21.15.37:0/1374692411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 0x558b8fb75b40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f68600092c0 tx=0x7f6860011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cff2528f1fab0d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.324+0000 7f684ffff640 1 -- 172.21.15.37:0/1374692411 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860013bb0 con 0x558b8fb6e550 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.324+0000 7f684ffff640 1 -- 172.21.15.37:0/1374692411 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6860013d50 con 0x558b8fb6e550 2024-03-26T06:37:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.324+0000 7f684ffff640 1 -- 172.21.15.37:0/1374692411 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68600103d0 con 0x558b8fb6e550 2024-03-26T06:37:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.325+0000 7f686fc18740 1 -- 172.21.15.37:0/1374692411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 msgr2=0x558b8fb75b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.325+0000 7f686fc18740 1 --2- 172.21.15.37:0/1374692411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 0x558b8fb75b40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f68600092c0 tx=0x7f6860011f50 comp rx=0 tx=0).stop 2024-03-26T06:37:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.325+0000 7f686fc18740 1 -- 172.21.15.37:0/1374692411 shutdown_connections 2024-03-26T06:37:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.325+0000 7f686fc18740 1 --2- 172.21.15.37:0/1374692411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 0x558b8fb75b40 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.325+0000 7f686fc18740 1 --2- 172.21.15.37:0/1374692411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb65d40 0x558b8fb75600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.325+0000 7f686fc18740 1 --2- 172.21.15.37:0/1374692411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb64fd0 0x558b8fb6dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.325+0000 7f686fc18740 1 -- 172.21.15.37:0/1374692411 >> 172.21.15.37:0/1374692411 conn(0x558b8fb2c5b0 msgr2=0x558b8fb80500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.325+0000 7f686fc18740 1 -- 172.21.15.37:0/1374692411 shutdown_connections 2024-03-26T06:37:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.325+0000 7f686fc18740 1 -- 172.21.15.37:0/1374692411 wait complete. 2024-03-26T06:37:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686fc18740 1 -- start start 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686fc18740 1 Processor -- start 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686fc18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb64fd0 0x558b8fb75610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686fc18740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb65d40 0x558b8fb67a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686fc18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 0x558b8fb67f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686fc18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681b4750 con 0x558b8fb6e550 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686fc18740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681b4030 con 0x558b8fb65d40 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686fc18740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681b1540 con 0x558b8fb64fd0 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686fc18740 10 client.0 _ll_register_callbacks cb 0x7f685c0974c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f6866d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 0x558b8fb67f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686659c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb64fd0 0x558b8fb75610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686659c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb64fd0 0x558b8fb75610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:49192/0 (socket says 172.21.15.37:49192) 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f686659c640 1 -- 172.21.15.37:0/3434019869 learned_addr learned my addr 172.21.15.37:0/3434019869 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f6866d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 0x558b8fb67f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52574/0 (socket says 172.21.15.37:52574) 2024-03-26T06:37:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.326+0000 7f6865d9b640 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb65d40 0x558b8fb67a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f6866d9d640 1 -- 172.21.15.37:0/3434019869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb64fd0 msgr2=0x558b8fb75610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f6866d9d640 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb64fd0 0x558b8fb75610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f6866d9d640 1 -- 172.21.15.37:0/3434019869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb65d40 msgr2=0x558b8fb67a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f6866d9d640 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb65d40 0x558b8fb67a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f6866d9d640 1 -- 172.21.15.37:0/3434019869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b8fb75e10 con 0x558b8fb6e550 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f6866d9d640 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 0x558b8fb67f90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6860000f80 tx=0x7f68600118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd190bbb1a479f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f684dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f684dffb640 1 -- 172.21.15.37:0/3434019869 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860029020 con 0x558b8fb6e550 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f684dffb640 1 -- 172.21.15.37:0/3434019869 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6860004c50 con 0x558b8fb6e550 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f686fc18740 1 -- 172.21.15.37:0/3434019869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6860013d50 con 0x558b8fb6e550 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f684dffb640 1 -- 172.21.15.37:0/3434019869 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6860034050 con 0x558b8fb6e550 2024-03-26T06:37:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f686fc18740 10 client.16728 fetch_fsmap learned FSMap version 56 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.327+0000 7f686fc18740 1 -- 172.21.15.37:0/3434019869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x558b8fbd53a0 con 0x558b8fb6e550 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.328+0000 7f684dffb640 1 -- 172.21.15.37:0/3434019869 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6860013d50 con 0x558b8fb6e550 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.328+0000 7f684dffb640 1 -- 172.21.15.37:0/3434019869 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f686000f040 con 0x558b8fb6e550 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.328+0000 7f686fc18740 10 client.16728 fetch_fsmap finished waiting for FSMap version 56 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.328+0000 7f686fc18740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.328+0000 7f686fc18740 20 client.16728 populate_metadata read hostname 'smithi037' 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.328+0000 7f686fc18740 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8fbd5b00 0x558b8fbd7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.328+0000 7f686fc18740 4 client.16728 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.328+0000 7f686fc18740 1 -- 172.21.15.37:0/3434019869 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558b8fad0e80 con 0x558b8fbd5b00 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.328+0000 7f686659c640 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8fbd5b00 0x558b8fbd7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.329+0000 7f686659c640 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8fbd5b00 0x558b8fbd7f20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51508f2aba8d7bca server_cookie=562fafaad0aea76f in_seq=0 out_seq=0 2024-03-26T06:37:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.329+0000 7f684dffb640 10 client.16728 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.610+0000 7f684dffb640 1 -- 172.21.15.37:0/3434019869 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558b8fad0e80 con 0x558b8fbd5b00 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.610+0000 7f684dffb640 10 client.16728 handle_command_reply: tid=0 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.610+0000 7f686fc18740 1 client.16728 shutdown 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.611+0000 7f686fc18740 1 -- 172.21.15.37:0/3434019869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 msgr2=0x558b8fb67f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.611+0000 7f686fc18740 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 0x558b8fb67f90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6860000f80 tx=0x7f68600118c0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.611+0000 7f686fc18740 1 -- 172.21.15.37:0/3434019869 shutdown_connections 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.611+0000 7f686fc18740 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x558b8fbd5b00 0x558b8fbd7f20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.611+0000 7f686fc18740 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8fb6e550 0x558b8fb67f90 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.611+0000 7f684dffb640 0 client.16728 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.611+0000 7f686fc18740 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b8fb65d40 0x558b8fb67a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.611+0000 7f686fc18740 1 --2- 172.21.15.37:0/3434019869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b8fb64fd0 0x558b8fb75610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.611+0000 7f686fc18740 1 -- 172.21.15.37:0/3434019869 >> 172.21.15.37:0/3434019869 conn(0x558b8fb2c5b0 msgr2=0x558b8fb80500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:57.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.612+0000 7f686fc18740 1 -- 172.21.15.37:0/3434019869 shutdown_connections 2024-03-26T06:37:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.612+0000 7f686fc18740 1 -- 172.21.15.37:0/3434019869 wait complete. 2024-03-26T06:37:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.612+0000 7f686fc18740 20 client.16728 trim_cache size 0 max 16384 2024-03-26T06:37:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:37:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:37:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:37:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:37:57.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 -- 172.21.15.37:0/1304423499 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f68500a2a90 msgr2=0x7f68500a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f68500a2a90 0x7f68500a4f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6858005ec0 tx=0x7f685802c5d0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 -- 172.21.15.37:0/1304423499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68681082a0 msgr2=0x7f686819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:37:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68681082a0 0x7f686819e620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f685c00e9f0 tx=0x7f685c00eeb0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 -- 172.21.15.37:0/1304423499 shutdown_connections 2024-03-26T06:37:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f68500a2a90 0x7f68500a4f50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f686810d8f0 0x7f68681a29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68681082a0 0x7f686819e620 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 --2- 172.21.15.37:0/1304423499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68681078e0 0x7f686819e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:37:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 -- 172.21.15.37:0/1304423499 >> 172.21.15.37:0/1304423499 conn(0x7f6868077020 msgr2=0x7f68680775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:37:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 -- 172.21.15.37:0/1304423499 shutdown_connections 2024-03-26T06:37:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:37:57.613+0000 7f686cf5b640 1 -- 172.21.15.37:0/1304423499 wait complete. 2024-03-26T06:37:57.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 85624 52.03 MB/sec warmup 92 sec latency 133.758 ms 2024-03-26T06:37:58.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:37:58 smithi066 ceph-mon[36504]: pgmap v471: 97 pgs: 97 active+clean; 861 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 8.7 MiB/s wr, 884 op/s 2024-03-26T06:37:58.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:37:58 smithi149 ceph-mon[36681]: pgmap v471: 97 pgs: 97 active+clean; 861 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 8.7 MiB/s wr, 884 op/s 2024-03-26T06:37:58.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:37:58 smithi037 ceph-mon[31974]: pgmap v471: 97 pgs: 97 active+clean; 861 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 8.7 MiB/s wr, 884 op/s 2024-03-26T06:37:58.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 86507 52.01 MB/sec warmup 93 sec latency 74.140 ms 2024-03-26T06:37:59.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 87498 52.01 MB/sec warmup 94 sec latency 51.730 ms 2024-03-26T06:38:00.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:00 smithi066 ceph-mon[36504]: pgmap v472: 97 pgs: 97 active+clean; 861 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.4 MiB/s wr, 624 op/s 2024-03-26T06:38:00.414 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:00 smithi149 ceph-mon[36681]: pgmap v472: 97 pgs: 97 active+clean; 861 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.4 MiB/s wr, 624 op/s 2024-03-26T06:38:00.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:00 smithi037 ceph-mon[31974]: pgmap v472: 97 pgs: 97 active+clean; 861 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.4 MiB/s wr, 624 op/s 2024-03-26T06:38:00.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 88278 51.89 MB/sec warmup 95 sec latency 111.433 ms 2024-03-26T06:38:01.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 88616 51.54 MB/sec warmup 96 sec latency 602.522 ms 2024-03-26T06:38:02.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:02 smithi066 ceph-mon[36504]: pgmap v473: 97 pgs: 97 active+clean; 882 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 9.7 MiB/s wr, 996 op/s 2024-03-26T06:38:02.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:02 smithi149 ceph-mon[36681]: pgmap v473: 97 pgs: 97 active+clean; 882 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 9.7 MiB/s wr, 996 op/s 2024-03-26T06:38:02.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:02 smithi037 ceph-mon[31974]: pgmap v473: 97 pgs: 97 active+clean; 882 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 9.7 MiB/s wr, 996 op/s 2024-03-26T06:38:02.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 89103 51.25 MB/sec warmup 97 sec latency 89.949 ms 2024-03-26T06:38:03.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 90114 51.36 MB/sec warmup 98 sec latency 59.270 ms 2024-03-26T06:38:04.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:04 smithi066 ceph-mon[36504]: pgmap v474: 97 pgs: 97 active+clean; 882 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 683 op/s 2024-03-26T06:38:04.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:04 smithi149 ceph-mon[36681]: pgmap v474: 97 pgs: 97 active+clean; 882 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 683 op/s 2024-03-26T06:38:04.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:04 smithi037 ceph-mon[31974]: pgmap v474: 97 pgs: 97 active+clean; 882 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 683 op/s 2024-03-26T06:38:04.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 91068 51.39 MB/sec warmup 99 sec latency 42.929 ms 2024-03-26T06:38:05.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 91407 51.06 MB/sec warmup 100 sec latency 466.759 ms 2024-03-26T06:38:06.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:06 smithi066 ceph-mon[36504]: pgmap v475: 97 pgs: 97 active+clean; 894 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.8 MiB/s wr, 832 op/s 2024-03-26T06:38:06.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:06 smithi066 ceph-mon[36504]: osdmap e108: 12 total, 12 up, 12 in 2024-03-26T06:38:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:06 smithi149 ceph-mon[36681]: pgmap v475: 97 pgs: 97 active+clean; 894 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.8 MiB/s wr, 832 op/s 2024-03-26T06:38:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:06 smithi149 ceph-mon[36681]: osdmap e108: 12 total, 12 up, 12 in 2024-03-26T06:38:06.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:06 smithi037 ceph-mon[31974]: pgmap v475: 97 pgs: 97 active+clean; 894 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.8 MiB/s wr, 832 op/s 2024-03-26T06:38:06.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:06 smithi037 ceph-mon[31974]: osdmap e108: 12 total, 12 up, 12 in 2024-03-26T06:38:06.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 91429 50.56 MB/sec warmup 101 sec latency 1105.361 ms 2024-03-26T06:38:07.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 91834 50.24 MB/sec warmup 102 sec latency 1067.900 ms 2024-03-26T06:38:08.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:08 smithi066 ceph-mon[36504]: pgmap v477: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 911 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.1 MiB/s wr, 983 op/s 2024-03-26T06:38:08.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:08 smithi149 ceph-mon[36681]: pgmap v477: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 911 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.1 MiB/s wr, 983 op/s 2024-03-26T06:38:08.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:08 smithi037 ceph-mon[31974]: pgmap v477: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 911 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.1 MiB/s wr, 983 op/s 2024-03-26T06:38:08.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 92399 50.05 MB/sec warmup 103 sec latency 111.468 ms 2024-03-26T06:38:09.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 93491 50.00 MB/sec warmup 104 sec latency 71.361 ms 2024-03-26T06:38:10.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:10 smithi066 ceph-mon[36504]: pgmap v478: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 911 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.1 MiB/s wr, 983 op/s 2024-03-26T06:38:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:10 smithi149 ceph-mon[36681]: pgmap v478: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 911 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.1 MiB/s wr, 983 op/s 2024-03-26T06:38:10.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:10 smithi037 ceph-mon[31974]: pgmap v478: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 911 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.1 MiB/s wr, 983 op/s 2024-03-26T06:38:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.748+0000 7f354d7ab640 1 -- 172.21.15.37:0/843620098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3548100e70 msgr2=0x7f354810d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.748+0000 7f354d7ab640 1 --2- 172.21.15.37:0/843620098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3548100e70 0x7f354810d010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3540009c10 tx=0x7f354002de60 comp rx=0 tx=0).stop 2024-03-26T06:38:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.748+0000 7f354d7ab640 1 -- 172.21.15.37:0/843620098 shutdown_connections 2024-03-26T06:38:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.748+0000 7f354d7ab640 1 --2- 172.21.15.37:0/843620098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3548100e70 0x7f354810d010 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.748+0000 7f354d7ab640 1 --2- 172.21.15.37:0/843620098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35481004d0 0x7f3548100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.748+0000 7f354d7ab640 1 --2- 172.21.15.37:0/843620098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3548105210 0x7f35481055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.748+0000 7f354d7ab640 1 -- 172.21.15.37:0/843620098 >> 172.21.15.37:0/843620098 conn(0x7f35480771d0 msgr2=0x7f35480775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.748+0000 7f354d7ab640 1 -- 172.21.15.37:0/843620098 shutdown_connections 2024-03-26T06:38:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.748+0000 7f354d7ab640 1 -- 172.21.15.37:0/843620098 wait complete. 2024-03-26T06:38:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.749+0000 7f354d7ab640 1 Processor -- start 2024-03-26T06:38:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.749+0000 7f354d7ab640 1 -- start start 2024-03-26T06:38:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.749+0000 7f354d7ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35481004d0 0x7f3548072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.749+0000 7f354d7ab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3548100e70 0x7f3548072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.749+0000 7f354d7ab640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3548105210 0x7f354806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.749+0000 7f354d7ab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354806b720 con 0x7f35481004d0 2024-03-26T06:38:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.749+0000 7f354d7ab640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354806b5a0 con 0x7f3548105210 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.749+0000 7f354d7ab640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354806b8a0 con 0x7f3548100e70 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f35477fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3548105210 0x7f354806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f3546ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35481004d0 0x7f3548072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f35477fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3548105210 0x7f354806c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:33856/0 (socket says 172.21.15.37:33856) 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f3546ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35481004d0 0x7f3548072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35036/0 (socket says 172.21.15.37:35036) 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f35477fe640 1 -- 172.21.15.37:0/3110168063 learned_addr learned my addr 172.21.15.37:0/3110168063 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f35467fc640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3548100e70 0x7f3548072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f35477fe640 1 -- 172.21.15.37:0/3110168063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3548100e70 msgr2=0x7f3548072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f35477fe640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3548100e70 0x7f3548072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f35477fe640 1 -- 172.21.15.37:0/3110168063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35481004d0 msgr2=0x7f3548072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f35477fe640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35481004d0 0x7f3548072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f35477fe640 1 -- 172.21.15.37:0/3110168063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354806d250 con 0x7f3548105210 2024-03-26T06:38:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.750+0000 7f3546ffd640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35481004d0 0x7f3548072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:38:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.751+0000 7f35477fe640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3548105210 0x7f354806c990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3540009560 tx=0x7f3540035610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.752+0000 7f352ffff640 1 -- 172.21.15.37:0/3110168063 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540003b70 con 0x7f3548105210 2024-03-26T06:38:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.752+0000 7f352ffff640 1 -- 172.21.15.37:0/3110168063 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3540043070 con 0x7f3548105210 2024-03-26T06:38:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.752+0000 7f354d7ab640 1 -- 172.21.15.37:0/3110168063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354806d4e0 con 0x7f3548105210 2024-03-26T06:38:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.752+0000 7f352ffff640 1 -- 172.21.15.37:0/3110168063 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540036690 con 0x7f3548105210 2024-03-26T06:38:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.752+0000 7f354d7ab640 1 -- 172.21.15.37:0/3110168063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35481aee70 con 0x7f3548105210 2024-03-26T06:38:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.756+0000 7f3550381740 1 Processor -- start 2024-03-26T06:38:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.757+0000 7f3550381740 1 -- start start 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.757+0000 7f352ffff640 1 -- 172.21.15.37:0/3110168063 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f354000c050 con 0x7f3548105210 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.757+0000 7f3550381740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2bad40 0x55b27a2b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.757+0000 7f3550381740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.757+0000 7f3550381740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2cbd60 0x55b27a2ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.757+0000 7f3550381740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27a21e5f0 con 0x55b27a2bad40 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.757+0000 7f3550381740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27a1c7930 con 0x55b27a2cbd60 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.757+0000 7f3550381740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27a21a690 con 0x55b27a2b9fd0 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.757+0000 7f3546ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.757+0000 7f3546ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2ba450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:43176/0 (socket says 172.21.15.37:43176) 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.757+0000 7f3546ffd640 1 -- 172.21.15.37:0/3851347890 learned_addr learned my addr 172.21.15.37:0/3851347890 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f35467fc640 1 --2- 172.21.15.37:0/3851347890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2cbd60 0x55b27a2ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f35467fc640 1 --2- 172.21.15.37:0/3851347890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2bad40 0x55b27a2b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f3546ffd640 1 -- 172.21.15.37:0/3851347890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2cbd60 msgr2=0x55b27a2ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f3546ffd640 1 --2- 172.21.15.37:0/3851347890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2cbd60 0x55b27a2ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f3546ffd640 1 -- 172.21.15.37:0/3851347890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2bad40 msgr2=0x55b27a2b9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f3546ffd640 1 --2- 172.21.15.37:0/3851347890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2bad40 0x55b27a2b9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f3546ffd640 1 -- 172.21.15.37:0/3851347890 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b279f33210 con 0x55b27a2b9fd0 2024-03-26T06:38:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f35467fc640 1 --2- 172.21.15.37:0/3851347890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2bad40 0x55b27a2b9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:38:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f3546ffd640 1 --2- 172.21.15.37:0/3851347890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2ba450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f35380114f0 tx=0x7f35380119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e25268bf978921b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f352ffff640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35300a2c40 0x7f35300a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f35467fc640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35300a2c40 0x7f35300a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.758+0000 7f352d7fa640 1 -- 172.21.15.37:0/3851347890 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538004020 con 0x55b27a2b9fd0 2024-03-26T06:38:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.759+0000 7f352ffff640 1 -- 172.21.15.37:0/3110168063 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f35400f1050 con 0x7f3548105210 2024-03-26T06:38:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.759+0000 7f352d7fa640 1 -- 172.21.15.37:0/3851347890 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35380041c0 con 0x55b27a2b9fd0 2024-03-26T06:38:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.759+0000 7f352d7fa640 1 -- 172.21.15.37:0/3851347890 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353800f9c0 con 0x55b27a2b9fd0 2024-03-26T06:38:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f35467fc640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35300a2c40 0x7f35300a5100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f353c0095f0 tx=0x7f353c005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f3550381740 1 -- 172.21.15.37:0/3851347890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 msgr2=0x55b27a2ba450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f3550381740 1 --2- 172.21.15.37:0/3851347890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2ba450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f35380114f0 tx=0x7f35380119b0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f3550381740 1 -- 172.21.15.37:0/3851347890 shutdown_connections 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f3550381740 1 --2- 172.21.15.37:0/3851347890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2cbd60 0x55b27a2ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f3550381740 1 --2- 172.21.15.37:0/3851347890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2ba450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f3550381740 1 --2- 172.21.15.37:0/3851347890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2bad40 0x55b27a2b9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f3550381740 1 -- 172.21.15.37:0/3851347890 >> 172.21.15.37:0/3851347890 conn(0x55b27a2815b0 msgr2=0x55b27a2b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f3550381740 1 -- 172.21.15.37:0/3851347890 shutdown_connections 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f3550381740 1 -- 172.21.15.37:0/3851347890 wait complete. 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f3550381740 1 -- start start 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.760+0000 7f3550381740 1 Processor -- start 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f3550381740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2b9fd0 0x55b27a2c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f3550381740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2bad40 0x55b27a2cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f3550381740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2cbd60 0x55b27a2c0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f3550381740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27a2bb8f0 con 0x55b27a2b9fd0 2024-03-26T06:38:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f3550381740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27a1dad90 con 0x55b27a2cbd60 2024-03-26T06:38:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f3550381740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b27a2bba70 con 0x55b27a2bad40 2024-03-26T06:38:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f3550381740 10 client.0 _ll_register_callbacks cb 0x55b279f32a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:38:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f35467fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2bad40 0x55b27a2cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f35467fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2bad40 0x55b27a2cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:43188/0 (socket says 172.21.15.37:43188) 2024-03-26T06:38:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f35467fc640 1 -- 172.21.15.37:0/89754998 learned_addr learned my addr 172.21.15.37:0/89754998 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f35477fe640 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2cbd60 0x55b27a2c0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.761+0000 7f3546ffd640 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2b9fd0 0x55b27a2c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.762+0000 7f35467fc640 1 -- 172.21.15.37:0/89754998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2cbd60 msgr2=0x55b27a2c0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.762+0000 7f35467fc640 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2cbd60 0x55b27a2c0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.762+0000 7f35467fc640 1 -- 172.21.15.37:0/89754998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2b9fd0 msgr2=0x55b27a2c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.762+0000 7f35467fc640 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2b9fd0 0x55b27a2c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.762+0000 7f35467fc640 1 -- 172.21.15.37:0/89754998 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b279fff620 con 0x55b27a2bad40 2024-03-26T06:38:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.762+0000 7f3546ffd640 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2b9fd0 0x55b27a2c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:38:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.762+0000 7f35467fc640 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2bad40 0x55b27a2cb790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f353c0093e0 tx=0x7f353c009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89b42fcf02c3265 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.762+0000 7f352aff5640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:38:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.762+0000 7f352aff5640 1 -- 172.21.15.37:0/89754998 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c01ec50 con 0x55b27a2bad40 2024-03-26T06:38:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.762+0000 7f3550381740 1 -- 172.21.15.37:0/89754998 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35380041c0 con 0x55b27a2bad40 2024-03-26T06:38:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.762+0000 7f352aff5640 1 -- 172.21.15.37:0/89754998 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f353c01edf0 con 0x55b27a2bad40 2024-03-26T06:38:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.763+0000 7f352aff5640 1 -- 172.21.15.37:0/89754998 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f353c005020 con 0x55b27a2bad40 2024-03-26T06:38:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.766+0000 7f3550381740 10 client.26125 fetch_fsmap learned FSMap version 56 2024-03-26T06:38:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.766+0000 7f3550381740 1 -- 172.21.15.37:0/89754998 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55b27a2e3650 con 0x55b27a2bad40 2024-03-26T06:38:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.767+0000 7f352aff5640 1 -- 172.21.15.37:0/89754998 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35380041c0 con 0x55b27a2bad40 2024-03-26T06:38:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.768+0000 7f352aff5640 1 -- 172.21.15.37:0/89754998 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f353c0284b0 con 0x55b27a2bad40 2024-03-26T06:38:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.768+0000 7f3550381740 10 client.26125 fetch_fsmap finished waiting for FSMap version 56 2024-03-26T06:38:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.768+0000 7f3550381740 10 client.26125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:38:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.768+0000 7f3550381740 20 client.26125 populate_metadata read hostname 'smithi037' 2024-03-26T06:38:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.768+0000 7f3550381740 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b27a2e4b60 0x55b27a2e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.768+0000 7f3550381740 4 client.26125 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:38:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.768+0000 7f3550381740 1 -- 172.21.15.37:0/89754998 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b27a22bb30 con 0x55b27a2e4b60 2024-03-26T06:38:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.770+0000 7f3546ffd640 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b27a2e4b60 0x55b27a2e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.771+0000 7f3546ffd640 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b27a2e4b60 0x55b27a2e6fc0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8293cd26e0b6521c server_cookie=ce3e95f024d48b6b in_seq=0 out_seq=0 2024-03-26T06:38:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.771+0000 7f352aff5640 10 client.26125 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.772+0000 7f352aff5640 1 -- 172.21.15.37:0/89754998 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b27a22bb30 con 0x55b27a2e4b60 2024-03-26T06:38:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.772+0000 7f352aff5640 10 client.26125 handle_command_reply: tid=0 2024-03-26T06:38:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.772+0000 7f3550381740 1 client.26125 shutdown 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.772+0000 7f3550381740 1 -- 172.21.15.37:0/89754998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2bad40 msgr2=0x55b27a2cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.772+0000 7f3550381740 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2bad40 0x55b27a2cb790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f353c0093e0 tx=0x7f353c009370 comp rx=0 tx=0).stop 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.772+0000 7f3550381740 1 -- 172.21.15.37:0/89754998 shutdown_connections 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.772+0000 7f3550381740 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b27a2e4b60 0x55b27a2e6fc0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.772+0000 7f3550381740 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2cbd60 0x55b27a2c0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.772+0000 7f3550381740 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2bad40 0x55b27a2cb790 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.772+0000 7f3550381740 1 --2- 172.21.15.37:0/89754998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2b9fd0 0x55b27a2c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.772+0000 7f3550381740 1 -- 172.21.15.37:0/89754998 >> 172.21.15.37:0/89754998 conn(0x55b27a2815b0 msgr2=0x55b27a2b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.773+0000 7f352aff5640 0 client.26125 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.773+0000 7f3550381740 1 -- 172.21.15.37:0/89754998 shutdown_connections 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.773+0000 7f3550381740 1 -- 172.21.15.37:0/89754998 wait complete. 2024-03-26T06:38:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.773+0000 7f3550381740 20 client.26125 trim_cache size 0 max 16384 2024-03-26T06:38:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f3550381740 1 Processor -- start 2024-03-26T06:38:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f3550381740 1 -- start start 2024-03-26T06:38:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f3550381740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f3550381740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2bad40 0x55b27a2c5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f3550381740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2cbd60 0x55b27a2c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f3546ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f3550381740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35481b3090 con 0x55b27a2cbd60 2024-03-26T06:38:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f3550381740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35481b4bc0 con 0x55b27a2bad40 2024-03-26T06:38:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f3546ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:43198/0 (socket says 172.21.15.37:43198) 2024-03-26T06:38:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f3550381740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35481b37b0 con 0x55b27a2b9fd0 2024-03-26T06:38:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f3546ffd640 1 -- 172.21.15.37:0/1296929920 learned_addr learned my addr 172.21.15.37:0/1296929920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.801+0000 7f35477fe640 1 --2- 172.21.15.37:0/1296929920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2cbd60 0x55b27a2c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.802+0000 7f35467fc640 1 --2- 172.21.15.37:0/1296929920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2bad40 0x55b27a2c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.802+0000 7f3546ffd640 1 -- 172.21.15.37:0/1296929920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2bad40 msgr2=0x55b27a2c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.802+0000 7f3546ffd640 1 --2- 172.21.15.37:0/1296929920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2bad40 0x55b27a2c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.802+0000 7f3546ffd640 1 -- 172.21.15.37:0/1296929920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2cbd60 msgr2=0x55b27a2c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.802+0000 7f3546ffd640 1 --2- 172.21.15.37:0/1296929920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2cbd60 0x55b27a2c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.802+0000 7f3546ffd640 1 -- 172.21.15.37:0/1296929920 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b27a2bf3c0 con 0x55b27a2b9fd0 2024-03-26T06:38:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.802+0000 7f3546ffd640 1 --2- 172.21.15.37:0/1296929920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2cb540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f353801a000 tx=0x7f3538005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3c15fc7c9ec81ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.802+0000 7f352cff9640 1 -- 172.21.15.37:0/1296929920 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538005100 con 0x55b27a2b9fd0 2024-03-26T06:38:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.803+0000 7f352cff9640 1 -- 172.21.15.37:0/1296929920 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f353800f5e0 con 0x55b27a2b9fd0 2024-03-26T06:38:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.803+0000 7f352cff9640 1 -- 172.21.15.37:0/1296929920 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538003810 con 0x55b27a2b9fd0 2024-03-26T06:38:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.804+0000 7f3550381740 1 -- 172.21.15.37:0/1296929920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 msgr2=0x55b27a2cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.804+0000 7f3550381740 1 --2- 172.21.15.37:0/1296929920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2cb540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f353801a000 tx=0x7f3538005fd0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.804+0000 7f3550381740 1 -- 172.21.15.37:0/1296929920 shutdown_connections 2024-03-26T06:38:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.804+0000 7f3550381740 1 --2- 172.21.15.37:0/1296929920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2cbd60 0x55b27a2c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.804+0000 7f3550381740 1 --2- 172.21.15.37:0/1296929920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2bad40 0x55b27a2c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.804+0000 7f3550381740 1 --2- 172.21.15.37:0/1296929920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2b9fd0 0x55b27a2cb540 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f353801a000 tx=0x7f3538005fd0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.804+0000 7f3550381740 1 -- 172.21.15.37:0/1296929920 >> 172.21.15.37:0/1296929920 conn(0x55b27a2815b0 msgr2=0x55b27a2b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.804+0000 7f35467fc640 1 --2- 172.21.15.37:0/1296929920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2bad40 0x55b27a2c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:38:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.805+0000 7f3550381740 1 -- 172.21.15.37:0/1296929920 shutdown_connections 2024-03-26T06:38:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.805+0000 7f3550381740 1 -- 172.21.15.37:0/1296929920 wait complete. 2024-03-26T06:38:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.805+0000 7f3550381740 1 -- start start 2024-03-26T06:38:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.805+0000 7f3550381740 1 Processor -- start 2024-03-26T06:38:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.806+0000 7f3550381740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2cbd60 0x55b27a2cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.806+0000 7f3550381740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2c7a80 0x55b27a2c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.806+0000 7f3550381740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2c8550 0x55b27a33b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.806+0000 7f3550381740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35481b4560 con 0x55b27a2c8550 2024-03-26T06:38:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.806+0000 7f3550381740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35481b3e40 con 0x55b27a2c7a80 2024-03-26T06:38:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.806+0000 7f3550381740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35481b1420 con 0x55b27a2cbd60 2024-03-26T06:38:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.806+0000 7f3550381740 10 client.0 _ll_register_callbacks cb 0x7f3538009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:38:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.806+0000 7f3546ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2cbd60 0x55b27a2cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.806+0000 7f35467fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2c7a80 0x55b27a2c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.806+0000 7f35467fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2c7a80 0x55b27a2c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:33882/0 (socket says 172.21.15.37:33882) 2024-03-26T06:38:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.806+0000 7f35467fc640 1 -- 172.21.15.37:0/2717706643 learned_addr learned my addr 172.21.15.37:0/2717706643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.807+0000 7f35467fc640 1 -- 172.21.15.37:0/2717706643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2cbd60 msgr2=0x55b27a2cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.807+0000 7f35467fc640 1 --2- 172.21.15.37:0/2717706643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2cbd60 0x55b27a2cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.807+0000 7f35467fc640 1 -- 172.21.15.37:0/2717706643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2c8550 msgr2=0x55b27a33b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.807+0000 7f35467fc640 1 --2- 172.21.15.37:0/2717706643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2c8550 0x55b27a33b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.807+0000 7f35467fc640 1 -- 172.21.15.37:0/2717706643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b27a33b720 con 0x55b27a2c7a80 2024-03-26T06:38:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.807+0000 7f35467fc640 1 --2- 172.21.15.37:0/2717706643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2c7a80 0x55b27a2c7f00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f353c010d00 tx=0x7f353c01ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a654c25c73f83c0b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.807+0000 7f352aff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:38:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.808+0000 7f352aff5640 1 -- 172.21.15.37:0/2717706643 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c0053c0 con 0x55b27a2c7a80 2024-03-26T06:38:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.808+0000 7f352aff5640 1 -- 172.21.15.37:0/2717706643 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f353c005560 con 0x55b27a2c7a80 2024-03-26T06:38:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.808+0000 7f3550381740 1 -- 172.21.15.37:0/2717706643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f353800f5e0 con 0x55b27a2c7a80 2024-03-26T06:38:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.808+0000 7f352aff5640 1 -- 172.21.15.37:0/2717706643 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f353c03e050 con 0x55b27a2c7a80 2024-03-26T06:38:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.808+0000 7f3550381740 10 client.26246 fetch_fsmap learned FSMap version 56 2024-03-26T06:38:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.808+0000 7f3550381740 1 -- 172.21.15.37:0/2717706643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55b27a33ca00 con 0x55b27a2c7a80 2024-03-26T06:38:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.808+0000 7f352aff5640 1 -- 172.21.15.37:0/2717706643 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f353800f5e0 con 0x55b27a2c7a80 2024-03-26T06:38:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.809+0000 7f352aff5640 1 -- 172.21.15.37:0/2717706643 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 56) v1 ==== 7950+0+0 (secure 0 0 0) 0x7f353c031510 con 0x55b27a2c7a80 2024-03-26T06:38:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.809+0000 7f3550381740 10 client.26246 fetch_fsmap finished waiting for FSMap version 56 2024-03-26T06:38:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.809+0000 7f3550381740 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:38:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.809+0000 7f3550381740 20 client.26246 populate_metadata read hostname 'smithi037' 2024-03-26T06:38:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.809+0000 7f3550381740 1 --2- 172.21.15.37:0/2717706643 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b27a33d220 0x55b27a33f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.809+0000 7f3550381740 4 client.26246 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:38:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.809+0000 7f3550381740 1 -- 172.21.15.37:0/2717706643 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b27a225e80 con 0x55b27a33d220 2024-03-26T06:38:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.809+0000 7f3546ffd640 1 --2- 172.21.15.37:0/2717706643 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b27a33d220 0x55b27a33f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.810+0000 7f352aff5640 10 client.26246 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.809+0000 7f3546ffd640 1 --2- 172.21.15.37:0/2717706643 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b27a33d220 0x55b27a33f640 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b139b87c4419cc78 server_cookie=8dc79d1da11be000 in_seq=0 out_seq=0 2024-03-26T06:38:10.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 94682 50.30 MB/sec warmup 105 sec latency 46.306 ms 2024-03-26T06:38:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.987+0000 7f352aff5640 1 -- 172.21.15.37:0/2717706643 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b27a225e80 con 0x55b27a33d220 2024-03-26T06:38:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.987+0000 7f352aff5640 10 client.26246 handle_command_reply: tid=0 2024-03-26T06:38:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.987+0000 7f3550381740 1 client.26246 shutdown 2024-03-26T06:38:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f3550381740 1 -- 172.21.15.37:0/2717706643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2c7a80 msgr2=0x55b27a2c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f3550381740 1 --2- 172.21.15.37:0/2717706643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2c7a80 0x55b27a2c7f00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f353c010d00 tx=0x7f353c01ee80 comp rx=0 tx=0).stop 2024-03-26T06:38:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f3550381740 1 -- 172.21.15.37:0/2717706643 shutdown_connections 2024-03-26T06:38:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f3550381740 1 --2- 172.21.15.37:0/2717706643 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55b27a33d220 0x55b27a33f640 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f3550381740 1 --2- 172.21.15.37:0/2717706643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b27a2c8550 0x55b27a33b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f3550381740 1 --2- 172.21.15.37:0/2717706643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b27a2c7a80 0x55b27a2c7f00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f3550381740 1 --2- 172.21.15.37:0/2717706643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b27a2cbd60 0x55b27a2cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f3550381740 1 -- 172.21.15.37:0/2717706643 >> 172.21.15.37:0/2717706643 conn(0x55b27a2815b0 msgr2=0x55b27a2bf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f352aff5640 0 client.26246 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f3550381740 1 -- 172.21.15.37:0/2717706643 shutdown_connections 2024-03-26T06:38:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f3550381740 1 -- 172.21.15.37:0/2717706643 wait complete. 2024-03-26T06:38:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.988+0000 7f3550381740 20 client.26246 trim_cache size 0 max 16384 2024-03-26T06:38:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:38:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:38:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:38:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:38:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 -- 172.21.15.37:0/3110168063 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35300a2c40 msgr2=0x7f35300a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35300a2c40 0x7f35300a5100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f353c0095f0 tx=0x7f353c005f90 comp rx=0 tx=0).stop 2024-03-26T06:38:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 -- 172.21.15.37:0/3110168063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3548105210 msgr2=0x7f354806c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3548105210 0x7f354806c990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3540009560 tx=0x7f3540035610 comp rx=0 tx=0).stop 2024-03-26T06:38:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 -- 172.21.15.37:0/3110168063 shutdown_connections 2024-03-26T06:38:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35300a2c40 0x7f35300a5100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3548105210 0x7f354806c990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3548100e70 0x7f3548072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 --2- 172.21.15.37:0/3110168063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35481004d0 0x7f3548072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 -- 172.21.15.37:0/3110168063 >> 172.21.15.37:0/3110168063 conn(0x7f35480771d0 msgr2=0x7f3548107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 -- 172.21.15.37:0/3110168063 shutdown_connections 2024-03-26T06:38:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:10.989+0000 7f354d7ab640 1 -- 172.21.15.37:0/3110168063 wait complete. 2024-03-26T06:38:11.848 INFO:tasks.workunit.client.0.smithi037.stdout: 10 95339 50.13 MB/sec warmup 106 sec latency 86.837 ms 2024-03-26T06:38:12.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:12 smithi066 ceph-mon[36504]: pgmap v479: 97 pgs: 97 active+clean; 923 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 26 MiB/s wr, 1.39k op/s 2024-03-26T06:38:12.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:12 smithi149 ceph-mon[36681]: pgmap v479: 97 pgs: 97 active+clean; 923 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 26 MiB/s wr, 1.39k op/s 2024-03-26T06:38:12.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:12 smithi037 ceph-mon[31974]: pgmap v479: 97 pgs: 97 active+clean; 923 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 26 MiB/s wr, 1.39k op/s 2024-03-26T06:38:12.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 96315 50.16 MB/sec warmup 107 sec latency 41.399 ms 2024-03-26T06:38:13.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 97504 50.27 MB/sec warmup 108 sec latency 36.977 ms 2024-03-26T06:38:14.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:14 smithi149 ceph-mon[36681]: pgmap v480: 97 pgs: 97 active+clean; 923 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 26 MiB/s wr, 1.39k op/s 2024-03-26T06:38:14.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:14 smithi037 ceph-mon[31974]: pgmap v480: 97 pgs: 97 active+clean; 923 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 26 MiB/s wr, 1.39k op/s 2024-03-26T06:38:14.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:14 smithi066 ceph-mon[36504]: pgmap v480: 97 pgs: 97 active+clean; 923 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 26 MiB/s wr, 1.39k op/s 2024-03-26T06:38:14.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 98297 50.25 MB/sec warmup 109 sec latency 66.877 ms 2024-03-26T06:38:15.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 99175 50.16 MB/sec warmup 110 sec latency 91.638 ms 2024-03-26T06:38:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:15 smithi066 ceph-mon[36504]: osdmap e109: 12 total, 12 up, 12 in 2024-03-26T06:38:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:15 smithi149 ceph-mon[36681]: osdmap e109: 12 total, 12 up, 12 in 2024-03-26T06:38:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:15 smithi037 ceph-mon[31974]: osdmap e109: 12 total, 12 up, 12 in 2024-03-26T06:38:16.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 100195 50.17 MB/sec warmup 111 sec latency 60.801 ms 2024-03-26T06:38:17.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:16 smithi066 ceph-mon[36504]: pgmap v482: 97 pgs: 97 active+clean; 946 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 26 MiB/s wr, 1.34k op/s 2024-03-26T06:38:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:16 smithi149 ceph-mon[36681]: pgmap v482: 97 pgs: 97 active+clean; 946 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 26 MiB/s wr, 1.34k op/s 2024-03-26T06:38:17.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:16 smithi037 ceph-mon[31974]: pgmap v482: 97 pgs: 97 active+clean; 946 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 26 MiB/s wr, 1.34k op/s 2024-03-26T06:38:17.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 101299 50.34 MB/sec warmup 112 sec latency 33.218 ms 2024-03-26T06:38:18.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 102128 50.25 MB/sec warmup 113 sec latency 59.572 ms 2024-03-26T06:38:19.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:18 smithi066 ceph-mon[36504]: pgmap v483: 97 pgs: 97 active+clean; 951 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.29k op/s 2024-03-26T06:38:19.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:18 smithi149 ceph-mon[36681]: pgmap v483: 97 pgs: 97 active+clean; 951 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.29k op/s 2024-03-26T06:38:19.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:18 smithi037 ceph-mon[31974]: pgmap v483: 97 pgs: 97 active+clean; 951 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.29k op/s 2024-03-26T06:38:19.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 103000 50.21 MB/sec warmup 114 sec latency 118.335 ms 2024-03-26T06:38:20.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 103867 50.23 MB/sec warmup 115 sec latency 73.884 ms 2024-03-26T06:38:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:20 smithi066 ceph-mon[36504]: pgmap v484: 97 pgs: 97 active+clean; 951 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.29k op/s 2024-03-26T06:38:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:20 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:38:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:20 smithi149 ceph-mon[36681]: pgmap v484: 97 pgs: 97 active+clean; 951 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.29k op/s 2024-03-26T06:38:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:20 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:38:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:20 smithi037 ceph-mon[31974]: pgmap v484: 97 pgs: 97 active+clean; 951 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.29k op/s 2024-03-26T06:38:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:20 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:38:21.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 104776 50.17 MB/sec warmup 116 sec latency 60.165 ms 2024-03-26T06:38:22.126 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:21 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:22.126 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:21 smithi066 ceph-mon[36504]: pgmap v485: 97 pgs: 97 active+clean; 999 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 10 MiB/s wr, 852 op/s 2024-03-26T06:38:22.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:21 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:22.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:21 smithi149 ceph-mon[36681]: pgmap v485: 97 pgs: 97 active+clean; 999 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 10 MiB/s wr, 852 op/s 2024-03-26T06:38:22.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:21 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:22.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:21 smithi037 ceph-mon[31974]: pgmap v485: 97 pgs: 97 active+clean; 999 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 10 MiB/s wr, 852 op/s 2024-03-26T06:38:22.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 105601 50.15 MB/sec warmup 117 sec latency 73.474 ms 2024-03-26T06:38:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:22 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:22 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:38:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:22 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:38:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:22 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:38:23.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:22 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:23.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:22 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:38:23.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:22 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:38:23.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:22 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:38:23.527 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:22 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:23.527 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:22 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:38:23.528 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:22 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:38:23.528 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:22 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:38:23.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 106566 50.11 MB/sec warmup 118 sec latency 63.327 ms 2024-03-26T06:38:24.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:23 smithi066 ceph-mon[36504]: pgmap v486: 97 pgs: 97 active+clean; 999 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 10 MiB/s wr, 852 op/s 2024-03-26T06:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.115+0000 7f282848c640 1 -- 172.21.15.37:0/2454569367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2820105510 msgr2=0x7f282010d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.115+0000 7f282848c640 1 --2- 172.21.15.37:0/2454569367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2820105510 0x7f282010d120 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2818009ca0 tx=0x7f281802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.115+0000 7f282848c640 1 -- 172.21.15.37:0/2454569367 shutdown_connections 2024-03-26T06:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.115+0000 7f282848c640 1 --2- 172.21.15.37:0/2454569367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2820105510 0x7f282010d120 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.115+0000 7f282848c640 1 --2- 172.21.15.37:0/2454569367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2820101520 0x7f2820104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.115+0000 7f282848c640 1 --2- 172.21.15.37:0/2454569367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2820100b70 0x7f2820100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.115+0000 7f282848c640 1 -- 172.21.15.37:0/2454569367 >> 172.21.15.37:0/2454569367 conn(0x7f2820076780 msgr2=0x7f28200fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.115+0000 7f282848c640 1 -- 172.21.15.37:0/2454569367 shutdown_connections 2024-03-26T06:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.115+0000 7f282848c640 1 -- 172.21.15.37:0/2454569367 wait complete. 2024-03-26T06:38:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f282848c640 1 Processor -- start 2024-03-26T06:38:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f282848c640 1 -- start start 2024-03-26T06:38:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f282848c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2820100b70 0x7f28200fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f282848c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2820105510 0x7f28200fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f282848c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28200fc640 0x7f28201a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f282848c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2820110600 con 0x7f2820100b70 2024-03-26T06:38:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f282848c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2820110480 con 0x7f28200fc640 2024-03-26T06:38:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f282848c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2820110780 con 0x7f2820105510 2024-03-26T06:38:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f2826a29640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28200fc640 0x7f28201a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f2826228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2820100b70 0x7f28200fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f2826a29640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28200fc640 0x7f28201a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:56044/0 (socket says 172.21.15.37:56044) 2024-03-26T06:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f2826228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2820100b70 0x7f28200fbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35042/0 (socket says 172.21.15.37:35042) 2024-03-26T06:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.116+0000 7f2826a29640 1 -- 172.21.15.37:0/1865007572 learned_addr learned my addr 172.21.15.37:0/1865007572 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.117+0000 7f2825a27640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2820105510 0x7f28200fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.117+0000 7f2826228640 1 -- 172.21.15.37:0/1865007572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2820105510 msgr2=0x7f28200fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.117+0000 7f2826228640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2820105510 0x7f28200fc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.117+0000 7f2826228640 1 -- 172.21.15.37:0/1865007572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28200fc640 msgr2=0x7f28201a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.117+0000 7f2826228640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28200fc640 0x7f28201a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.117+0000 7f2826228640 1 -- 172.21.15.37:0/1865007572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28201a6420 con 0x7f2820100b70 2024-03-26T06:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.117+0000 7f2826228640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2820100b70 0x7f28200fbbc0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f281400e9f0 tx=0x7f281400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.118+0000 7f28137fe640 1 -- 172.21.15.37:0/1865007572 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281400ccc0 con 0x7f2820100b70 2024-03-26T06:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.118+0000 7f28137fe640 1 -- 172.21.15.37:0/1865007572 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2814005020 con 0x7f2820100b70 2024-03-26T06:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.118+0000 7f28137fe640 1 -- 172.21.15.37:0/1865007572 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28140049e0 con 0x7f2820100b70 2024-03-26T06:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.119+0000 7f282848c640 1 -- 172.21.15.37:0/1865007572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28201a6710 con 0x7f2820100b70 2024-03-26T06:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.119+0000 7f282848c640 1 -- 172.21.15.37:0/1865007572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28201a6be0 con 0x7f2820100b70 2024-03-26T06:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.122+0000 7f28137fe640 1 -- 172.21.15.37:0/1865007572 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2814005560 con 0x7f2820100b70 2024-03-26T06:38:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.123+0000 7f282b073740 1 Processor -- start 2024-03-26T06:38:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.123+0000 7f28137fe640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f280c0a2940 0x7f280c0a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f28137fe640 1 -- 172.21.15.37:0/1865007572 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f28140cf2d0 con 0x7f2820100b70 2024-03-26T06:38:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f2826a29640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f280c0a2940 0x7f280c0a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f282b073740 1 -- start start 2024-03-26T06:38:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f282b073740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091298d40 0x560091297a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f282b073740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 0x560091298450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f282b073740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600912a1550 0x5600912a3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f282b073740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600911fc5f0 con 0x560091298d40 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f282b073740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600911a5930 con 0x560091297fd0 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f282b073740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600911f8690 con 0x5600912a1550 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f2826228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091298d40 0x560091297a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f2826228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091298d40 0x560091297a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35058/0 (socket says 172.21.15.37:35058) 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f2826228640 1 -- 172.21.15.37:0/1554760137 learned_addr learned my addr 172.21.15.37:0/1554760137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.124+0000 7f2825a27640 1 --2- 172.21.15.37:0/1554760137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 0x560091298450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2826a29640 1 --2- 172.21.15.37:0/1554760137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600912a1550 0x5600912a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2826a29640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f280c0a2940 0x7f280c0a4e00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2818009a60 tx=0x7f2818005c50 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2826228640 1 -- 172.21.15.37:0/1554760137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600912a1550 msgr2=0x5600912a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2826228640 1 --2- 172.21.15.37:0/1554760137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600912a1550 0x5600912a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2826228640 1 -- 172.21.15.37:0/1554760137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 msgr2=0x560091298450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2826228640 1 --2- 172.21.15.37:0/1554760137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 0x560091298450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2826228640 1 -- 172.21.15.37:0/1554760137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560090f11210 con 0x560091298d40 2024-03-26T06:38:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2825a27640 1 --2- 172.21.15.37:0/1554760137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 0x560091298450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:38:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2826228640 1 --2- 172.21.15.37:0/1554760137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091298d40 0x560091297a90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2814098300 tx=0x7f281409ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8088c4072414c4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2810bfe640 1 -- 172.21.15.37:0/1554760137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28140ce040 con 0x560091298d40 2024-03-26T06:38:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2810bfe640 1 -- 172.21.15.37:0/1554760137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2814099c30 con 0x560091298d40 2024-03-26T06:38:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.125+0000 7f2810bfe640 1 -- 172.21.15.37:0/1554760137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28140862d0 con 0x560091298d40 2024-03-26T06:38:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.126+0000 7f282b073740 1 -- 172.21.15.37:0/1554760137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091298d40 msgr2=0x560091297a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.126+0000 7f282b073740 1 --2- 172.21.15.37:0/1554760137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091298d40 0x560091297a90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2814098300 tx=0x7f281409ba70 comp rx=0 tx=0).stop 2024-03-26T06:38:24.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.126+0000 7f282b073740 1 -- 172.21.15.37:0/1554760137 shutdown_connections 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.126+0000 7f282b073740 1 --2- 172.21.15.37:0/1554760137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600912a1550 0x5600912a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.126+0000 7f282b073740 1 --2- 172.21.15.37:0/1554760137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 0x560091298450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.126+0000 7f282b073740 1 --2- 172.21.15.37:0/1554760137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091298d40 0x560091297a90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.126+0000 7f282b073740 1 -- 172.21.15.37:0/1554760137 >> 172.21.15.37:0/1554760137 conn(0x56009125f5b0 msgr2=0x560091292cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.127+0000 7f282b073740 1 -- 172.21.15.37:0/1554760137 shutdown_connections 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.127+0000 7f282b073740 1 -- 172.21.15.37:0/1554760137 wait complete. 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.127+0000 7f282b073740 1 -- start start 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.127+0000 7f282b073740 1 Processor -- start 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.127+0000 7f282b073740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 0x56009129e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.127+0000 7f282b073740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560091298d40 0x56009129eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.127+0000 7f282b073740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 0x56009129f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.127+0000 7f282b073740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600912a4220 con 0x5600912a1550 2024-03-26T06:38:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f282b073740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600911b8d90 con 0x560091297fd0 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f282b073740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600912a43a0 con 0x560091298d40 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2826228640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 0x56009129e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2826228640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 0x56009129e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:56066/0 (socket says 172.21.15.37:56066) 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2825a27640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560091298d40 0x56009129eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2826228640 1 -- 172.21.15.37:0/1546718813 learned_addr learned my addr 172.21.15.37:0/1546718813 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2826a29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 0x56009129f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f282b073740 10 client.0 _ll_register_callbacks cb 0x560090f10a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2826a29640 1 -- 172.21.15.37:0/1546718813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560091298d40 msgr2=0x56009129eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2826a29640 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560091298d40 0x56009129eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2826a29640 1 -- 172.21.15.37:0/1546718813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 msgr2=0x56009129e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2826a29640 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 0x56009129e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2826a29640 1 -- 172.21.15.37:0/1546718813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560090fdd620 con 0x5600912a1550 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2826228640 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 0x56009129e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:38:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2826a29640 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 0x56009129f0e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2818006010 tx=0x7f281802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c27b5519568bc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f2809ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.128+0000 7f282b073740 1 -- 172.21.15.37:0/1546718813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2814099c30 con 0x5600912a1550 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.129+0000 7f2809ffb640 1 -- 172.21.15.37:0/1546718813 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818059050 con 0x5600912a1550 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.129+0000 7f2809ffb640 1 -- 172.21.15.37:0/1546718813 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2818003d30 con 0x5600912a1550 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.129+0000 7f2809ffb640 1 -- 172.21.15.37:0/1546718813 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f281804f070 con 0x5600912a1550 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.129+0000 7f282b073740 10 client.16752 fetch_fsmap learned FSMap version 58 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.129+0000 7f282b073740 1 -- 172.21.15.37:0/1546718813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=58}) v3 -- 0x5600912b04d0 con 0x5600912a1550 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.129+0000 7f2809ffb640 1 -- 172.21.15.37:0/1546718813 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 58) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2818004040 con 0x5600912a1550 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.129+0000 7f282b073740 10 client.16752 fetch_fsmap finished waiting for FSMap version 58 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.130+0000 7f282b073740 10 client.16752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.130+0000 7f282b073740 20 client.16752 populate_metadata read hostname 'smithi037' 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.130+0000 7f282b073740 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600912b1db0 0x5600912b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.130+0000 7f282b073740 4 client.16752 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:38:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.130+0000 7f282b073740 1 -- 172.21.15.37:0/1546718813 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2814099c30 con 0x5600912b1db0 2024-03-26T06:38:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.130+0000 7f2826228640 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600912b1db0 0x5600912b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.130+0000 7f2826228640 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600912b1db0 0x5600912b4270 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2937e5a184d5861 server_cookie=e97e85a2bcf88a1d in_seq=0 out_seq=0 2024-03-26T06:38:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.130+0000 7f2809ffb640 10 client.16752 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.133+0000 7f2809ffb640 1 -- 172.21.15.37:0/1546718813 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f2814099c30 con 0x5600912b1db0 2024-03-26T06:38:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.133+0000 7f2809ffb640 10 client.16752 handle_command_reply: tid=0 2024-03-26T06:38:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.133+0000 7f282b073740 1 client.16752 shutdown 2024-03-26T06:38:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.133+0000 7f282b073740 1 -- 172.21.15.37:0/1546718813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 msgr2=0x56009129f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.133+0000 7f282b073740 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 0x56009129f0e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2818006010 tx=0x7f281802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.134+0000 7f282b073740 1 -- 172.21.15.37:0/1546718813 shutdown_connections 2024-03-26T06:38:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.134+0000 7f282b073740 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600912b1db0 0x5600912b4270 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.134+0000 7f282b073740 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 0x56009129f0e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.134+0000 7f2809ffb640 0 client.16752 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.134+0000 7f282b073740 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560091298d40 0x56009129eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.134+0000 7f282b073740 1 --2- 172.21.15.37:0/1546718813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091297fd0 0x56009129e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.134+0000 7f282b073740 1 -- 172.21.15.37:0/1546718813 >> 172.21.15.37:0/1546718813 conn(0x56009125f5b0 msgr2=0x560091294a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.134+0000 7f282b073740 1 -- 172.21.15.37:0/1546718813 shutdown_connections 2024-03-26T06:38:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.134+0000 7f282b073740 1 -- 172.21.15.37:0/1546718813 wait complete. 2024-03-26T06:38:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.134+0000 7f282b073740 20 client.16752 trim_cache size 0 max 16384 2024-03-26T06:38:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.153+0000 7f282b073740 1 Processor -- start 2024-03-26T06:38:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.153+0000 7f282b073740 1 -- start start 2024-03-26T06:38:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.153+0000 7f282b073740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091297fd0 0x5600912a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.153+0000 7f282b073740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 0x5600912a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.153+0000 7f282b073740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600912a1550 0x5600912a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f282b073740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201aa770 con 0x560091297fd0 2024-03-26T06:38:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f282b073740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201ac2a0 con 0x560091298d40 2024-03-26T06:38:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f282b073740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201aae90 con 0x5600912a1550 2024-03-26T06:38:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2826228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091297fd0 0x5600912a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2826228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091297fd0 0x5600912a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35070/0 (socket says 172.21.15.37:35070) 2024-03-26T06:38:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2826228640 1 -- 172.21.15.37:0/278025019 learned_addr learned my addr 172.21.15.37:0/278025019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2826a29640 1 --2- 172.21.15.37:0/278025019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600912a1550 0x5600912a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2825a27640 1 --2- 172.21.15.37:0/278025019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 0x5600912a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2826a29640 1 -- 172.21.15.37:0/278025019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 msgr2=0x5600912a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2826a29640 1 --2- 172.21.15.37:0/278025019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 0x5600912a8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2826a29640 1 -- 172.21.15.37:0/278025019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091297fd0 msgr2=0x5600912a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2826a29640 1 --2- 172.21.15.37:0/278025019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091297fd0 0x5600912a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2826a29640 1 -- 172.21.15.37:0/278025019 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600912972a0 con 0x5600912a1550 2024-03-26T06:38:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2826228640 1 --2- 172.21.15.37:0/278025019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091297fd0 0x5600912a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:38:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.154+0000 7f2826a29640 1 --2- 172.21.15.37:0/278025019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600912a1550 0x5600912a8b50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2818037040 tx=0x7f281804b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62cf6ce1b4ae8b48 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.155+0000 7f280bfff640 1 -- 172.21.15.37:0/278025019 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818059070 con 0x5600912a1550 2024-03-26T06:38:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.155+0000 7f280bfff640 1 -- 172.21.15.37:0/278025019 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2818044030 con 0x5600912a1550 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.155+0000 7f280bfff640 1 -- 172.21.15.37:0/278025019 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818039040 con 0x5600912a1550 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.155+0000 7f282b073740 1 -- 172.21.15.37:0/278025019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600912a1550 msgr2=0x5600912a8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.155+0000 7f282b073740 1 --2- 172.21.15.37:0/278025019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600912a1550 0x5600912a8b50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2818037040 tx=0x7f281804b000 comp rx=0 tx=0).stop 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.155+0000 7f282b073740 1 -- 172.21.15.37:0/278025019 shutdown_connections 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.155+0000 7f282b073740 1 --2- 172.21.15.37:0/278025019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600912a1550 0x5600912a8b50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.156+0000 7f282b073740 1 --2- 172.21.15.37:0/278025019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 0x5600912a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.156+0000 7f282b073740 1 --2- 172.21.15.37:0/278025019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560091297fd0 0x5600912a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.156+0000 7f282b073740 1 -- 172.21.15.37:0/278025019 >> 172.21.15.37:0/278025019 conn(0x56009125f5b0 msgr2=0x5600912927f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.156+0000 7f282b073740 1 -- 172.21.15.37:0/278025019 shutdown_connections 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.156+0000 7f282b073740 1 -- 172.21.15.37:0/278025019 wait complete. 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.156+0000 7f282b073740 1 -- start start 2024-03-26T06:38:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.156+0000 7f282b073740 1 Processor -- start 2024-03-26T06:38:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f282b073740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560091297fd0 0x5600912a8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f282b073740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 0x5600912a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f282b073740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 0x5600912a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f282b073740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201abc40 con 0x5600912a1550 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f282b073740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201ab520 con 0x560091298d40 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f282b073740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201a8ac0 con 0x560091297fd0 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f282b073740 10 client.0 _ll_register_callbacks cb 0x7f2814097bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f2825a27640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 0x5600912a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f2825a27640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 0x5600912a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:56086/0 (socket says 172.21.15.37:56086) 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f2825a27640 1 -- 172.21.15.37:0/2656235522 learned_addr learned my addr 172.21.15.37:0/2656235522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f2826a29640 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 0x5600912a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f2826228640 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560091297fd0 0x5600912a8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f2825a27640 1 -- 172.21.15.37:0/2656235522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560091297fd0 msgr2=0x5600912a8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f2825a27640 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560091297fd0 0x5600912a8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f2825a27640 1 -- 172.21.15.37:0/2656235522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 msgr2=0x5600912a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f2825a27640 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 0x5600912a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.157+0000 7f2825a27640 1 -- 172.21.15.37:0/2656235522 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600912a4390 con 0x560091298d40 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.158+0000 7f2826a29640 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 0x5600912a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.158+0000 7f2825a27640 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 0x5600912a4bd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28200fd600 tx=0x7f281c00fb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac4b9e2a800b7fb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.158+0000 7f2809ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.158+0000 7f2809ffb640 1 -- 172.21.15.37:0/2656235522 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c02a030 con 0x560091298d40 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.158+0000 7f282b073740 1 -- 172.21.15.37:0/2656235522 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2818044030 con 0x560091298d40 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.158+0000 7f2809ffb640 1 -- 172.21.15.37:0/2656235522 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f281c004020 con 0x560091298d40 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.158+0000 7f2809ffb640 1 -- 172.21.15.37:0/2656235522 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f281c033050 con 0x560091298d40 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.158+0000 7f282b073740 10 client.26264 fetch_fsmap learned FSMap version 58 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.158+0000 7f282b073740 1 -- 172.21.15.37:0/2656235522 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5600913090f0 con 0x560091298d40 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.159+0000 7f2809ffb640 1 -- 172.21.15.37:0/2656235522 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2818044030 con 0x560091298d40 2024-03-26T06:38:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.159+0000 7f2809ffb640 1 -- 172.21.15.37:0/2656235522 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 58) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f281c004c90 con 0x560091298d40 2024-03-26T06:38:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.159+0000 7f282b073740 10 client.26264 fetch_fsmap finished waiting for FSMap version 58 2024-03-26T06:38:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.159+0000 7f282b073740 10 client.26264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:38:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.159+0000 7f282b073740 20 client.26264 populate_metadata read hostname 'smithi037' 2024-03-26T06:38:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.159+0000 7f282b073740 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600912b1db0 0x56009130b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.159+0000 7f282b073740 4 client.26264 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:38:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.159+0000 7f282b073740 1 -- 172.21.15.37:0/2656235522 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560091209b30 con 0x5600912b1db0 2024-03-26T06:38:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.159+0000 7f2826228640 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600912b1db0 0x56009130b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.160+0000 7f2826228640 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600912b1db0 0x56009130b980 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792ad469f115692c server_cookie=69921dd54982531a in_seq=0 out_seq=0 2024-03-26T06:38:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.160+0000 7f2809ffb640 10 client.26264 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:24.165 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:23 smithi149 ceph-mon[36681]: pgmap v486: 97 pgs: 97 active+clean; 999 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 10 MiB/s wr, 852 op/s 2024-03-26T06:38:24.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:23 smithi037 ceph-mon[31974]: pgmap v486: 97 pgs: 97 active+clean; 999 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 10 MiB/s wr, 852 op/s 2024-03-26T06:38:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.404+0000 7f2809ffb640 1 -- 172.21.15.37:0/2656235522 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560091209b30 con 0x5600912b1db0 2024-03-26T06:38:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f2809ffb640 10 client.26264 handle_command_reply: tid=0 2024-03-26T06:38:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 1 client.26264 shutdown 2024-03-26T06:38:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 1 -- 172.21.15.37:0/2656235522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 msgr2=0x5600912a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 0x5600912a4bd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28200fd600 tx=0x7f281c00fb40 comp rx=0 tx=0).stop 2024-03-26T06:38:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 1 -- 172.21.15.37:0/2656235522 shutdown_connections 2024-03-26T06:38:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5600912b1db0 0x56009130b980 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600912a1550 0x5600912a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f2809ffb640 0 client.26264 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560091298d40 0x5600912a4bd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 1 --2- 172.21.15.37:0/2656235522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560091297fd0 0x5600912a8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 1 -- 172.21.15.37:0/2656235522 >> 172.21.15.37:0/2656235522 conn(0x56009125f5b0 msgr2=0x56009129cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 1 -- 172.21.15.37:0/2656235522 shutdown_connections 2024-03-26T06:38:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 1 -- 172.21.15.37:0/2656235522 wait complete. 2024-03-26T06:38:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.405+0000 7f282b073740 20 client.26264 trim_cache size 0 max 16384 2024-03-26T06:38:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:38:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:38:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:38:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.406+0000 7f282848c640 1 -- 172.21.15.37:0/1865007572 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f280c0a2940 msgr2=0x7f280c0a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.406+0000 7f282848c640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f280c0a2940 0x7f280c0a4e00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2818009a60 tx=0x7f2818005c50 comp rx=0 tx=0).stop 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.406+0000 7f282848c640 1 -- 172.21.15.37:0/1865007572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2820100b70 msgr2=0x7f28200fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.406+0000 7f282848c640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2820100b70 0x7f28200fbbc0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f281400e9f0 tx=0x7f281400eeb0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.407+0000 7f282848c640 1 -- 172.21.15.37:0/1865007572 shutdown_connections 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.407+0000 7f282848c640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f280c0a2940 0x7f280c0a4e00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.407+0000 7f282848c640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28200fc640 0x7f28201a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.407+0000 7f282848c640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2820105510 0x7f28200fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.407+0000 7f282848c640 1 --2- 172.21.15.37:0/1865007572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2820100b70 0x7f28200fbbc0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.407+0000 7f282848c640 1 -- 172.21.15.37:0/1865007572 >> 172.21.15.37:0/1865007572 conn(0x7f2820076780 msgr2=0x7f28200ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.408+0000 7f282848c640 1 -- 172.21.15.37:0/1865007572 shutdown_connections 2024-03-26T06:38:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:24.408+0000 7f282848c640 1 -- 172.21.15.37:0/1865007572 wait complete. 2024-03-26T06:38:24.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 107445 50.16 MB/sec warmup 119 sec latency 131.338 ms 2024-03-26T06:38:25.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:24 smithi066 ceph-mon[36504]: daemon mds.c restarted 2024-03-26T06:38:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:24 smithi149 ceph-mon[36681]: daemon mds.c restarted 2024-03-26T06:38:25.161 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:38:24 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: ignoring --setuser ceph since I am not root 2024-03-26T06:38:25.162 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:38:24 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: ignoring --setgroup ceph since I am not root 2024-03-26T06:38:25.162 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:38:24 smithi149 ceph-mds[67761]: -- 172.21.15.149:0/2934400977 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c3eb25fa40 con 0x55c3eb233000 2024-03-26T06:38:25.162 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:38:24 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: starting mds.c at 2024-03-26T06:38:25.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:24 smithi037 ceph-mon[31974]: daemon mds.c restarted 2024-03-26T06:38:26.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:25 smithi066 ceph-mon[36504]: osdmap e110: 12 total, 12 up, 12 in 2024-03-26T06:38:26.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:25 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6836/2880689237,v1:172.21.15.149:6837/2880689237] up:boot 2024-03-26T06:38:26.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:25 smithi066 ceph-mon[36504]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:38:26.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:25 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:38:26.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:25 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:26.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:25 smithi066 ceph-mon[36504]: pgmap v488: 97 pgs: 97 active+clean; 1008 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.3 MiB/s wr, 759 op/s 2024-03-26T06:38:26.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:25 smithi149 ceph-mon[36681]: osdmap e110: 12 total, 12 up, 12 in 2024-03-26T06:38:26.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:25 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6836/2880689237,v1:172.21.15.149:6837/2880689237] up:boot 2024-03-26T06:38:26.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:25 smithi149 ceph-mon[36681]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:38:26.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:25 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:38:26.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:25 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:26.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:25 smithi149 ceph-mon[36681]: pgmap v488: 97 pgs: 97 active+clean; 1008 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.3 MiB/s wr, 759 op/s 2024-03-26T06:38:26.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:25 smithi037 ceph-mon[31974]: osdmap e110: 12 total, 12 up, 12 in 2024-03-26T06:38:26.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:25 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6836/2880689237,v1:172.21.15.149:6837/2880689237] up:boot 2024-03-26T06:38:26.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:25 smithi037 ceph-mon[31974]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T06:38:26.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:25 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:38:26.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:25 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:26.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:25 smithi037 ceph-mon[31974]: pgmap v488: 97 pgs: 97 active+clean; 1008 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.3 MiB/s wr, 759 op/s 2024-03-26T06:38:26.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 109254 47.22 MB/sec execute 1 sec latency 92.901 ms 2024-03-26T06:38:27.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 110211 53.77 MB/sec execute 2 sec latency 86.234 ms 2024-03-26T06:38:28.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:28 smithi066 ceph-mon[36504]: pgmap v489: 97 pgs: 97 active+clean; 1021 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 10 MiB/s wr, 842 op/s 2024-03-26T06:38:28.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:28 smithi149 ceph-mon[36681]: pgmap v489: 97 pgs: 97 active+clean; 1021 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 10 MiB/s wr, 842 op/s 2024-03-26T06:38:28.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:28 smithi037 ceph-mon[31974]: pgmap v489: 97 pgs: 97 active+clean; 1021 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 10 MiB/s wr, 842 op/s 2024-03-26T06:38:28.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 111089 50.21 MB/sec execute 3 sec latency 56.691 ms 2024-03-26T06:38:29.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 112121 49.65 MB/sec execute 4 sec latency 73.755 ms 2024-03-26T06:38:30.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:30 smithi066 ceph-mon[36504]: pgmap v490: 97 pgs: 97 active+clean; 1021 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 10 MiB/s wr, 842 op/s 2024-03-26T06:38:30.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:30 smithi149 ceph-mon[36681]: pgmap v490: 97 pgs: 97 active+clean; 1021 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 10 MiB/s wr, 842 op/s 2024-03-26T06:38:30.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:30 smithi037 ceph-mon[31974]: pgmap v490: 97 pgs: 97 active+clean; 1021 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 10 MiB/s wr, 842 op/s 2024-03-26T06:38:30.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 113042 52.14 MB/sec execute 5 sec latency 205.972 ms 2024-03-26T06:38:31.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 113206 44.70 MB/sec execute 6 sec latency 815.826 ms 2024-03-26T06:38:32.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:32 smithi066 ceph-mon[36504]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.7 MiB/s wr, 809 op/s 2024-03-26T06:38:32.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:32 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:32.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:32 smithi149 ceph-mon[36681]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.7 MiB/s wr, 809 op/s 2024-03-26T06:38:32.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:32 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:32.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:32 smithi037 ceph-mon[31974]: pgmap v491: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.7 MiB/s wr, 809 op/s 2024-03-26T06:38:32.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:32 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:32.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 113328 39.13 MB/sec execute 7 sec latency 1409.078 ms 2024-03-26T06:38:33.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 113716 37.14 MB/sec execute 8 sec latency 174.361 ms 2024-03-26T06:38:34.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:34 smithi066 ceph-mon[36504]: pgmap v492: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.7 MiB/s wr, 809 op/s 2024-03-26T06:38:34.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:34 smithi149 ceph-mon[36681]: pgmap v492: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.7 MiB/s wr, 809 op/s 2024-03-26T06:38:34.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:34 smithi037 ceph-mon[31974]: pgmap v492: 97 pgs: 97 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.7 MiB/s wr, 809 op/s 2024-03-26T06:38:34.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 114351 36.31 MB/sec execute 9 sec latency 170.313 ms 2024-03-26T06:38:35.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 115247 37.42 MB/sec execute 10 sec latency 148.265 ms 2024-03-26T06:38:36.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:36 smithi066 ceph-mon[36504]: pgmap v493: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-03-26T06:38:36.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:36 smithi149 ceph-mon[36681]: pgmap v493: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-03-26T06:38:36.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:36 smithi037 ceph-mon[31974]: pgmap v493: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-03-26T06:38:36.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 116132 38.85 MB/sec execute 11 sec latency 70.098 ms 2024-03-26T06:38:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.516+0000 7f9c263e1640 1 -- 172.21.15.37:0/2381092834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c20101620 msgr2=0x7f9c2010d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.516+0000 7f9c263e1640 1 --2- 172.21.15.37:0/2381092834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c20101620 0x7f9c2010d110 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009cd0 tx=0x7f9c1802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.516+0000 7f9c263e1640 1 -- 172.21.15.37:0/2381092834 shutdown_connections 2024-03-26T06:38:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.516+0000 7f9c263e1640 1 --2- 172.21.15.37:0/2381092834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c20101620 0x7f9c2010d110 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.516+0000 7f9c263e1640 1 --2- 172.21.15.37:0/2381092834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20100c60 0x7f9c201010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.516+0000 7f9c263e1640 1 --2- 172.21.15.37:0/2381092834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20106350 0x7f9c20106730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.516+0000 7f9c263e1640 1 -- 172.21.15.37:0/2381092834 >> 172.21.15.37:0/2381092834 conn(0x7f9c20075530 msgr2=0x7f9c20075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.516+0000 7f9c263e1640 1 -- 172.21.15.37:0/2381092834 shutdown_connections 2024-03-26T06:38:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.516+0000 7f9c263e1640 1 -- 172.21.15.37:0/2381092834 wait complete. 2024-03-26T06:38:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.517+0000 7f9c263e1640 1 Processor -- start 2024-03-26T06:38:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.517+0000 7f9c263e1640 1 -- start start 2024-03-26T06:38:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.517+0000 7f9c263e1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20100c60 0x7f9c2019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.517+0000 7f9c263e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c20101620 0x7f9c2019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c263e1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20106350 0x7f9c201968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c263e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c20110680 con 0x7f9c20101620 2024-03-26T06:38:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c263e1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c20110500 con 0x7f9c20106350 2024-03-26T06:38:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c263e1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c20110800 con 0x7f9c20100c60 2024-03-26T06:38:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c20101620 0x7f9c2019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20100c60 0x7f9c2019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20100c60 0x7f9c2019c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:43900/0 (socket says 172.21.15.37:43900) 2024-03-26T06:38:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c20101620 0x7f9c2019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33966/0 (socket says 172.21.15.37:33966) 2024-03-26T06:38:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1ffff640 1 -- 172.21.15.37:0/2026346037 learned_addr learned my addr 172.21.15.37:0/2026346037 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c2497e640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20106350 0x7f9c201968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1ffff640 1 -- 172.21.15.37:0/2026346037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20106350 msgr2=0x7f9c201968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1ffff640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20106350 0x7f9c201968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1ffff640 1 -- 172.21.15.37:0/2026346037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c20101620 msgr2=0x7f9c2019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1ffff640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c20101620 0x7f9c2019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1ffff640 1 -- 172.21.15.37:0/2026346037 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c20197060 con 0x7f9c20100c60 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c2497e640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20106350 0x7f9c201968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c20101620 0x7f9c2019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1ffff640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20100c60 0x7f9c2019c730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400eb00 tx=0x7f9c14009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1d7fa640 1 -- 172.21.15.37:0/2026346037 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1401c070 con 0x7f9c20100c60 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1d7fa640 1 -- 172.21.15.37:0/2026346037 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c1400bc30 con 0x7f9c20100c60 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c263e1640 1 -- 172.21.15.37:0/2026346037 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c20197350 con 0x7f9c20100c60 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.518+0000 7f9c1d7fa640 1 -- 172.21.15.37:0/2026346037 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c140166a0 con 0x7f9c20100c60 2024-03-26T06:38:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.519+0000 7f9c263e1640 1 -- 172.21.15.37:0/2026346037 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c200ffb50 con 0x7f9c20100c60 2024-03-26T06:38:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.521+0000 7f9c1d7fa640 1 -- 172.21.15.37:0/2026346037 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f9c1401f070 con 0x7f9c20100c60 2024-03-26T06:38:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.521+0000 7f9c1d7fa640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9c100a2a90 0x7f9c100a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.521+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9c100a2a90 0x7f9c100a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.521+0000 7f9c1d7fa640 1 -- 172.21.15.37:0/2026346037 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f9c140d02f0 con 0x7f9c20100c60 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c28fb6740 1 Processor -- start 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9c100a2a90 0x7f9c100a4f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9c20075ac0 tx=0x7f9c0c005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c28fb6740 1 -- start start 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c28fb6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8ac8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c28fb6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ac9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c28fb6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8ad4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c28fb6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a8a2d5f0 con 0x5597a8ac9d40 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c28fb6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a89d6930 con 0x5597a8ac8fd0 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c28fb6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a8a29690 con 0x5597a8ad2550 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ac9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ac9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:46768/0 (socket says 172.21.15.37:46768) 2024-03-26T06:38:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.522+0000 7f9c1f7fe640 1 -- 172.21.15.37:0/1089218430 learned_addr learned my addr 172.21.15.37:0/1089218430 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.523+0000 7f9c2497e640 1 --2- 172.21.15.37:0/1089218430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8ad4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.523+0000 7f9c1ffff640 1 --2- 172.21.15.37:0/1089218430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8ac8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.523+0000 7f9c1f7fe640 1 -- 172.21.15.37:0/1089218430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 msgr2=0x5597a8ad4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.523+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/1089218430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8ad4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.523+0000 7f9c1f7fe640 1 -- 172.21.15.37:0/1089218430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 msgr2=0x5597a8ac8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.523+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/1089218430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8ac8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.523+0000 7f9c1f7fe640 1 -- 172.21.15.37:0/1089218430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597a8742210 con 0x5597a8ac8fd0 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.523+0000 7f9c1ffff640 1 --2- 172.21.15.37:0/1089218430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8ac8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.523+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/1089218430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ac9450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c004240 tx=0x7f9c0c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4632212bf5586 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.523+0000 7f9c0a7fc640 1 -- 172.21.15.37:0/1089218430 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00f480 con 0x5597a8ac8fd0 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.524+0000 7f9c0a7fc640 1 -- 172.21.15.37:0/1089218430 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c0c00fb00 con 0x5597a8ac8fd0 2024-03-26T06:38:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.524+0000 7f9c0a7fc640 1 -- 172.21.15.37:0/1089218430 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c01eb80 con 0x5597a8ac8fd0 2024-03-26T06:38:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.525+0000 7f9c28fb6740 1 -- 172.21.15.37:0/1089218430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 msgr2=0x5597a8ac9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.525+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/1089218430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ac9450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c004240 tx=0x7f9c0c014000 comp rx=0 tx=0).stop 2024-03-26T06:38:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.525+0000 7f9c28fb6740 1 -- 172.21.15.37:0/1089218430 shutdown_connections 2024-03-26T06:38:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.525+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/1089218430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8ad4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.525+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/1089218430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ac9450 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.525+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/1089218430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8ac8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.525+0000 7f9c28fb6740 1 -- 172.21.15.37:0/1089218430 >> 172.21.15.37:0/1089218430 conn(0x5597a8a905b0 msgr2=0x5597a8ac3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.525+0000 7f9c28fb6740 1 -- 172.21.15.37:0/1089218430 shutdown_connections 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.525+0000 7f9c28fb6740 1 -- 172.21.15.37:0/1089218430 wait complete. 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.525+0000 7f9c28fb6740 1 -- start start 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.526+0000 7f9c28fb6740 1 Processor -- start 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.526+0000 7f9c28fb6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8acf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.526+0000 7f9c28fb6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8acfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c28fb6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8acffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c1ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8acf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c1ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8acf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:46770/0 (socket says 172.21.15.37:46770) 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c1ffff640 1 -- 172.21.15.37:0/396994625 learned_addr learned my addr 172.21.15.37:0/396994625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8acfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c28fb6740 1 -- 172.21.15.37:0/396994625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a8ad52d0 con 0x5597a8ac9d40 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c2497e640 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8acffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c28fb6740 1 -- 172.21.15.37:0/396994625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a89e9d90 con 0x5597a8ac8fd0 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c28fb6740 1 -- 172.21.15.37:0/396994625 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a8ad5450 con 0x5597a8ad2550 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c28fb6740 10 client.0 _ll_register_callbacks cb 0x5597a8741a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c1f7fe640 1 -- 172.21.15.37:0/396994625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 msgr2=0x5597a8acffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8acffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c1f7fe640 1 -- 172.21.15.37:0/396994625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 msgr2=0x5597a8acf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8acf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.527+0000 7f9c1f7fe640 1 -- 172.21.15.37:0/396994625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597a880e620 con 0x5597a8ac9d40 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.528+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8acfa60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c004240 tx=0x7f9c0c00f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f21a63098012e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.528+0000 7f9bff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.528+0000 7f9bff7fe640 1 -- 172.21.15.37:0/396994625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00f960 con 0x5597a8ac9d40 2024-03-26T06:38:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.528+0000 7f9bff7fe640 1 -- 172.21.15.37:0/396994625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f9c0c005280 con 0x5597a8ac9d40 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.528+0000 7f9c28fb6740 1 -- 172.21.15.37:0/396994625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c0c00fb00 con 0x5597a8ac9d40 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.528+0000 7f9bff7fe640 1 -- 172.21.15.37:0/396994625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c0c0054a0 con 0x5597a8ac9d40 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.528+0000 7f9c28fb6740 10 client.16773 fetch_fsmap learned FSMap version 61 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.528+0000 7f9c28fb6740 1 -- 172.21.15.37:0/396994625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5597a8ae12e0 con 0x5597a8ac9d40 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.528+0000 7f9bff7fe640 1 -- 172.21.15.37:0/396994625 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c0c01ece0 con 0x5597a8ac9d40 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.529+0000 7f9bff7fe640 1 -- 172.21.15.37:0/396994625 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 61) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9c0c026400 con 0x5597a8ac9d40 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.529+0000 7f9c28fb6740 10 client.16773 fetch_fsmap finished waiting for FSMap version 61 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.529+0000 7f9c28fb6740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.529+0000 7f9c28fb6740 20 client.16773 populate_metadata read hostname 'smithi037' 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.529+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5597a8ae2690 0x5597a8ae4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.529+0000 7f9c28fb6740 4 client.16773 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:38:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.529+0000 7f9c28fb6740 1 -- 172.21.15.37:0/396994625 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c0c00fb00 con 0x5597a8ae2690 2024-03-26T06:38:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.529+0000 7f9c1ffff640 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5597a8ae2690 0x5597a8ae4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.530+0000 7f9c1ffff640 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5597a8ae2690 0x5597a8ae4a80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fafdcbeb6cba482a server_cookie=3c065511975d8e13 in_seq=0 out_seq=0 2024-03-26T06:38:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.530+0000 7f9bff7fe640 10 client.16773 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.535+0000 7f9bff7fe640 1 -- 172.21.15.37:0/396994625 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f9c0c00fb00 con 0x5597a8ae2690 2024-03-26T06:38:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.535+0000 7f9bff7fe640 10 client.16773 handle_command_reply: tid=0 2024-03-26T06:38:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.535+0000 7f9c28fb6740 1 client.16773 shutdown 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.535+0000 7f9c28fb6740 1 -- 172.21.15.37:0/396994625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 msgr2=0x5597a8acfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.535+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8acfa60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c004240 tx=0x7f9c0c00f6b0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.535+0000 7f9c28fb6740 1 -- 172.21.15.37:0/396994625 shutdown_connections 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.535+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5597a8ae2690 0x5597a8ae4a80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.535+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8acffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.535+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8acfa60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.535+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/396994625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8acf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.535+0000 7f9c28fb6740 1 -- 172.21.15.37:0/396994625 >> 172.21.15.37:0/396994625 conn(0x5597a8a905b0 msgr2=0x5597a8ac5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.536+0000 7f9bff7fe640 0 client.16773 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.536+0000 7f9c28fb6740 1 -- 172.21.15.37:0/396994625 shutdown_connections 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.536+0000 7f9c28fb6740 1 -- 172.21.15.37:0/396994625 wait complete. 2024-03-26T06:38:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.536+0000 7f9c28fb6740 20 client.16773 trim_cache size 0 max 16384 2024-03-26T06:38:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.555+0000 7f9c28fb6740 1 Processor -- start 2024-03-26T06:38:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.555+0000 7f9c28fb6740 1 -- start start 2024-03-26T06:38:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.555+0000 7f9c28fb6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.555+0000 7f9c28fb6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8acddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.555+0000 7f9c28fb6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8ace330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c1ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c1ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:46774/0 (socket says 172.21.15.37:46774) 2024-03-26T06:38:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c1ffff640 1 -- 172.21.15.37:0/4245077328 learned_addr learned my addr 172.21.15.37:0/4245077328 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4245077328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201aaa70 con 0x5597a8ac9d40 2024-03-26T06:38:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4245077328 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201ac5a0 con 0x5597a8ac8fd0 2024-03-26T06:38:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4245077328 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201ab190 con 0x5597a8ad2550 2024-03-26T06:38:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/4245077328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8acddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c2497e640 1 --2- 172.21.15.37:0/4245077328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8ace330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c1f7fe640 1 -- 172.21.15.37:0/4245077328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 msgr2=0x5597a8ace330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/4245077328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8ace330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c1f7fe640 1 -- 172.21.15.37:0/4245077328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 msgr2=0x5597a8ad1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/4245077328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.556+0000 7f9c1f7fe640 1 -- 172.21.15.37:0/4245077328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597a8ac82a0 con 0x5597a8ac9d40 2024-03-26T06:38:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.557+0000 7f9c2497e640 1 --2- 172.21.15.37:0/4245077328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8ace330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:38:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.557+0000 7f9c1ffff640 1 --2- 172.21.15.37:0/4245077328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:38:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.557+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/4245077328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8acddf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c01e730 tx=0x7f9c0c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71302ddd5f6756e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.558+0000 7f9c09ffb640 1 -- 172.21.15.37:0/4245077328 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c03d070 con 0x5597a8ac9d40 2024-03-26T06:38:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.558+0000 7f9c09ffb640 1 -- 172.21.15.37:0/4245077328 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c0c026b40 con 0x5597a8ac9d40 2024-03-26T06:38:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.558+0000 7f9c09ffb640 1 -- 172.21.15.37:0/4245077328 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00f960 con 0x5597a8ac9d40 2024-03-26T06:38:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.559+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4245077328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 msgr2=0x5597a8acddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.559+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/4245077328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8acddf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c01e730 tx=0x7f9c0c011f80 comp rx=0 tx=0).stop 2024-03-26T06:38:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.559+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4245077328 shutdown_connections 2024-03-26T06:38:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.559+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/4245077328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ad2550 0x5597a8ace330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.559+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/4245077328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac9d40 0x5597a8acddf0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.559+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/4245077328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.559+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4245077328 >> 172.21.15.37:0/4245077328 conn(0x5597a8a905b0 msgr2=0x5597a8aca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.559+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4245077328 shutdown_connections 2024-03-26T06:38:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.560+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4245077328 wait complete. 2024-03-26T06:38:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.560+0000 7f9c28fb6740 1 -- start start 2024-03-26T06:38:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.560+0000 7f9c28fb6740 1 Processor -- start 2024-03-26T06:38:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.561+0000 7f9c28fb6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.561+0000 7f9c28fb6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ac9d40 0x5597a8ad9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.561+0000 7f9c1ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.561+0000 7f9c1ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34012/0 (socket says 172.21.15.37:34012) 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.561+0000 7f9c1ffff640 1 -- 172.21.15.37:0/4030898739 learned_addr learned my addr 172.21.15.37:0/4030898739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.561+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ad2550 0x5597a8b30e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.561+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4030898739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201abf40 con 0x5597a8ac8fd0 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.561+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4030898739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201ab820 con 0x5597a8ad2550 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.561+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4030898739 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201a8d10 con 0x5597a8ac9d40 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.562+0000 7f9c28fb6740 10 client.0 _ll_register_callbacks cb 0x7f9c0c009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.562+0000 7f9c1ffff640 1 -- 172.21.15.37:0/4030898739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ac9d40 msgr2=0x5597a8ad9c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.562+0000 7f9c1ffff640 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ac9d40 0x5597a8ad9c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.562+0000 7f9c1ffff640 1 -- 172.21.15.37:0/4030898739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ad2550 msgr2=0x5597a8b30e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.562+0000 7f9c1ffff640 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ad2550 0x5597a8b30e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.562+0000 7f9c1ffff640 1 -- 172.21.15.37:0/4030898739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597a8b31680 con 0x5597a8ac8fd0 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.562+0000 7f9c1ffff640 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad9730 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9c1409c9c0 tx=0x7f9c14098e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11e10981533a5b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.562+0000 7f9bff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:38:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.562+0000 7f9bff7fe640 1 -- 172.21.15.37:0/4030898739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c140b9020 con 0x5597a8ac8fd0 2024-03-26T06:38:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.562+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4030898739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597a8a3ab30 con 0x5597a8ac8fd0 2024-03-26T06:38:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.562+0000 7f9bff7fe640 1 -- 172.21.15.37:0/4030898739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f9c140990a0 con 0x5597a8ac8fd0 2024-03-26T06:38:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.563+0000 7f9bff7fe640 1 -- 172.21.15.37:0/4030898739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c140a5120 con 0x5597a8ac8fd0 2024-03-26T06:38:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.563+0000 7f9c28fb6740 10 client.16779 fetch_fsmap learned FSMap version 61 2024-03-26T06:38:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.563+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4030898739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5597a8b41fa0 con 0x5597a8ac8fd0 2024-03-26T06:38:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.563+0000 7f9bff7fe640 1 -- 172.21.15.37:0/4030898739 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c1409be80 con 0x5597a8ac8fd0 2024-03-26T06:38:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.564+0000 7f9bff7fe640 1 -- 172.21.15.37:0/4030898739 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 61) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9c14098910 con 0x5597a8ac8fd0 2024-03-26T06:38:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.564+0000 7f9c28fb6740 10 client.16779 fetch_fsmap finished waiting for FSMap version 61 2024-03-26T06:38:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.564+0000 7f9c28fb6740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:38:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.564+0000 7f9c28fb6740 20 client.16779 populate_metadata read hostname 'smithi037' 2024-03-26T06:38:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.564+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5597a8b427b0 0x5597a8b44bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.564+0000 7f9c28fb6740 4 client.16779 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:38:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.564+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4030898739 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5597a8a3ab30 con 0x5597a8b427b0 2024-03-26T06:38:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.564+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5597a8b427b0 0x5597a8b44bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.565+0000 7f9c1f7fe640 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5597a8b427b0 0x5597a8b44bd0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14ff0fda97024d8 server_cookie=76c30568203092 in_seq=0 out_seq=0 2024-03-26T06:38:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.565+0000 7f9bff7fe640 10 client.16779 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.814+0000 7f9bff7fe640 1 -- 172.21.15.37:0/4030898739 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5597a8a3ab30 con 0x5597a8b427b0 2024-03-26T06:38:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.814+0000 7f9bff7fe640 10 client.16779 handle_command_reply: tid=0 2024-03-26T06:38:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.814+0000 7f9c28fb6740 1 client.16779 shutdown 2024-03-26T06:38:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.814+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4030898739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac8fd0 msgr2=0x5597a8ad9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.814+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad9730 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9c1409c9c0 tx=0x7f9c14098e00 comp rx=0 tx=0).stop 2024-03-26T06:38:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.815+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4030898739 shutdown_connections 2024-03-26T06:38:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.815+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5597a8b427b0 0x5597a8b44bd0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.815+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5597a8ad2550 0x5597a8b30e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.815+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597a8ac9d40 0x5597a8ad9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.815+0000 7f9c28fb6740 1 --2- 172.21.15.37:0/4030898739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597a8ac8fd0 0x5597a8ad9730 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.815+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4030898739 >> 172.21.15.37:0/4030898739 conn(0x5597a8a905b0 msgr2=0x5597a8acdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.815+0000 7f9bff7fe640 0 client.16779 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.815+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4030898739 shutdown_connections 2024-03-26T06:38:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.815+0000 7f9c28fb6740 1 -- 172.21.15.37:0/4030898739 wait complete. 2024-03-26T06:38:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.815+0000 7f9c28fb6740 20 client.16779 trim_cache size 0 max 16384 2024-03-26T06:38:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:38:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:38:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:38:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:38:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 -- 172.21.15.37:0/2026346037 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9c100a2a90 msgr2=0x7f9c100a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9c100a2a90 0x7f9c100a4f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9c20075ac0 tx=0x7f9c0c005f90 comp rx=0 tx=0).stop 2024-03-26T06:38:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 -- 172.21.15.37:0/2026346037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20100c60 msgr2=0x7f9c2019c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20100c60 0x7f9c2019c730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400eb00 tx=0x7f9c14009f20 comp rx=0 tx=0).stop 2024-03-26T06:38:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 -- 172.21.15.37:0/2026346037 shutdown_connections 2024-03-26T06:38:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f9c100a2a90 0x7f9c100a4f50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20106350 0x7f9c201968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c20101620 0x7f9c2019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 --2- 172.21.15.37:0/2026346037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20100c60 0x7f9c2019c730 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 -- 172.21.15.37:0/2026346037 >> 172.21.15.37:0/2026346037 conn(0x7f9c20075530 msgr2=0x7f9c200fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 -- 172.21.15.37:0/2026346037 shutdown_connections 2024-03-26T06:38:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:37.816+0000 7f9c263e1640 1 -- 172.21.15.37:0/2026346037 wait complete. 2024-03-26T06:38:37.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 116866 38.53 MB/sec execute 12 sec latency 165.423 ms 2024-03-26T06:38:38.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:38 smithi066 ceph-mon[36504]: pgmap v494: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-03-26T06:38:38.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:38 smithi149 ceph-mon[36681]: pgmap v494: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-03-26T06:38:38.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:38 smithi037 ceph-mon[31974]: pgmap v494: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-03-26T06:38:38.849 INFO:tasks.workunit.client.0.smithi037.stdout: 10 117855 39.72 MB/sec execute 13 sec latency 51.795 ms 2024-03-26T06:38:39.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 118758 40.49 MB/sec execute 14 sec latency 57.975 ms 2024-03-26T06:38:40.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:40 smithi066 ceph-mon[36504]: pgmap v495: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.01k op/s 2024-03-26T06:38:40.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:40 smithi149 ceph-mon[36681]: pgmap v495: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.01k op/s 2024-03-26T06:38:40.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:40 smithi037 ceph-mon[31974]: pgmap v495: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.01k op/s 2024-03-26T06:38:40.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 119578 40.55 MB/sec execute 15 sec latency 191.902 ms 2024-03-26T06:38:41.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 120334 40.88 MB/sec execute 16 sec latency 551.151 ms 2024-03-26T06:38:42.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:42 smithi066 ceph-mon[36504]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 25 MiB/s wr, 1.37k op/s 2024-03-26T06:38:42.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:42 smithi149 ceph-mon[36681]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 25 MiB/s wr, 1.37k op/s 2024-03-26T06:38:42.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:42 smithi037 ceph-mon[31974]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 25 MiB/s wr, 1.37k op/s 2024-03-26T06:38:42.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 121158 40.84 MB/sec execute 17 sec latency 98.894 ms 2024-03-26T06:38:43.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 122046 41.60 MB/sec execute 18 sec latency 77.648 ms 2024-03-26T06:38:44.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:44 smithi066 ceph-mon[36504]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 1.05k op/s 2024-03-26T06:38:44.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:44 smithi149 ceph-mon[36681]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 1.05k op/s 2024-03-26T06:38:44.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:44 smithi037 ceph-mon[31974]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 1.05k op/s 2024-03-26T06:38:44.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 123096 41.82 MB/sec execute 19 sec latency 46.453 ms 2024-03-26T06:38:45.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 124106 42.80 MB/sec execute 20 sec latency 50.008 ms 2024-03-26T06:38:46.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:46 smithi066 ceph-mon[36504]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.22k op/s 2024-03-26T06:38:46.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:46 smithi149 ceph-mon[36681]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.22k op/s 2024-03-26T06:38:46.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:46 smithi037 ceph-mon[31974]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.22k op/s 2024-03-26T06:38:46.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 125093 43.27 MB/sec execute 21 sec latency 69.445 ms 2024-03-26T06:38:47.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:47 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:47.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:47 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:47.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:47 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:47.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 126015 43.63 MB/sec execute 22 sec latency 42.727 ms 2024-03-26T06:38:48.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:48 smithi066 ceph-mon[36504]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.05k op/s 2024-03-26T06:38:48.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:48 smithi149 ceph-mon[36681]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.05k op/s 2024-03-26T06:38:48.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:48 smithi037 ceph-mon[31974]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.05k op/s 2024-03-26T06:38:48.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 127013 43.94 MB/sec execute 23 sec latency 53.495 ms 2024-03-26T06:38:49.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 128019 44.61 MB/sec execute 24 sec latency 81.300 ms 2024-03-26T06:38:50.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:50 smithi066 ceph-mon[36504]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-03-26T06:38:50.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:50 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:50.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:50 smithi149 ceph-mon[36681]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-03-26T06:38:50.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:50 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:50.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:50 smithi037 ceph-mon[31974]: pgmap v500: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-03-26T06:38:50.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:50 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:50.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 128915 44.43 MB/sec execute 25 sec latency 109.193 ms 2024-03-26T06:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.933+0000 7fbe26353640 1 -- 172.21.15.37:0/1035481259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe20106670 msgr2=0x7fbe2010e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.933+0000 7fbe26353640 1 --2- 172.21.15.37:0/1035481259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe20106670 0x7fbe2010e1d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009cd0 tx=0x7fbe1802c140 comp rx=0 tx=0).stop 2024-03-26T06:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.934+0000 7fbe26353640 1 -- 172.21.15.37:0/1035481259 shutdown_connections 2024-03-26T06:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.934+0000 7fbe26353640 1 --2- 172.21.15.37:0/1035481259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe20106670 0x7fbe2010e1d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.934+0000 7fbe26353640 1 --2- 172.21.15.37:0/1035481259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe20100410 0x7fbe20105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.934+0000 7fbe26353640 1 --2- 172.21.15.37:0/1035481259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe200ffa60 0x7fbe200ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.934+0000 7fbe26353640 1 -- 172.21.15.37:0/1035481259 >> 172.21.15.37:0/1035481259 conn(0x7fbe20076780 msgr2=0x7fbe200fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.934+0000 7fbe26353640 1 -- 172.21.15.37:0/1035481259 shutdown_connections 2024-03-26T06:38:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.934+0000 7fbe26353640 1 -- 172.21.15.37:0/1035481259 wait complete. 2024-03-26T06:38:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.935+0000 7fbe26353640 1 Processor -- start 2024-03-26T06:38:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.935+0000 7fbe26353640 1 -- start start 2024-03-26T06:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.936+0000 7fbe26353640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe200ffa60 0x7fbe20072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.936+0000 7fbe26353640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe20106670 0x7fbe2006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.936+0000 7fbe26353640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe2006d1f0 0x7fbe2006d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.936+0000 7fbe26353640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2006b760 con 0x7fbe200ffa60 2024-03-26T06:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.936+0000 7fbe26353640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2006b5e0 con 0x7fbe2006d1f0 2024-03-26T06:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.936+0000 7fbe26353640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2006b8e0 con 0x7fbe20106670 2024-03-26T06:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.936+0000 7fbe1ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe200ffa60 0x7fbe20072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.936+0000 7fbe1ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe200ffa60 0x7fbe20072bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41564/0 (socket says 172.21.15.37:41564) 2024-03-26T06:38:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.936+0000 7fbe1ffff640 1 -- 172.21.15.37:0/2242781234 learned_addr learned my addr 172.21.15.37:0/2242781234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.936+0000 7fbe248f0640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe2006d1f0 0x7fbe2006d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.937+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe20106670 0x7fbe2006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.937+0000 7fbe1ffff640 1 -- 172.21.15.37:0/2242781234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe20106670 msgr2=0x7fbe2006ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.937+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe20106670 0x7fbe2006ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.937+0000 7fbe1ffff640 1 -- 172.21.15.37:0/2242781234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe2006d1f0 msgr2=0x7fbe2006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.937+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe2006d1f0 0x7fbe2006d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.937+0000 7fbe1ffff640 1 -- 172.21.15.37:0/2242781234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe20100900 con 0x7fbe200ffa60 2024-03-26T06:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.937+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe200ffa60 0x7fbe20072bf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000e9f0 tx=0x7fbe1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.938+0000 7fbe1d7fa640 1 -- 172.21.15.37:0/2242781234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000cd80 con 0x7fbe200ffa60 2024-03-26T06:38:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.938+0000 7fbe1d7fa640 1 -- 172.21.15.37:0/2242781234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe10010910 con 0x7fbe200ffa60 2024-03-26T06:38:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.938+0000 7fbe1d7fa640 1 -- 172.21.15.37:0/2242781234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10018980 con 0x7fbe200ffa60 2024-03-26T06:38:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.938+0000 7fbe26353640 1 -- 172.21.15.37:0/2242781234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe20100bf0 con 0x7fbe200ffa60 2024-03-26T06:38:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.939+0000 7fbe26353640 1 -- 172.21.15.37:0/2242781234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe20101030 con 0x7fbe200ffa60 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.943+0000 7fbe29011740 1 Processor -- start 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.943+0000 7fbe1d7fa640 1 -- 172.21.15.37:0/2242781234 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fbe10010ac0 con 0x7fbe200ffa60 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.943+0000 7fbe29011740 1 -- start start 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.943+0000 7fbe29011740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2948d40 0x5571a2947a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.944+0000 7fbe29011740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 0x5571a2948450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.944+0000 7fbe29011740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a295c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.944+0000 7fbe29011740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a28ac5f0 con 0x5571a2948d40 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.944+0000 7fbe29011740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a2855930 con 0x5571a2959d60 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.944+0000 7fbe29011740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a28a8690 con 0x5571a2947fd0 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.944+0000 7fbe248f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2948d40 0x5571a2947a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.944+0000 7fbe248f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2948d40 0x5571a2947a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41580/0 (socket says 172.21.15.37:41580) 2024-03-26T06:38:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.944+0000 7fbe248f0640 1 -- 172.21.15.37:0/3715878174 learned_addr learned my addr 172.21.15.37:0/3715878174 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.944+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/3715878174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a295c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.944+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/3715878174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 0x5571a2948450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.945+0000 7fbe1d7fa640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbe080a2b70 0x7fbe080a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.945+0000 7fbe1f7fe640 1 -- 172.21.15.37:0/3715878174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 msgr2=0x5571a2948450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.945+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/3715878174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 0x5571a2948450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.945+0000 7fbe1f7fe640 1 -- 172.21.15.37:0/3715878174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2948d40 msgr2=0x5571a2947a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.945+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/3715878174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2948d40 0x5571a2947a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.945+0000 7fbe1f7fe640 1 -- 172.21.15.37:0/3715878174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a25c1210 con 0x5571a2959d60 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.945+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/3715878174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 0x5571a2948450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.945+0000 7fbe1d7fa640 1 -- 172.21.15.37:0/2242781234 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fbe100cf240 con 0x7fbe200ffa60 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.945+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/3715878174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a295c150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe2006e420 tx=0x7fbe1400acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7881d05f4328ddf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.946+0000 7fbe067fc640 1 -- 172.21.15.37:0/3715878174 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14015070 con 0x5571a2959d60 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.946+0000 7fbe067fc640 1 -- 172.21.15.37:0/3715878174 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe140113f0 con 0x5571a2959d60 2024-03-26T06:38:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.946+0000 7fbe067fc640 1 -- 172.21.15.37:0/3715878174 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe140185c0 con 0x5571a2959d60 2024-03-26T06:38:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.947+0000 7fbe248f0640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbe080a2b70 0x7fbe080a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.947+0000 7fbe29011740 1 -- 172.21.15.37:0/3715878174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 msgr2=0x5571a295c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.947+0000 7fbe29011740 1 --2- 172.21.15.37:0/3715878174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a295c150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe2006e420 tx=0x7fbe1400acc0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.947+0000 7fbe248f0640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbe080a2b70 0x7fbe080a5030 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009a60 tx=0x7fbe1802e040 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.948+0000 7fbe29011740 1 -- 172.21.15.37:0/3715878174 shutdown_connections 2024-03-26T06:38:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.948+0000 7fbe29011740 1 --2- 172.21.15.37:0/3715878174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a295c150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.948+0000 7fbe29011740 1 --2- 172.21.15.37:0/3715878174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 0x5571a2948450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.948+0000 7fbe29011740 1 --2- 172.21.15.37:0/3715878174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2948d40 0x5571a2947a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.948+0000 7fbe29011740 1 -- 172.21.15.37:0/3715878174 >> 172.21.15.37:0/3715878174 conn(0x5571a290f5b0 msgr2=0x5571a2942cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.948+0000 7fbe29011740 1 -- 172.21.15.37:0/3715878174 shutdown_connections 2024-03-26T06:38:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.948+0000 7fbe29011740 1 -- 172.21.15.37:0/3715878174 wait complete. 2024-03-26T06:38:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.949+0000 7fbe29011740 1 -- start start 2024-03-26T06:38:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.949+0000 7fbe29011740 1 Processor -- start 2024-03-26T06:38:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.949+0000 7fbe29011740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 0x5571a2956dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.949+0000 7fbe29011740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2948d40 0x5571a2957300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.949+0000 7fbe29011740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a2957840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.949+0000 7fbe29011740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a295ca90 con 0x5571a2947fd0 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.949+0000 7fbe29011740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a2868d90 con 0x5571a2959d60 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe29011740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a295cc10 con 0x5571a2948d40 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe29011740 10 client.0 _ll_register_callbacks cb 0x5571a25c0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe248f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a2957840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe1f7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2948d40 0x5571a2957300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe248f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a2957840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:45034/0 (socket says 172.21.15.37:45034) 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe248f0640 1 -- 172.21.15.37:0/1794218036 learned_addr learned my addr 172.21.15.37:0/1794218036 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 0x5571a2956dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe1ffff640 1 -- 172.21.15.37:0/1794218036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2948d40 msgr2=0x5571a2957300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2948d40 0x5571a2957300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe1ffff640 1 -- 172.21.15.37:0/1794218036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 msgr2=0x5571a2957840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a2957840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe1ffff640 1 -- 172.21.15.37:0/1794218036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571a268d620 con 0x5571a2947fd0 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2948d40 0x5571a2957300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.950+0000 7fbe248f0640 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a2957840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.951+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 0x5571a2956dc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbe100cc960 tx=0x7fbe1009b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c0ff6eee81610a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.951+0000 7fbdff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.951+0000 7fbdff7fe640 1 -- 172.21.15.37:0/1794218036 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe100ccdf0 con 0x5571a2947fd0 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.951+0000 7fbdff7fe640 1 -- 172.21.15.37:0/1794218036 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fbe100d2dc0 con 0x5571a2947fd0 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.951+0000 7fbe29011740 1 -- 172.21.15.37:0/1794218036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe140113f0 con 0x5571a2947fd0 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.951+0000 7fbdff7fe640 1 -- 172.21.15.37:0/1794218036 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe1009b870 con 0x5571a2947fd0 2024-03-26T06:38:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.951+0000 7fbe29011740 10 client.16791 fetch_fsmap learned FSMap version 63 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.951+0000 7fbe29011740 1 -- 172.21.15.37:0/1794218036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5571a2968bc0 con 0x5571a2947fd0 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.952+0000 7fbdff7fe640 1 -- 172.21.15.37:0/1794218036 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe1009f9c0 con 0x5571a2947fd0 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.952+0000 7fbdff7fe640 1 -- 172.21.15.37:0/1794218036 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 63) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fbe10010430 con 0x5571a2947fd0 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.952+0000 7fbe29011740 10 client.16791 fetch_fsmap finished waiting for FSMap version 63 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.952+0000 7fbe29011740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.952+0000 7fbe29011740 20 client.16791 populate_metadata read hostname 'smithi037' 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.952+0000 7fbe29011740 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5571a296a500 0x5571a296c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.952+0000 7fbe29011740 4 client.16791 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.952+0000 7fbe29011740 1 -- 172.21.15.37:0/1794218036 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe140113f0 con 0x5571a296a500 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.952+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5571a296a500 0x5571a296c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.953+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5571a296a500 0x5571a296c920 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46990ce339d731a1 server_cookie=2c04bd05a619248a in_seq=0 out_seq=0 2024-03-26T06:38:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.953+0000 7fbdff7fe640 10 client.16791 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.956+0000 7fbdff7fe640 1 -- 172.21.15.37:0/1794218036 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fbe140113f0 con 0x5571a296a500 2024-03-26T06:38:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.956+0000 7fbdff7fe640 10 client.16791 handle_command_reply: tid=0 2024-03-26T06:38:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 1 client.16791 shutdown 2024-03-26T06:38:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 1 -- 172.21.15.37:0/1794218036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 msgr2=0x5571a2956dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 0x5571a2956dc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbe100cc960 tx=0x7fbe1009b3b0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 1 -- 172.21.15.37:0/1794218036 shutdown_connections 2024-03-26T06:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5571a296a500 0x5571a296c920 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a2957840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbdff7fe640 0 client.16791 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2948d40 0x5571a2957300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 1 --2- 172.21.15.37:0/1794218036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 0x5571a2956dc0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 1 -- 172.21.15.37:0/1794218036 >> 172.21.15.37:0/1794218036 conn(0x5571a290f5b0 msgr2=0x5571a2944a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 1 -- 172.21.15.37:0/1794218036 shutdown_connections 2024-03-26T06:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 1 -- 172.21.15.37:0/1794218036 wait complete. 2024-03-26T06:38:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.957+0000 7fbe29011740 20 client.16791 trim_cache size 0 max 16384 2024-03-26T06:38:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.979+0000 7fbe29011740 1 Processor -- start 2024-03-26T06:38:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.979+0000 7fbe29011740 1 -- start start 2024-03-26T06:38:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.979+0000 7fbe29011740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 0x5571a29594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.979+0000 7fbe29011740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2948d40 0x5571a2952040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.980+0000 7fbe1ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 0x5571a29594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.980+0000 7fbe1ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 0x5571a29594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41612/0 (socket says 172.21.15.37:41612) 2024-03-26T06:38:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.980+0000 7fbe1ffff640 1 -- 172.21.15.37:0/1816427339 learned_addr learned my addr 172.21.15.37:0/1816427339 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.980+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/1816427339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2948d40 0x5571a2952040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.980+0000 7fbe29011740 1 --2- 172.21.15.37:0/1816427339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a2952580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.980+0000 7fbe29011740 1 -- 172.21.15.37:0/1816427339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe201b3320 con 0x5571a2947fd0 2024-03-26T06:38:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.980+0000 7fbe29011740 1 -- 172.21.15.37:0/1816427339 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe201b4dc0 con 0x5571a2959d60 2024-03-26T06:38:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.980+0000 7fbe29011740 1 -- 172.21.15.37:0/1816427339 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe201b39e0 con 0x5571a2948d40 2024-03-26T06:38:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.980+0000 7fbe248f0640 1 --2- 172.21.15.37:0/1816427339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a2952580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.981+0000 7fbe1ffff640 1 -- 172.21.15.37:0/1816427339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2948d40 msgr2=0x5571a2952040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.981+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/1816427339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2948d40 0x5571a2952040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.981+0000 7fbe1ffff640 1 -- 172.21.15.37:0/1816427339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 msgr2=0x5571a2952580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.981+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/1816427339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a2952580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.981+0000 7fbe1ffff640 1 -- 172.21.15.37:0/1816427339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a2966ee0 con 0x5571a2947fd0 2024-03-26T06:38:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.981+0000 7fbe248f0640 1 --2- 172.21.15.37:0/1816427339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a2952580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:38:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.981+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/1816427339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 0x5571a29594f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbe1009b570 tx=0x7fbe1009b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1794bc97edb5bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.981+0000 7fbe05ffb640 1 -- 172.21.15.37:0/1816427339 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10097b10 con 0x5571a2947fd0 2024-03-26T06:38:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.981+0000 7fbe05ffb640 1 -- 172.21.15.37:0/1816427339 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe100d2820 con 0x5571a2947fd0 2024-03-26T06:38:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.981+0000 7fbe05ffb640 1 -- 172.21.15.37:0/1816427339 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1009cda0 con 0x5571a2947fd0 2024-03-26T06:38:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.982+0000 7fbe29011740 1 -- 172.21.15.37:0/1816427339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 msgr2=0x5571a29594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.982+0000 7fbe29011740 1 --2- 172.21.15.37:0/1816427339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 0x5571a29594f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbe1009b570 tx=0x7fbe1009b2d0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.982+0000 7fbe29011740 1 -- 172.21.15.37:0/1816427339 shutdown_connections 2024-03-26T06:38:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.982+0000 7fbe29011740 1 --2- 172.21.15.37:0/1816427339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2959d60 0x5571a2952580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.982+0000 7fbe29011740 1 --2- 172.21.15.37:0/1816427339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2948d40 0x5571a2952040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.982+0000 7fbe29011740 1 --2- 172.21.15.37:0/1816427339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2947fd0 0x5571a29594f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.982+0000 7fbe29011740 1 -- 172.21.15.37:0/1816427339 >> 172.21.15.37:0/1816427339 conn(0x5571a290f5b0 msgr2=0x5571a29712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.983+0000 7fbe29011740 1 -- 172.21.15.37:0/1816427339 shutdown_connections 2024-03-26T06:38:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.983+0000 7fbe29011740 1 -- 172.21.15.37:0/1816427339 wait complete. 2024-03-26T06:38:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.983+0000 7fbe29011740 1 -- start start 2024-03-26T06:38:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.984+0000 7fbe29011740 1 Processor -- start 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.984+0000 7fbe29011740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 0x5571a2960ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.984+0000 7fbe29011740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2948d40 0x5571a29610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.984+0000 7fbe29011740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2959d60 0x5571a295f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.984+0000 7fbe1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2948d40 0x5571a29610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.984+0000 7fbe29011740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe201b4760 con 0x5571a2959d60 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.984+0000 7fbe29011740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe201b4040 con 0x5571a2948d40 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.984+0000 7fbe29011740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe201b15c0 con 0x5571a2947fd0 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.984+0000 7fbe1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2948d40 0x5571a29610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:45056/0 (socket says 172.21.15.37:45056) 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.984+0000 7fbe1f7fe640 1 -- 172.21.15.37:0/2090267215 learned_addr learned my addr 172.21.15.37:0/2090267215 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.985+0000 7fbe29011740 10 client.0 _ll_register_callbacks cb 0x7fbe1400a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.985+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 0x5571a2960ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.985+0000 7fbe1f7fe640 1 -- 172.21.15.37:0/2090267215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 msgr2=0x5571a2960ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.985+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 0x5571a2960ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.985+0000 7fbe1f7fe640 1 -- 172.21.15.37:0/2090267215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2959d60 msgr2=0x5571a295f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:38:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.985+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2959d60 0x5571a295f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.985+0000 7fbe1f7fe640 1 -- 172.21.15.37:0/2090267215 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571a295f820 con 0x5571a2948d40 2024-03-26T06:38:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.985+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 0x5571a2960ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:38:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.985+0000 7fbe1f7fe640 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2948d40 0x5571a29610e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400a780 tx=0x7fbe140046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5648ad5d21ba12fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:38:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.985+0000 7fbdff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:38:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.986+0000 7fbdff7fe640 1 -- 172.21.15.37:0/2090267215 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14015040 con 0x5571a2948d40 2024-03-26T06:38:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.986+0000 7fbdff7fe640 1 -- 172.21.15.37:0/2090267215 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fbe140275f0 con 0x5571a2948d40 2024-03-26T06:38:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.986+0000 7fbe29011740 1 -- 172.21.15.37:0/2090267215 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571a28b9b30 con 0x5571a2948d40 2024-03-26T06:38:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.986+0000 7fbdff7fe640 1 -- 172.21.15.37:0/2090267215 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe140277f0 con 0x5571a2948d40 2024-03-26T06:38:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.986+0000 7fbe29011740 10 client.26300 fetch_fsmap learned FSMap version 63 2024-03-26T06:38:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.986+0000 7fbe29011740 1 -- 172.21.15.37:0/2090267215 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5571a29c14f0 con 0x5571a2948d40 2024-03-26T06:38:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.987+0000 7fbdff7fe640 1 -- 172.21.15.37:0/2090267215 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5571a28b9b30 con 0x5571a2948d40 2024-03-26T06:38:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.987+0000 7fbdff7fe640 1 -- 172.21.15.37:0/2090267215 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 63) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fbe14027ce0 con 0x5571a2948d40 2024-03-26T06:38:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.987+0000 7fbe29011740 10 client.26300 fetch_fsmap finished waiting for FSMap version 63 2024-03-26T06:38:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.987+0000 7fbe29011740 10 client.26300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:38:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.987+0000 7fbe29011740 20 client.26300 populate_metadata read hostname 'smithi037' 2024-03-26T06:38:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.987+0000 7fbe29011740 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5571a296a500 0x5571a29c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:38:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.987+0000 7fbe29011740 4 client.26300 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:38:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.987+0000 7fbe29011740 1 -- 172.21.15.37:0/2090267215 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5571a28b3e80 con 0x5571a296a500 2024-03-26T06:38:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.987+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5571a296a500 0x5571a29c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:38:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.988+0000 7fbdff7fe640 10 client.26300 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:50.988+0000 7fbe1ffff640 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5571a296a500 0x5571a29c3a80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f18e19638a2a126 server_cookie=bd39c01ec0043540 in_seq=0 out_seq=0 2024-03-26T06:38:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.215+0000 7fbdff7fe640 1 -- 172.21.15.37:0/2090267215 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5571a28b3e80 con 0x5571a296a500 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.215+0000 7fbdff7fe640 10 client.26300 handle_command_reply: tid=0 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.215+0000 7fbe29011740 1 client.26300 shutdown 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.215+0000 7fbe29011740 1 -- 172.21.15.37:0/2090267215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2948d40 msgr2=0x5571a29610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.215+0000 7fbe29011740 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2948d40 0x5571a29610e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400a780 tx=0x7fbe140046c0 comp rx=0 tx=0).stop 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.215+0000 7fbe29011740 1 -- 172.21.15.37:0/2090267215 shutdown_connections 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.215+0000 7fbe29011740 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x5571a296a500 0x5571a29c3a80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.215+0000 7fbe29011740 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5571a2959d60 0x5571a295f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.215+0000 7fbe29011740 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571a2948d40 0x5571a29610e0 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400a780 tx=0x7fbe140046c0 comp rx=0 tx=0).stop 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.215+0000 7fbe29011740 1 --2- 172.21.15.37:0/2090267215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571a2947fd0 0x5571a2960ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.215+0000 7fbe29011740 1 -- 172.21.15.37:0/2090267215 >> 172.21.15.37:0/2090267215 conn(0x5571a290f5b0 msgr2=0x5571a2952040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.216+0000 7fbdff7fe640 0 client.26300 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:38:51.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.216+0000 7fbe29011740 1 -- 172.21.15.37:0/2090267215 shutdown_connections 2024-03-26T06:38:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.216+0000 7fbe29011740 1 -- 172.21.15.37:0/2090267215 wait complete. 2024-03-26T06:38:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.216+0000 7fbe29011740 20 client.26300 trim_cache size 0 max 16384 2024-03-26T06:38:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:38:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:38:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:38:51.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:38:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 -- 172.21.15.37:0/2242781234 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbe080a2b70 msgr2=0x7fbe080a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbe080a2b70 0x7fbe080a5030 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009a60 tx=0x7fbe1802e040 comp rx=0 tx=0).stop 2024-03-26T06:38:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 -- 172.21.15.37:0/2242781234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe200ffa60 msgr2=0x7fbe20072bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:38:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe200ffa60 0x7fbe20072bf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000e9f0 tx=0x7fbe1000eeb0 comp rx=0 tx=0).stop 2024-03-26T06:38:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 -- 172.21.15.37:0/2242781234 shutdown_connections 2024-03-26T06:38:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fbe080a2b70 0x7fbe080a5030 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe2006d1f0 0x7fbe2006d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe20106670 0x7fbe2006ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 --2- 172.21.15.37:0/2242781234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe200ffa60 0x7fbe20072bf0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:38:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 -- 172.21.15.37:0/2242781234 >> 172.21.15.37:0/2242781234 conn(0x7fbe20076780 msgr2=0x7fbe20104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:38:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 -- 172.21.15.37:0/2242781234 shutdown_connections 2024-03-26T06:38:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:38:51.217+0000 7fbe26353640 1 -- 172.21.15.37:0/2242781234 wait complete. 2024-03-26T06:38:51.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 129298 43.59 MB/sec execute 26 sec latency 370.731 ms 2024-03-26T06:38:52.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:52 smithi149 ceph-mon[36681]: pgmap v501: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 10 MiB/s wr, 955 op/s 2024-03-26T06:38:52.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:52 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:52.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:52 smithi037 ceph-mon[31974]: pgmap v501: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 10 MiB/s wr, 955 op/s 2024-03-26T06:38:52.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:52 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:52.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:52 smithi066 ceph-mon[36504]: pgmap v501: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 10 MiB/s wr, 955 op/s 2024-03-26T06:38:52.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:52 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:52.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 129323 42.04 MB/sec execute 27 sec latency 817.588 ms 2024-03-26T06:38:53.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 129765 41.33 MB/sec execute 28 sec latency 981.730 ms 2024-03-26T06:38:54.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:54 smithi149 ceph-mon[36681]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.9 MiB/s wr, 597 op/s 2024-03-26T06:38:54.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:54 smithi037 ceph-mon[31974]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.9 MiB/s wr, 597 op/s 2024-03-26T06:38:54.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:54 smithi066 ceph-mon[36504]: pgmap v502: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.9 MiB/s wr, 597 op/s 2024-03-26T06:38:54.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 130640 41.65 MB/sec execute 29 sec latency 72.695 ms 2024-03-26T06:38:55.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 131619 42.12 MB/sec execute 30 sec latency 70.073 ms 2024-03-26T06:38:56.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:56 smithi149 ceph-mon[36681]: pgmap v503: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 12 MiB/s wr, 884 op/s 2024-03-26T06:38:56.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:56 smithi037 ceph-mon[31974]: pgmap v503: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 12 MiB/s wr, 884 op/s 2024-03-26T06:38:56.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:56 smithi066 ceph-mon[36504]: pgmap v503: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 12 MiB/s wr, 884 op/s 2024-03-26T06:38:56.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 132339 41.75 MB/sec execute 31 sec latency 55.250 ms 2024-03-26T06:38:57.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:57 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:57.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:57 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:57.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:57 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:57.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 133362 42.20 MB/sec execute 32 sec latency 50.486 ms 2024-03-26T06:38:58.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:58 smithi149 ceph-mon[36681]: pgmap v504: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-03-26T06:38:58.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:38:58 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:58.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:58 smithi037 ceph-mon[31974]: pgmap v504: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-03-26T06:38:58.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:38:58 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:58.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:58 smithi066 ceph-mon[36504]: pgmap v504: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-03-26T06:38:58.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:38:58 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:38:58.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 134421 42.78 MB/sec execute 33 sec latency 42.629 ms 2024-03-26T06:38:59.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 135277 42.88 MB/sec execute 34 sec latency 74.350 ms 2024-03-26T06:39:00.401 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:00 smithi066 ceph-mon[36504]: pgmap v505: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 1.09k op/s 2024-03-26T06:39:00.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:00 smithi149 ceph-mon[36681]: pgmap v505: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 1.09k op/s 2024-03-26T06:39:00.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:00 smithi037 ceph-mon[31974]: pgmap v505: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 1.09k op/s 2024-03-26T06:39:00.850 INFO:tasks.workunit.client.0.smithi037.stdout: 10 136134 42.83 MB/sec execute 35 sec latency 98.975 ms 2024-03-26T06:39:01.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:01 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:39:01.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:01 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:39:01.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:01 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:39:01.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 136134 41.64 MB/sec execute 36 sec latency 1099.050 ms 2024-03-26T06:39:02.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:02 smithi149 ceph-mon[36681]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-03-26T06:39:02.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:02 smithi037 ceph-mon[31974]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-03-26T06:39:02.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:02 smithi066 ceph-mon[36504]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-03-26T06:39:02.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 136219 40.65 MB/sec execute 37 sec latency 1981.692 ms 2024-03-26T06:39:03.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 136284 39.64 MB/sec execute 38 sec latency 958.281 ms 2024-03-26T06:39:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.321+0000 7f46a8473640 1 -- 172.21.15.37:0/3693564446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01012e0 msgr2=0x7f46a010d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.321+0000 7f46a8473640 1 --2- 172.21.15.37:0/3693564446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01012e0 0x7f46a010d090 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4698009ca0 tx=0x7f469802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.321+0000 7f46a8473640 1 -- 172.21.15.37:0/3693564446 shutdown_connections 2024-03-26T06:39:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.321+0000 7f46a8473640 1 --2- 172.21.15.37:0/3693564446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01012e0 0x7f46a010d090 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.321+0000 7f46a8473640 1 --2- 172.21.15.37:0/3693564446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a0100920 0x7f46a0100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.321+0000 7f46a8473640 1 --2- 172.21.15.37:0/3693564446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46a0105200 0x7f46a01055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.321+0000 7f46a8473640 1 -- 172.21.15.37:0/3693564446 >> 172.21.15.37:0/3693564446 conn(0x7f46a0076780 msgr2=0x7f46a00fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.321+0000 7f46a8473640 1 -- 172.21.15.37:0/3693564446 shutdown_connections 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.321+0000 7f46a8473640 1 -- 172.21.15.37:0/3693564446 wait complete. 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a8473640 1 Processor -- start 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a8473640 1 -- start start 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a8473640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a0100920 0x7f46a01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a8473640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a01012e0 0x7f46a0198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a8473640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46a0105200 0x7f46a01a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a8473640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a0110580 con 0x7f46a0100920 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a8473640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a0110400 con 0x7f46a0105200 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a8473640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a0110700 con 0x7f46a01012e0 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a620f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a0100920 0x7f46a01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a620f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a0100920 0x7f46a01987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35364/0 (socket says 172.21.15.37:35364) 2024-03-26T06:39:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a620f640 1 -- 172.21.15.37:0/430968927 learned_addr learned my addr 172.21.15.37:0/430968927 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a01012e0 0x7f46a0198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.322+0000 7f46a6a10640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46a0105200 0x7f46a01a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46a620f640 1 -- 172.21.15.37:0/430968927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a01012e0 msgr2=0x7f46a0198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46a620f640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a01012e0 0x7f46a0198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46a620f640 1 -- 172.21.15.37:0/430968927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46a0105200 msgr2=0x7f46a01a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46a620f640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46a0105200 0x7f46a01a3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46a620f640 1 -- 172.21.15.37:0/430968927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a01a3d80 con 0x7f46a0100920 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46a6a10640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46a0105200 0x7f46a01a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46a620f640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a0100920 0x7f46a01987c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f469c00d950 tx=0x7f469c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46977fe640 1 -- 172.21.15.37:0/430968927 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c017030 con 0x7f46a0100920 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46977fe640 1 -- 172.21.15.37:0/430968927 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f469c004020 con 0x7f46a0100920 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46a8473640 1 -- 172.21.15.37:0/430968927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a01a3f50 con 0x7f46a0100920 2024-03-26T06:39:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46977fe640 1 -- 172.21.15.37:0/430968927 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c0047a0 con 0x7f46a0100920 2024-03-26T06:39:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.323+0000 7f46a8473640 1 -- 172.21.15.37:0/430968927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a01a4460 con 0x7f46a0100920 2024-03-26T06:39:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.325+0000 7f46977fe640 1 -- 172.21.15.37:0/430968927 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f469c0041d0 con 0x7f46a0100920 2024-03-26T06:39:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.326+0000 7f46ab053740 1 Processor -- start 2024-03-26T06:39:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.326+0000 7f46ab053740 1 -- start start 2024-03-26T06:39:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.326+0000 7f46977fe640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f468c0a2a90 0x7f468c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.326+0000 7f46ab053740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee83a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.326+0000 7f46ab053740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571ee84450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.326+0000 7f46ab053740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571ee8f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.326+0000 7f46ab053740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571ede85f0 con 0x55571ee84d40 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.326+0000 7f46ab053740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571ed91930 con 0x55571ee8d550 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.326+0000 7f46ab053740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571ede4690 con 0x55571ee83fd0 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a6a10640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f468c0a2a90 0x7f468c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a620f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee83a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a620f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee83a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35374/0 (socket says 172.21.15.37:35374) 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a620f640 1 -- 172.21.15.37:0/2941481066 learned_addr learned my addr 172.21.15.37:0/2941481066 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/2941481066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571ee84450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46977fe640 1 -- 172.21.15.37:0/430968927 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f469c0ce780 con 0x7f46a0100920 2024-03-26T06:39:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a6a10640 1 --2- 172.21.15.37:0/2941481066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571ee8f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a620f640 1 -- 172.21.15.37:0/2941481066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 msgr2=0x55571ee84450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a620f640 1 --2- 172.21.15.37:0/2941481066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571ee84450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a620f640 1 -- 172.21.15.37:0/2941481066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 msgr2=0x55571ee8f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a620f640 1 --2- 172.21.15.37:0/2941481066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571ee8f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a620f640 1 -- 172.21.15.37:0/2941481066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55571eafd210 con 0x55571ee84d40 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/2941481066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571ee84450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a6a10640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f468c0a2a90 0x7f468c0a4f50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4698005e30 tx=0x7f4698032ec0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f46a620f640 1 --2- 172.21.15.37:0/2941481066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee83a90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f469c0cbf00 tx=0x7f469c005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2965729ce49f97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f4694bfe640 1 -- 172.21.15.37:0/2941481066 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c09a710 con 0x55571ee84d40 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f4694bfe640 1 -- 172.21.15.37:0/2941481066 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f469c0a4410 con 0x55571ee84d40 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.327+0000 7f4694bfe640 1 -- 172.21.15.37:0/2941481066 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c09b4b0 con 0x55571ee84d40 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.328+0000 7f46ab053740 1 -- 172.21.15.37:0/2941481066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 msgr2=0x55571ee83a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.328+0000 7f46ab053740 1 --2- 172.21.15.37:0/2941481066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee83a90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f469c0cbf00 tx=0x7f469c005da0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.328+0000 7f46ab053740 1 -- 172.21.15.37:0/2941481066 shutdown_connections 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.328+0000 7f46ab053740 1 --2- 172.21.15.37:0/2941481066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571ee8f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.328+0000 7f46ab053740 1 --2- 172.21.15.37:0/2941481066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571ee84450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.328+0000 7f46ab053740 1 --2- 172.21.15.37:0/2941481066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee83a90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.328+0000 7f46ab053740 1 -- 172.21.15.37:0/2941481066 >> 172.21.15.37:0/2941481066 conn(0x55571ee4b5b0 msgr2=0x55571ee7ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.328+0000 7f46ab053740 1 -- 172.21.15.37:0/2941481066 shutdown_connections 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.328+0000 7f46ab053740 1 -- 172.21.15.37:0/2941481066 wait complete. 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46ab053740 1 -- start start 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46ab053740 1 Processor -- start 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46ab053740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571ee8a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46ab053740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee8ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46ab053740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571ee8b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46ab053740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571ee90220 con 0x55571ee84d40 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46ab053740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571eda4d90 con 0x55571ee8d550 2024-03-26T06:39:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46ab053740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571ee903a0 con 0x55571ee83fd0 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46ab053740 10 client.0 _ll_register_callbacks cb 0x55571eafca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46a5a0e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee8ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46a5a0e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee8ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35390/0 (socket says 172.21.15.37:35390) 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46a5a0e640 1 -- 172.21.15.37:0/1526890278 learned_addr learned my addr 172.21.15.37:0/1526890278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46a6a10640 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571ee8b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46a620f640 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571ee8a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46a5a0e640 1 -- 172.21.15.37:0/1526890278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 msgr2=0x55571ee8a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571ee8a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46a5a0e640 1 -- 172.21.15.37:0/1526890278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 msgr2=0x55571ee8b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571ee8b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.329+0000 7f46a5a0e640 1 -- 172.21.15.37:0/1526890278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55571ebc9620 con 0x55571ee84d40 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee8ab40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f469000ba60 tx=0x7f469000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246c84c30f4ed207 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f4689ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f4689ffb640 1 -- 172.21.15.37:0/1526890278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690011440 con 0x55571ee84d40 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f4689ffb640 1 -- 172.21.15.37:0/1526890278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f46900115e0 con 0x55571ee84d40 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f46ab053740 1 -- 172.21.15.37:0/1526890278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f469c0a4410 con 0x55571ee84d40 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f4689ffb640 1 -- 172.21.15.37:0/1526890278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4690005020 con 0x55571ee84d40 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f46ab053740 10 client.16806 fetch_fsmap learned FSMap version 67 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f46ab053740 1 -- 172.21.15.37:0/1526890278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55571ee9c5f0 con 0x55571ee84d40 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f4689ffb640 1 -- 172.21.15.37:0/1526890278 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f469c0a4410 con 0x55571ee84d40 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f4689ffb640 1 -- 172.21.15.37:0/1526890278 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f46900047a0 con 0x55571ee84d40 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f46ab053740 10 client.16806 fetch_fsmap finished waiting for FSMap version 67 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f46ab053740 10 client.16806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f46ab053740 20 client.16806 populate_metadata read hostname 'smithi037' 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f46ab053740 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55571ee9db00 0x55571ee9fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f46ab053740 4 client.16806 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:39:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.330+0000 7f46ab053740 1 -- 172.21.15.37:0/1526890278 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55571edf5b30 con 0x55571ee9db00 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.331+0000 7f46a620f640 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55571ee9db00 0x55571ee9fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.331+0000 7f46a620f640 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55571ee9db00 0x55571ee9fef0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9fc5bdfe5fa014f server_cookie=bb0b39d7cfdb0d7d in_seq=0 out_seq=0 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.331+0000 7f4689ffb640 10 client.16806 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f4689ffb640 1 -- 172.21.15.37:0/1526890278 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55571edf5b30 con 0x55571ee9db00 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f4689ffb640 10 client.16806 handle_command_reply: tid=0 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 1 client.16806 shutdown 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 1 -- 172.21.15.37:0/1526890278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 msgr2=0x55571ee8ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee8ab40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f469000ba60 tx=0x7f469000bf20 comp rx=0 tx=0).stop 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 1 -- 172.21.15.37:0/1526890278 shutdown_connections 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55571ee9db00 0x55571ee9fef0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571ee8b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee8ab40 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f4689ffb640 0 client.16806 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 1 --2- 172.21.15.37:0/1526890278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571ee8a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 1 -- 172.21.15.37:0/1526890278 >> 172.21.15.37:0/1526890278 conn(0x55571ee4b5b0 msgr2=0x55571ee80a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 1 -- 172.21.15.37:0/1526890278 shutdown_connections 2024-03-26T06:39:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 1 -- 172.21.15.37:0/1526890278 wait complete. 2024-03-26T06:39:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.332+0000 7f46ab053740 20 client.16806 trim_cache size 0 max 16384 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.345+0000 7f46ab053740 1 Processor -- start 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.345+0000 7f46ab053740 1 -- start start 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46ab053740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee83fd0 0x55571ee8cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46ab053740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee94a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46ab053740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee8d550 0x55571ee87900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46ab053740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a01acb50 con 0x55571ee84d40 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46ab053740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a01ae500 con 0x55571ee83fd0 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46ab053740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a01ad1e0 con 0x55571ee8d550 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a5a0e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee94a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a5a0e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee94a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35404/0 (socket says 172.21.15.37:35404) 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a5a0e640 1 -- 172.21.15.37:0/280405577 learned_addr learned my addr 172.21.15.37:0/280405577 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a620f640 1 --2- 172.21.15.37:0/280405577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee83fd0 0x55571ee8cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a5a0e640 1 -- 172.21.15.37:0/280405577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee8d550 msgr2=0x55571ee87900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a6a10640 1 --2- 172.21.15.37:0/280405577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee8d550 0x55571ee87900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/280405577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee8d550 0x55571ee87900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a5a0e640 1 -- 172.21.15.37:0/280405577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee83fd0 msgr2=0x55571ee8cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/280405577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee83fd0 0x55571ee8cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a5a0e640 1 -- 172.21.15.37:0/280405577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55571ee832a0 con 0x55571ee84d40 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a6a10640 1 --2- 172.21.15.37:0/280405577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee8d550 0x55571ee87900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a620f640 1 --2- 172.21.15.37:0/280405577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee83fd0 0x55571ee8cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/280405577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee94a00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4690001eb0 tx=0x7f4690003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b268517d73dd8af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f468bfff640 1 -- 172.21.15.37:0/280405577 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690030070 con 0x55571ee84d40 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f468bfff640 1 -- 172.21.15.37:0/280405577 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4690004e00 con 0x55571ee84d40 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.346+0000 7f468bfff640 1 -- 172.21.15.37:0/280405577 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690027040 con 0x55571ee84d40 2024-03-26T06:39:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.347+0000 7f46ab053740 1 -- 172.21.15.37:0/280405577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 msgr2=0x55571ee94a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.347+0000 7f46ab053740 1 --2- 172.21.15.37:0/280405577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee94a00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4690001eb0 tx=0x7f4690003b00 comp rx=0 tx=0).stop 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.347+0000 7f46ab053740 1 -- 172.21.15.37:0/280405577 shutdown_connections 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.347+0000 7f46ab053740 1 --2- 172.21.15.37:0/280405577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee8d550 0x55571ee87900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.347+0000 7f46ab053740 1 --2- 172.21.15.37:0/280405577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571ee94a00 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.347+0000 7f46ab053740 1 --2- 172.21.15.37:0/280405577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee83fd0 0x55571ee8cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.347+0000 7f46ab053740 1 -- 172.21.15.37:0/280405577 >> 172.21.15.37:0/280405577 conn(0x55571ee4b5b0 msgr2=0x55571ee88f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.347+0000 7f46ab053740 1 -- 172.21.15.37:0/280405577 shutdown_connections 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.347+0000 7f46ab053740 1 -- 172.21.15.37:0/280405577 wait complete. 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.347+0000 7f46ab053740 1 -- start start 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.347+0000 7f46ab053740 1 Processor -- start 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46ab053740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571eef23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46ab053740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571eef2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46ab053740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571eeec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46ab053740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a01adea0 con 0x55571ee84d40 2024-03-26T06:39:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46ab053740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a01ad840 con 0x55571ee8d550 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46ab053740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a01aadf0 con 0x55571ee83fd0 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46ab053740 10 client.0 _ll_register_callbacks cb 0x7f469c004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a5a0e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571eef2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a5a0e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571eef2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35416/0 (socket says 172.21.15.37:35416) 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a5a0e640 1 -- 172.21.15.37:0/1285835647 learned_addr learned my addr 172.21.15.37:0/1285835647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a620f640 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571eef23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a6a10640 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571eeec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a5a0e640 1 -- 172.21.15.37:0/1285835647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 msgr2=0x55571eef23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571eef23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a5a0e640 1 -- 172.21.15.37:0/1285835647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 msgr2=0x55571eeec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571eeec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a5a0e640 1 -- 172.21.15.37:0/1285835647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55571eeecee0 con 0x55571ee84d40 2024-03-26T06:39:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a6a10640 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571eeec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f46a5a0e640 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571eef2920 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4690004d30 tx=0x7f46900042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310153d358482b34 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f4689ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.348+0000 7f4689ffb640 1 -- 172.21.15.37:0/1285835647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690030070 con 0x55571ee84d40 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f4689ffb640 1 -- 172.21.15.37:0/1285835647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f469000b450 con 0x55571ee84d40 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f46ab053740 1 -- 172.21.15.37:0/1285835647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4690004e00 con 0x55571ee84d40 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f4689ffb640 1 -- 172.21.15.37:0/1285835647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4690027040 con 0x55571ee84d40 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f46ab053740 10 client.16812 fetch_fsmap learned FSMap version 67 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f46ab053740 1 -- 172.21.15.37:0/1285835647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55571eef9b60 con 0x55571ee84d40 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f4689ffb640 1 -- 172.21.15.37:0/1285835647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4690004e00 con 0x55571ee84d40 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f4689ffb640 1 -- 172.21.15.37:0/1285835647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f469003c070 con 0x55571ee84d40 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f46ab053740 10 client.16812 fetch_fsmap finished waiting for FSMap version 67 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f46ab053740 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f46ab053740 20 client.16812 populate_metadata read hostname 'smithi037' 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f46ab053740 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55571eefa160 0x55571eefc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f46ab053740 4 client.16812 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:39:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f46ab053740 1 -- 172.21.15.37:0/1285835647 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55571edefe80 con 0x55571eefa160 2024-03-26T06:39:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.349+0000 7f46a620f640 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55571eefa160 0x55571eefc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.350+0000 7f4689ffb640 10 client.16812 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.350+0000 7f46a620f640 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55571eefa160 0x55571eefc550 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb86d17c122ec592 server_cookie=40fcad8095d431e7 in_seq=0 out_seq=0 2024-03-26T06:39:04.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:04 smithi149 ceph-mon[36681]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.08k op/s 2024-03-26T06:39:04.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:04 smithi037 ceph-mon[31974]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.08k op/s 2024-03-26T06:39:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f4689ffb640 1 -- 172.21.15.37:0/1285835647 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55571edefe80 con 0x55571eefa160 2024-03-26T06:39:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f4689ffb640 10 client.16812 handle_command_reply: tid=0 2024-03-26T06:39:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 1 client.16812 shutdown 2024-03-26T06:39:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 1 -- 172.21.15.37:0/1285835647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 msgr2=0x55571eef2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571eef2920 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4690004d30 tx=0x7f46900042e0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 1 -- 172.21.15.37:0/1285835647 shutdown_connections 2024-03-26T06:39:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55571eefa160 0x55571eefc550 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55571ee8d550 0x55571eeec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571ee84d40 0x55571eef2920 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f4689ffb640 0 client.16812 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 1 --2- 172.21.15.37:0/1285835647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55571ee83fd0 0x55571eef23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 1 -- 172.21.15.37:0/1285835647 >> 172.21.15.37:0/1285835647 conn(0x55571ee4b5b0 msgr2=0x55571ee8ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 1 -- 172.21.15.37:0/1285835647 shutdown_connections 2024-03-26T06:39:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 1 -- 172.21.15.37:0/1285835647 wait complete. 2024-03-26T06:39:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.483+0000 7f46ab053740 20 client.16812 trim_cache size 0 max 16384 2024-03-26T06:39:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:39:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:39:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:39:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:39:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 -- 172.21.15.37:0/430968927 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f468c0a2a90 msgr2=0x7f468c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f468c0a2a90 0x7f468c0a4f50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4698005e30 tx=0x7f4698032ec0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 -- 172.21.15.37:0/430968927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a0100920 msgr2=0x7f46a01987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a0100920 0x7f46a01987c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f469c00d950 tx=0x7f469c00de10 comp rx=0 tx=0).stop 2024-03-26T06:39:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 -- 172.21.15.37:0/430968927 shutdown_connections 2024-03-26T06:39:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f468c0a2a90 0x7f468c0a4f50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46a0105200 0x7f46a01a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a01012e0 0x7f46a0198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 --2- 172.21.15.37:0/430968927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a0100920 0x7f46a01987c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 -- 172.21.15.37:0/430968927 >> 172.21.15.37:0/430968927 conn(0x7f46a0076780 msgr2=0x7f46a0102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 -- 172.21.15.37:0/430968927 shutdown_connections 2024-03-26T06:39:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:04.484+0000 7f46a8473640 1 -- 172.21.15.37:0/430968927 wait complete. 2024-03-26T06:39:04.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:04 smithi066 ceph-mon[36504]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.08k op/s 2024-03-26T06:39:04.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 136284 38.62 MB/sec execute 39 sec latency 1958.384 ms 2024-03-26T06:39:05.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 136284 37.66 MB/sec execute 40 sec latency 2958.473 ms 2024-03-26T06:39:06.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:06 smithi037 ceph-mon[31974]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-03-26T06:39:06.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:06 smithi066 ceph-mon[36504]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-03-26T06:39:06.661 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:06 smithi149 ceph-mon[36681]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-03-26T06:39:06.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 136747 37.20 MB/sec execute 41 sec latency 3455.300 ms 2024-03-26T06:39:07.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 136943 36.67 MB/sec execute 42 sec latency 127.517 ms 2024-03-26T06:39:08.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:08 smithi037 ceph-mon[31974]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 945 op/s 2024-03-26T06:39:08.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:08 smithi066 ceph-mon[36504]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 945 op/s 2024-03-26T06:39:08.661 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:08 smithi149 ceph-mon[36681]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 945 op/s 2024-03-26T06:39:08.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 137860 37.12 MB/sec execute 43 sec latency 67.681 ms 2024-03-26T06:39:09.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 138760 37.41 MB/sec execute 44 sec latency 210.482 ms 2024-03-26T06:39:10.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:10 smithi037 ceph-mon[31974]: pgmap v510: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-03-26T06:39:10.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:10 smithi066 ceph-mon[36504]: pgmap v510: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-03-26T06:39:10.661 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:10 smithi149 ceph-mon[36681]: pgmap v510: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.3 MiB/s wr, 453 op/s 2024-03-26T06:39:10.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 139279 37.13 MB/sec execute 45 sec latency 109.654 ms 2024-03-26T06:39:11.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:11 smithi037 ceph-mon[31974]: osdmap e111: 12 total, 12 up, 12 in 2024-03-26T06:39:11.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:11 smithi066 ceph-mon[36504]: osdmap e111: 12 total, 12 up, 12 in 2024-03-26T06:39:11.661 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:11 smithi149 ceph-mon[36681]: osdmap e111: 12 total, 12 up, 12 in 2024-03-26T06:39:11.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 140057 37.19 MB/sec execute 46 sec latency 84.787 ms 2024-03-26T06:39:12.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:12 smithi037 ceph-mon[31974]: pgmap v512: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.7 MiB/s wr, 628 op/s 2024-03-26T06:39:12.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:12 smithi066 ceph-mon[36504]: pgmap v512: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.7 MiB/s wr, 628 op/s 2024-03-26T06:39:12.661 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:12 smithi149 ceph-mon[36681]: pgmap v512: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.7 MiB/s wr, 628 op/s 2024-03-26T06:39:12.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 141013 37.71 MB/sec execute 47 sec latency 40.585 ms 2024-03-26T06:39:13.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 141954 37.81 MB/sec execute 48 sec latency 65.317 ms 2024-03-26T06:39:14.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:14 smithi037 ceph-mon[31974]: pgmap v513: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.7 MiB/s wr, 628 op/s 2024-03-26T06:39:14.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:14 smithi066 ceph-mon[36504]: pgmap v513: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.7 MiB/s wr, 628 op/s 2024-03-26T06:39:14.661 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:14 smithi149 ceph-mon[36681]: pgmap v513: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.7 MiB/s wr, 628 op/s 2024-03-26T06:39:14.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 142955 38.22 MB/sec execute 49 sec latency 54.677 ms 2024-03-26T06:39:15.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 143747 38.31 MB/sec execute 50 sec latency 45.256 ms 2024-03-26T06:39:16.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:16 smithi037 ceph-mon[31974]: pgmap v514: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.5 MiB/s wr, 725 op/s 2024-03-26T06:39:16.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:16 smithi066 ceph-mon[36504]: pgmap v514: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.5 MiB/s wr, 725 op/s 2024-03-26T06:39:16.661 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:16 smithi149 ceph-mon[36681]: pgmap v514: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.5 MiB/s wr, 725 op/s 2024-03-26T06:39:16.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 144526 38.34 MB/sec execute 51 sec latency 71.201 ms 2024-03-26T06:39:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.606+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3490004081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a8069330 msgr2=0x7fc2a810e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.606+0000 7fc2ae1c6640 1 --2- 172.21.15.37:0/3490004081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a8069330 0x7fc2a810e170 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009ca0 tx=0x7fc2a002c190 comp rx=0 tx=0).stop 2024-03-26T06:39:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.606+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3490004081 shutdown_connections 2024-03-26T06:39:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.606+0000 7fc2ae1c6640 1 --2- 172.21.15.37:0/3490004081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a8069330 0x7fc2a810e170 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.606+0000 7fc2ae1c6640 1 --2- 172.21.15.37:0/3490004081 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2a8068970 0x7fc2a8068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.606+0000 7fc2ae1c6640 1 --2- 172.21.15.37:0/3490004081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2a8107400 0x7fc2a81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.606+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3490004081 >> 172.21.15.37:0/3490004081 conn(0x7fc2a8077040 msgr2=0x7fc2a8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.606+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3490004081 shutdown_connections 2024-03-26T06:39:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.606+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3490004081 wait complete. 2024-03-26T06:39:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.607+0000 7fc2ae1c6640 1 Processor -- start 2024-03-26T06:39:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.607+0000 7fc2ae1c6640 1 -- start start 2024-03-26T06:39:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.607+0000 7fc2ae1c6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2a8068970 0x7fc2a819d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2ae1c6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2a8069330 0x7fc2a819dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2ae1c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a8107400 0x7fc2a8197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2ae1c6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8111640 con 0x7fc2a8107400 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2ae1c6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81114c0 con 0x7fc2a8069330 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2ae1c6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81117c0 con 0x7fc2a8068970 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2a6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2a8069330 0x7fc2a819dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2a7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a8107400 0x7fc2a8197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2a6ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2a8069330 0x7fc2a819dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:56970/0 (socket says 172.21.15.37:56970) 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2a6ffd640 1 -- 172.21.15.37:0/3858488209 learned_addr learned my addr 172.21.15.37:0/3858488209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2a8068970 0x7fc2a819d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2a6ffd640 1 -- 172.21.15.37:0/3858488209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2a8068970 msgr2=0x7fc2a819d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2a6ffd640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2a8068970 0x7fc2a819d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2a6ffd640 1 -- 172.21.15.37:0/3858488209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a8107400 msgr2=0x7fc2a8197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2a6ffd640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a8107400 0x7fc2a8197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2a6ffd640 1 -- 172.21.15.37:0/3858488209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8198080 con 0x7fc2a8069330 2024-03-26T06:39:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.608+0000 7fc2a6ffd640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2a8069330 0x7fc2a819dc40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00e9f0 tx=0x7fc29c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.609+0000 7fc2a4ff9640 1 -- 172.21.15.37:0/3858488209 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00cd80 con 0x7fc2a8069330 2024-03-26T06:39:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.609+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3858488209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a8198370 con 0x7fc2a8069330 2024-03-26T06:39:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.609+0000 7fc2a4ff9640 1 -- 172.21.15.37:0/3858488209 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc29c010910 con 0x7fc2a8069330 2024-03-26T06:39:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.609+0000 7fc2a4ff9640 1 -- 172.21.15.37:0/3858488209 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c018990 con 0x7fc2a8069330 2024-03-26T06:39:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.609+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3858488209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a8100440 con 0x7fc2a8069330 2024-03-26T06:39:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.612+0000 7fc2a4ff9640 1 -- 172.21.15.37:0/3858488209 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc29c010ac0 con 0x7fc2a8069330 2024-03-26T06:39:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.612+0000 7fc2b0d9b740 1 Processor -- start 2024-03-26T06:39:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.612+0000 7fc2b0d9b740 1 -- start start 2024-03-26T06:39:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.612+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.612+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e3fd0 0x55c9725e4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.612+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c9725ef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.612+0000 7fc2b0d9b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9725485f0 con 0x55c9725e4d40 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.612+0000 7fc2b0d9b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9724f1930 con 0x55c9725e3fd0 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.612+0000 7fc2b0d9b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c972544690 con 0x55c9725ed550 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a4ff9640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc2900a2a10 0x7fc2900a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a6ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c9725ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41084/0 (socket says 172.21.15.37:41084) 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a7fff640 1 -- 172.21.15.37:0/1845418476 learned_addr learned my addr 172.21.15.37:0/1845418476 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a4ff9640 1 -- 172.21.15.37:0/3858488209 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc29c0cf1f0 con 0x7fc2a8069330 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/1845418476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e3fd0 0x55c9725e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a7fff640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc2900a2a10 0x7fc2900a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a7fff640 1 -- 172.21.15.37:0/1845418476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 msgr2=0x55c9725ef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a7fff640 1 --2- 172.21.15.37:0/1845418476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c9725ef940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a7fff640 1 -- 172.21.15.37:0/1845418476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e3fd0 msgr2=0x55c9725e4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a7fff640 1 --2- 172.21.15.37:0/1845418476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e3fd0 0x55c9725e4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.613+0000 7fc2a7fff640 1 -- 172.21.15.37:0/1845418476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c97225d210 con 0x55c9725e4d40 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.614+0000 7fc2a7fff640 1 --2- 172.21.15.37:0/1845418476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e3a90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0037040 tx=0x7fc2a0032620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ed47a4ec43813c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.614+0000 7fc28dffb640 1 -- 172.21.15.37:0/1845418476 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0050070 con 0x55c9725e4d40 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.614+0000 7fc28dffb640 1 -- 172.21.15.37:0/1845418476 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2a0032ba0 con 0x55c9725e4d40 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.614+0000 7fc28dffb640 1 -- 172.21.15.37:0/1845418476 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0004830 con 0x55c9725e4d40 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.615+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/1845418476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 msgr2=0x55c9725e3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.615+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/1845418476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e3a90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0037040 tx=0x7fc2a0032620 comp rx=0 tx=0).stop 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.615+0000 7fc2a7fff640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc2900a2a10 0x7fc2900a4ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc2a004d000 tx=0x7fc2a0034540 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.615+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/1845418476 shutdown_connections 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.615+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/1845418476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c9725ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.615+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/1845418476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e3fd0 0x55c9725e4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.615+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/1845418476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e3a90 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.615+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/1845418476 >> 172.21.15.37:0/1845418476 conn(0x55c9725ab5b0 msgr2=0x55c9725decc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.615+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/1845418476 shutdown_connections 2024-03-26T06:39:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.615+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/1845418476 wait complete. 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.615+0000 7fc2b0d9b740 1 -- start start 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.616+0000 7fc2b0d9b740 1 Processor -- start 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.616+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725e3fd0 0x55c9725ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.616+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725eab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.616+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725ed550 0x55c9725eb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.616+0000 7fc2a6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.616+0000 7fc2a77fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725e3fd0 0x55c9725ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.616+0000 7fc2a6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725eab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41094/0 (socket says 172.21.15.37:41094) 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.616+0000 7fc2a77fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725e3fd0 0x55c9725ea5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:46664/0 (socket says 172.21.15.37:46664) 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.616+0000 7fc2a6ffd640 1 -- 172.21.15.37:0/3980167536 learned_addr learned my addr 172.21.15.37:0/3980167536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.616+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/3980167536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9725f0280 con 0x55c9725e4d40 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.617+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/3980167536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c972504d90 con 0x55c9725ed550 2024-03-26T06:39:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.617+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/3980167536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9725f0400 con 0x55c9725e3fd0 2024-03-26T06:39:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.617+0000 7fc2b0d9b740 10 client.0 _ll_register_callbacks cb 0x55c97225ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:39:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.617+0000 7fc2a77fe640 1 -- 172.21.15.37:0/3980167536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725ed550 msgr2=0x55c9725eb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.617+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/3980167536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725ed550 0x55c9725eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.617+0000 7fc2a77fe640 1 -- 172.21.15.37:0/3980167536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 msgr2=0x55c9725eab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.617+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/3980167536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725eab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.617+0000 7fc2a77fe640 1 -- 172.21.15.37:0/3980167536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c972329620 con 0x55c9725e3fd0 2024-03-26T06:39:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.617+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/3980167536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725e3fd0 0x55c9725ea5d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc298005d00 tx=0x7fc298012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2071f246afeac075 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.617+0000 7fc286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:39:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc286ffd640 1 -- 172.21.15.37:0/3980167536 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298004220 con 0x55c9725e3fd0 2024-03-26T06:39:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc286ffd640 1 -- 172.21.15.37:0/3980167536 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc2980043c0 con 0x55c9725e3fd0 2024-03-26T06:39:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/3980167536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2a0032ba0 con 0x55c9725e3fd0 2024-03-26T06:39:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc286ffd640 1 -- 172.21.15.37:0/3980167536 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc298005020 con 0x55c9725e3fd0 2024-03-26T06:39:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc2b0d9b740 10 client.26212 fetch_fsmap learned FSMap version 67 2024-03-26T06:39:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/3980167536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55c9725fc5d0 con 0x55c9725e3fd0 2024-03-26T06:39:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc286ffd640 1 -- 172.21.15.37:0/3980167536 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2a0032ba0 con 0x55c9725e3fd0 2024-03-26T06:39:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc286ffd640 1 -- 172.21.15.37:0/3980167536 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 67) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc298019030 con 0x55c9725e3fd0 2024-03-26T06:39:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc2b0d9b740 10 client.26212 fetch_fsmap finished waiting for FSMap version 67 2024-03-26T06:39:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc2b0d9b740 10 client.26212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:39:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc2b0d9b740 20 client.26212 populate_metadata read hostname 'smithi037' 2024-03-26T06:39:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.618+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/3980167536 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c9725fda90 0x55c9725ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.619+0000 7fc2b0d9b740 4 client.26212 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:39:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.619+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/3980167536 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c972555b30 con 0x55c9725fda90 2024-03-26T06:39:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.619+0000 7fc2a6ffd640 1 --2- 172.21.15.37:0/3980167536 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c9725fda90 0x55c9725ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.619+0000 7fc286ffd640 10 client.26212 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.619+0000 7fc2a6ffd640 1 --2- 172.21.15.37:0/3980167536 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c9725fda90 0x55c9725ffec0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81be9e1e0c4018d server_cookie=c42704bc2ab6ec09 in_seq=0 out_seq=0 2024-03-26T06:39:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.623+0000 7fc286ffd640 1 -- 172.21.15.37:0/3980167536 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c972555b30 con 0x55c9725fda90 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.623+0000 7fc286ffd640 10 client.26212 handle_command_reply: tid=0 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.623+0000 7fc2b0d9b740 1 client.26212 shutdown 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.623+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/3980167536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725e3fd0 msgr2=0x55c9725ea5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.623+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/3980167536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725e3fd0 0x55c9725ea5d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc298005d00 tx=0x7fc298012820 comp rx=0 tx=0).stop 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.624+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/3980167536 shutdown_connections 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.624+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/3980167536 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c9725fda90 0x55c9725ffec0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.624+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/3980167536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725ed550 0x55c9725eb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.624+0000 7fc286ffd640 0 client.26212 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.624+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/3980167536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725eab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.624+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/3980167536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725e3fd0 0x55c9725ea5d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.624+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/3980167536 >> 172.21.15.37:0/3980167536 conn(0x55c9725ab5b0 msgr2=0x55c9725e0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.624+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/3980167536 shutdown_connections 2024-03-26T06:39:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.624+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/3980167536 wait complete. 2024-03-26T06:39:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.624+0000 7fc2b0d9b740 20 client.26212 trim_cache size 0 max 16384 2024-03-26T06:39:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.645+0000 7fc2b0d9b740 1 Processor -- start 2024-03-26T06:39:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.645+0000 7fc2b0d9b740 1 -- start start 2024-03-26T06:39:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.645+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e3fd0 0x55c9725ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.645+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.645+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c9725e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.645+0000 7fc2b0d9b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81aaba0 con 0x55c9725e4d40 2024-03-26T06:39:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.645+0000 7fc2b0d9b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81ac550 con 0x55c9725e3fd0 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.645+0000 7fc2b0d9b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81ab230 con 0x55c9725ed550 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.645+0000 7fc2a6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc2a6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41096/0 (socket says 172.21.15.37:41096) 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc2a6ffd640 1 -- 172.21.15.37:0/874176659 learned_addr learned my addr 172.21.15.37:0/874176659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/874176659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e3fd0 0x55c9725ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc2a7fff640 1 --2- 172.21.15.37:0/874176659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c9725e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc2a77fe640 1 -- 172.21.15.37:0/874176659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 msgr2=0x55c9725e7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/874176659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c9725e7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc2a77fe640 1 -- 172.21.15.37:0/874176659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 msgr2=0x55c9725e7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/874176659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc2a77fe640 1 -- 172.21.15.37:0/874176659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9725e32a0 con 0x55c9725e3fd0 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc2a6ffd640 1 --2- 172.21.15.37:0/874176659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:39:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/874176659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e3fd0 0x55c9725ecd00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc298006010 tx=0x7fc29800b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81dd4a2dc71581b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.646+0000 7fc28d7fa640 1 -- 172.21.15.37:0/874176659 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298000e60 con 0x55c9725e3fd0 2024-03-26T06:39:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.647+0000 7fc28d7fa640 1 -- 172.21.15.37:0/874176659 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc298010c50 con 0x55c9725e3fd0 2024-03-26T06:39:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.647+0000 7fc28d7fa640 1 -- 172.21.15.37:0/874176659 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298004660 con 0x55c9725e3fd0 2024-03-26T06:39:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.647+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/874176659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e3fd0 msgr2=0x55c9725ecd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.647+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/874176659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e3fd0 0x55c9725ecd00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc298006010 tx=0x7fc29800b710 comp rx=0 tx=0).stop 2024-03-26T06:39:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.647+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/874176659 shutdown_connections 2024-03-26T06:39:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.647+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/874176659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c9725e7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.648+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/874176659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e4d40 0x55c9725e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.648+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/874176659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e3fd0 0x55c9725ecd00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.648+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/874176659 >> 172.21.15.37:0/874176659 conn(0x55c9725ab5b0 msgr2=0x55c9725e8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.648+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/874176659 shutdown_connections 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.648+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/874176659 wait complete. 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.648+0000 7fc2b0d9b740 1 -- start start 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.648+0000 7fc2b0d9b740 1 Processor -- start 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.649+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e3fd0 0x55c9725f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.649+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e4d40 0x55c9725f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.649+0000 7fc2b0d9b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c972653ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.649+0000 7fc2b0d9b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81abef0 con 0x55c9725e3fd0 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.649+0000 7fc2b0d9b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81ab890 con 0x55c9725e4d40 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.649+0000 7fc2b0d9b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81a8e60 con 0x55c9725ed550 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.649+0000 7fc2b0d9b740 10 client.0 _ll_register_callbacks cb 0x7fc2a0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:39:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.649+0000 7fc2a77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e3fd0 0x55c9725f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.649+0000 7fc2a77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e3fd0 0x55c9725f36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41106/0 (socket says 172.21.15.37:41106) 2024-03-26T06:39:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.649+0000 7fc2a77fe640 1 -- 172.21.15.37:0/2697676494 learned_addr learned my addr 172.21.15.37:0/2697676494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.649+0000 7fc2a7fff640 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c972653ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.650+0000 7fc2a77fe640 1 -- 172.21.15.37:0/2697676494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 msgr2=0x55c972653ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.650+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c972653ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.650+0000 7fc2a77fe640 1 -- 172.21.15.37:0/2697676494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e4d40 msgr2=0x55c9725f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.650+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e4d40 0x55c9725f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.650+0000 7fc2a77fe640 1 -- 172.21.15.37:0/2697676494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c972654530 con 0x55c9725e3fd0 2024-03-26T06:39:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.650+0000 7fc2a77fe640 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e3fd0 0x55c9725f36c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc298010a50 tx=0x7fc298010a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b72f0bba0a2827 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.650+0000 7fc286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.650+0000 7fc286ffd640 1 -- 172.21.15.37:0/2697676494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2980051c0 con 0x55c9725e3fd0 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.650+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/2697676494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc298010c50 con 0x55c9725e3fd0 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc286ffd640 1 -- 172.21.15.37:0/2697676494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc298004660 con 0x55c9725e3fd0 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc286ffd640 1 -- 172.21.15.37:0/2697676494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc29803a050 con 0x55c9725e3fd0 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc2b0d9b740 10 client.16830 fetch_fsmap learned FSMap version 67 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/2697676494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55c972655740 con 0x55c9725e3fd0 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc286ffd640 1 -- 172.21.15.37:0/2697676494 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc298010c50 con 0x55c9725e3fd0 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc286ffd640 1 -- 172.21.15.37:0/2697676494 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc29803d020 con 0x55c9725e3fd0 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc2b0d9b740 10 client.16830 fetch_fsmap finished waiting for FSMap version 67 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc2b0d9b740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc2b0d9b740 20 client.16830 populate_metadata read hostname 'smithi037' 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c9725fda90 0x55c972657d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc2b0d9b740 4 client.16830 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:39:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.651+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/2697676494 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c97254fe80 con 0x55c9725fda90 2024-03-26T06:39:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.652+0000 7fc2a6ffd640 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c9725fda90 0x55c972657d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.653+0000 7fc2a6ffd640 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c9725fda90 0x55c972657d60 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a592cc637b08b64a server_cookie=96726fd00c4e27fd in_seq=0 out_seq=0 2024-03-26T06:39:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.653+0000 7fc286ffd640 10 client.16830 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:17.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 145439 38.37 MB/sec execute 52 sec latency 110.769 ms 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.891+0000 7fc286ffd640 1 -- 172.21.15.37:0/2697676494 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c97254fe80 con 0x55c9725fda90 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.891+0000 7fc286ffd640 10 client.16830 handle_command_reply: tid=0 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.891+0000 7fc2b0d9b740 1 client.16830 shutdown 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.891+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/2697676494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e3fd0 msgr2=0x55c9725f36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.891+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e3fd0 0x55c9725f36c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc298010a50 tx=0x7fc298010a80 comp rx=0 tx=0).stop 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.892+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/2697676494 shutdown_connections 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.892+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55c9725fda90 0x55c972657d60 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.892+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9725ed550 0x55c972653ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.892+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c9725e4d40 0x55c9725f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.892+0000 7fc286ffd640 0 client.16830 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.892+0000 7fc2b0d9b740 1 --2- 172.21.15.37:0/2697676494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9725e3fd0 0x55c9725f36c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.892+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/2697676494 >> 172.21.15.37:0/2697676494 conn(0x55c9725ab5b0 msgr2=0x55c9725dde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.892+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/2697676494 shutdown_connections 2024-03-26T06:39:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.892+0000 7fc2b0d9b740 1 -- 172.21.15.37:0/2697676494 wait complete. 2024-03-26T06:39:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.892+0000 7fc2b0d9b740 20 client.16830 trim_cache size 0 max 16384 2024-03-26T06:39:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:39:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:39:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:39:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3858488209 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc2900a2a10 msgr2=0x7fc2900a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc2900a2a10 0x7fc2900a4ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc2a004d000 tx=0x7fc2a0034540 comp rx=0 tx=0).stop 2024-03-26T06:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3858488209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2a8069330 msgr2=0x7fc2a819dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2a8069330 0x7fc2a819dc40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00e9f0 tx=0x7fc29c00eeb0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3858488209 shutdown_connections 2024-03-26T06:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc2900a2a10 0x7fc2900a4ed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a8107400 0x7fc2a8197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2a8069330 0x7fc2a819dc40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 --2- 172.21.15.37:0/3858488209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2a8068970 0x7fc2a819d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3858488209 >> 172.21.15.37:0/3858488209 conn(0x7fc2a8077040 msgr2=0x7fc2a80fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3858488209 shutdown_connections 2024-03-26T06:39:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:17.893+0000 7fc2ae1c6640 1 -- 172.21.15.37:0/3858488209 wait complete. 2024-03-26T06:39:18.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:18 smithi037 ceph-mon[31974]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 771 op/s 2024-03-26T06:39:18.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:18 smithi066 ceph-mon[36504]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 771 op/s 2024-03-26T06:39:18.661 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:18 smithi149 ceph-mon[36681]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 771 op/s 2024-03-26T06:39:18.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 146515 38.90 MB/sec execute 53 sec latency 104.960 ms 2024-03-26T06:39:19.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 147338 38.96 MB/sec execute 54 sec latency 57.742 ms 2024-03-26T06:39:20.851 INFO:tasks.workunit.client.0.smithi037.stdout: 10 148321 39.36 MB/sec execute 55 sec latency 82.654 ms 2024-03-26T06:39:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:20 smithi066 ceph-mon[36504]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 771 op/s 2024-03-26T06:39:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:20 smithi066 ceph-mon[36504]: osdmap e112: 12 total, 12 up, 12 in 2024-03-26T06:39:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:20 smithi149 ceph-mon[36681]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 771 op/s 2024-03-26T06:39:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:20 smithi149 ceph-mon[36681]: osdmap e112: 12 total, 12 up, 12 in 2024-03-26T06:39:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:20 smithi037 ceph-mon[31974]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 771 op/s 2024-03-26T06:39:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:20 smithi037 ceph-mon[31974]: osdmap e112: 12 total, 12 up, 12 in 2024-03-26T06:39:21.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 149034 39.20 MB/sec execute 56 sec latency 142.618 ms 2024-03-26T06:39:22.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 149293 38.70 MB/sec execute 57 sec latency 635.768 ms 2024-03-26T06:39:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:22 smithi066 ceph-mon[36504]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 9.4 MiB/s wr, 758 op/s 2024-03-26T06:39:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:22 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:39:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:22 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:39:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:22 smithi149 ceph-mon[36681]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 9.4 MiB/s wr, 758 op/s 2024-03-26T06:39:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:22 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:39:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:22 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:39:23.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:22 smithi037 ceph-mon[31974]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 9.4 MiB/s wr, 758 op/s 2024-03-26T06:39:23.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:22 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:39:23.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:22 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:39:23.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 149344 38.09 MB/sec execute 58 sec latency 1355.843 ms 2024-03-26T06:39:24.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:23 smithi149 ceph-mon[36681]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 9.4 MiB/s wr, 758 op/s 2024-03-26T06:39:24.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:23 smithi037 ceph-mon[31974]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 9.4 MiB/s wr, 758 op/s 2024-03-26T06:39:24.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:23 smithi066 ceph-mon[36504]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 9.4 MiB/s wr, 758 op/s 2024-03-26T06:39:24.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 149764 37.83 MB/sec execute 59 sec latency 256.571 ms 2024-03-26T06:39:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:24 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:39:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:24 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:39:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:24 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:39:25.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:24 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:39:25.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:24 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:39:25.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:24 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:39:25.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:24 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:39:25.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:24 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:39:25.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:24 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:39:25.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 150136 37.58 MB/sec execute 60 sec latency 554.188 ms 2024-03-26T06:39:26.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:25 smithi149 ceph-mon[36681]: pgmap v520: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 855 op/s 2024-03-26T06:39:26.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:25 smithi037 ceph-mon[31974]: pgmap v520: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 855 op/s 2024-03-26T06:39:26.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:25 smithi066 ceph-mon[36504]: pgmap v520: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 855 op/s 2024-03-26T06:39:26.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 150998 37.77 MB/sec execute 61 sec latency 78.403 ms 2024-03-26T06:39:27.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 151759 37.90 MB/sec execute 62 sec latency 58.343 ms 2024-03-26T06:39:28.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:28 smithi066 ceph-mon[36504]: pgmap v521: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.11k op/s 2024-03-26T06:39:28.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:28 smithi149 ceph-mon[36681]: pgmap v521: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.11k op/s 2024-03-26T06:39:28.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:28 smithi037 ceph-mon[31974]: pgmap v521: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.11k op/s 2024-03-26T06:39:28.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 152482 37.74 MB/sec execute 63 sec latency 108.723 ms 2024-03-26T06:39:29.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 153694 38.20 MB/sec execute 64 sec latency 60.964 ms 2024-03-26T06:39:30.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 154797 38.81 MB/sec execute 65 sec latency 47.551 ms 2024-03-26T06:39:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.029+0000 7fd2b19a9640 1 -- 172.21.15.37:0/1173179972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2ac068970 msgr2=0x7fd2ac068dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.029+0000 7fd2b19a9640 1 --2- 172.21.15.37:0/1173179972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2ac068970 0x7fd2ac068dd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd294009900 tx=0x7fd29402bcd0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.029+0000 7fd2b19a9640 1 -- 172.21.15.37:0/1173179972 shutdown_connections 2024-03-26T06:39:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.029+0000 7fd2b19a9640 1 --2- 172.21.15.37:0/1173179972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ac069310 0x7fd2ac10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.029+0000 7fd2b19a9640 1 --2- 172.21.15.37:0/1173179972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2ac068970 0x7fd2ac068dd0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.029+0000 7fd2b19a9640 1 --2- 172.21.15.37:0/1173179972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac1073e0 0x7fd2ac1077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.029+0000 7fd2b19a9640 1 -- 172.21.15.37:0/1173179972 >> 172.21.15.37:0/1173179972 conn(0x7fd2ac077040 msgr2=0x7fd2ac077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.029+0000 7fd2b19a9640 1 -- 172.21.15.37:0/1173179972 shutdown_connections 2024-03-26T06:39:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.029+0000 7fd2b19a9640 1 -- 172.21.15.37:0/1173179972 wait complete. 2024-03-26T06:39:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.030+0000 7fd2b19a9640 1 Processor -- start 2024-03-26T06:39:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.030+0000 7fd2b19a9640 1 -- start start 2024-03-26T06:39:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.030+0000 7fd2b19a9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ac068970 0x7fd2ac19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.030+0000 7fd2b19a9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac069310 0x7fd2ac19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.030+0000 7fd2b19a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2ac1073e0 0x7fd2ac199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.030+0000 7fd2b19a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac111640 con 0x7fd2ac1073e0 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.030+0000 7fd2b19a9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac1114c0 con 0x7fd2ac068970 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.030+0000 7fd2b19a9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac1117c0 con 0x7fd2ac069310 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.030+0000 7fd2a27fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac069310 0x7fd2ac19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.030+0000 7fd2a27fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac069310 0x7fd2ac19fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:58502/0 (socket says 172.21.15.37:58502) 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.030+0000 7fd2a27fc640 1 -- 172.21.15.37:0/2485380645 learned_addr learned my addr 172.21.15.37:0/2485380645 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2aaffd640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ac068970 0x7fd2ac19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2ab7fe640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2ac1073e0 0x7fd2ac199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2a27fc640 1 -- 172.21.15.37:0/2485380645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ac068970 msgr2=0x7fd2ac19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2a27fc640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ac068970 0x7fd2ac19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2a27fc640 1 -- 172.21.15.37:0/2485380645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2ac1073e0 msgr2=0x7fd2ac199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2a27fc640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2ac1073e0 0x7fd2ac199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2a27fc640 1 -- 172.21.15.37:0/2485380645 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac19a010 con 0x7fd2ac069310 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2ab7fe640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2ac1073e0 0x7fd2ac199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:39:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2a27fc640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac069310 0x7fd2ac19fc10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd29402efd0 tx=0x7fd29402cec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2b09a7640 1 -- 172.21.15.37:0/2485380645 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294004040 con 0x7fd2ac069310 2024-03-26T06:39:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2b19a9640 1 -- 172.21.15.37:0/2485380645 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac19a2a0 con 0x7fd2ac069310 2024-03-26T06:39:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2b09a7640 1 -- 172.21.15.37:0/2485380645 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2940054e0 con 0x7fd2ac069310 2024-03-26T06:39:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2b19a9640 1 -- 172.21.15.37:0/2485380645 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac1080c0 con 0x7fd2ac069310 2024-03-26T06:39:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.031+0000 7fd2b09a7640 1 -- 172.21.15.37:0/2485380645 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294003ad0 con 0x7fd2ac069310 2024-03-26T06:39:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.034+0000 7fd2b4584740 1 Processor -- start 2024-03-26T06:39:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.034+0000 7fd2b4584740 1 -- start start 2024-03-26T06:39:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.034+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91063d40 0x55ef91062a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.034+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 0x55ef91063450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.034+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef9106c550 0x55ef9106e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.034+0000 7fd2b4584740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef90fc75f0 con 0x55ef9106c550 2024-03-26T06:39:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.034+0000 7fd2b4584740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef90f70930 con 0x55ef91062fd0 2024-03-26T06:39:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.034+0000 7fd2b4584740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef90fc3690 con 0x55ef91063d40 2024-03-26T06:39:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2aaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 0x55ef91063450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2ab7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91063d40 0x55ef91062a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2aaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 0x55ef91063450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:36888/0 (socket says 172.21.15.37:36888) 2024-03-26T06:39:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2aaffd640 1 -- 172.21.15.37:0/1399100754 learned_addr learned my addr 172.21.15.37:0/1399100754 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2b09a7640 1 -- 172.21.15.37:0/2485380645 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fd2940041e0 con 0x7fd2ac069310 2024-03-26T06:39:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2ab7fe640 1 -- 172.21.15.37:0/1399100754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 msgr2=0x55ef91063450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2ab7fe640 1 --2- 172.21.15.37:0/1399100754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 0x55ef91063450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2ab7fe640 1 -- 172.21.15.37:0/1399100754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef9106c550 msgr2=0x55ef9106e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2ab7fe640 1 --2- 172.21.15.37:0/1399100754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef9106c550 0x55ef9106e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2ab7fe640 1 -- 172.21.15.37:0/1399100754 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef90cdc210 con 0x55ef91063d40 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2ab7fe640 1 --2- 172.21.15.37:0/1399100754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91063d40 0x55ef91062a90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009f40 tx=0x7fd2a4011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=269b6fec3334c91e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2a1ffb640 1 -- 172.21.15.37:0/1399100754 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4004d20 con 0x55ef91063d40 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2a1ffb640 1 -- 172.21.15.37:0/1399100754 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2a40129f0 con 0x55ef91063d40 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.035+0000 7fd2a1ffb640 1 -- 172.21.15.37:0/1399100754 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4004d20 con 0x55ef91063d40 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.036+0000 7fd2b09a7640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fd2980a2a10 0x7fd2980a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.036+0000 7fd2b09a7640 1 -- 172.21.15.37:0/2485380645 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd2940f8ec0 con 0x7fd2ac069310 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.036+0000 7fd2aaffd640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fd2980a2a10 0x7fd2980a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.037+0000 7fd2b4584740 1 -- 172.21.15.37:0/1399100754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91063d40 msgr2=0x55ef91062a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.037+0000 7fd2b4584740 1 --2- 172.21.15.37:0/1399100754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91063d40 0x55ef91062a90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009f40 tx=0x7fd2a4011870 comp rx=0 tx=0).stop 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.037+0000 7fd2aaffd640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fd2980a2a10 0x7fd2980a4ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55ef910644a0 tx=0x7fd29c005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.037+0000 7fd2b4584740 1 -- 172.21.15.37:0/1399100754 shutdown_connections 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.037+0000 7fd2b4584740 1 --2- 172.21.15.37:0/1399100754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef9106c550 0x55ef9106e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.037+0000 7fd2b4584740 1 --2- 172.21.15.37:0/1399100754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 0x55ef91063450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.037+0000 7fd2b4584740 1 --2- 172.21.15.37:0/1399100754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91063d40 0x55ef91062a90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.037+0000 7fd2b4584740 1 -- 172.21.15.37:0/1399100754 >> 172.21.15.37:0/1399100754 conn(0x55ef9102a5b0 msgr2=0x55ef9105dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.037+0000 7fd2b4584740 1 -- 172.21.15.37:0/1399100754 shutdown_connections 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.037+0000 7fd2b4584740 1 -- 172.21.15.37:0/1399100754 wait complete. 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.037+0000 7fd2b4584740 1 -- start start 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2b4584740 1 Processor -- start 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 0x55ef91069530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 0x55ef91069a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef9106c550 0x55ef91069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2b4584740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef91091210 con 0x55ef91063d40 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2b4584740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef90f83d90 con 0x55ef9106c550 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2b4584740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef91091390 con 0x55ef91062fd0 2024-03-26T06:39:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2b4584740 10 client.0 _ll_register_callbacks cb 0x55ef90cdba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2aaffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 0x55ef91069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef9106c550 0x55ef91069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2aaffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 0x55ef91069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:58510/0 (socket says 172.21.15.37:58510) 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef9106c550 0x55ef91069fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:36890/0 (socket says 172.21.15.37:36890) 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2aaffd640 1 -- 172.21.15.37:0/3169368026 learned_addr learned my addr 172.21.15.37:0/3169368026 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2a27fc640 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 0x55ef91069a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2aaffd640 1 -- 172.21.15.37:0/3169368026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef9106c550 msgr2=0x55ef91069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2aaffd640 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef9106c550 0x55ef91069fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2aaffd640 1 -- 172.21.15.37:0/3169368026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 msgr2=0x55ef91069a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2aaffd640 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 0x55ef91069a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2aaffd640 1 -- 172.21.15.37:0/3169368026 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef90da8620 con 0x55ef91062fd0 2024-03-26T06:39:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.038+0000 7fd2a27fc640 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 0x55ef91069a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.039+0000 7fd2aaffd640 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 0x55ef91069530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd29c004620 tx=0x7fd29c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f57c378586788e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.039+0000 7fd28affd640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.039+0000 7fd28affd640 1 -- 172.21.15.37:0/3169368026 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c01f640 con 0x55ef91062fd0 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.039+0000 7fd2b4584740 1 -- 172.21.15.37:0/3169368026 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2a40129f0 con 0x55ef91062fd0 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.039+0000 7fd28affd640 1 -- 172.21.15.37:0/3169368026 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd29c01f7e0 con 0x55ef91062fd0 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.039+0000 7fd28affd640 1 -- 172.21.15.37:0/3169368026 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd29c01f9e0 con 0x55ef91062fd0 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.039+0000 7fd2b4584740 10 client.26230 fetch_fsmap learned FSMap version 68 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.039+0000 7fd2b4584740 1 -- 172.21.15.37:0/3169368026 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55ef9109d600 con 0x55ef91062fd0 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.040+0000 7fd28affd640 1 -- 172.21.15.37:0/3169368026 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2a40129f0 con 0x55ef91062fd0 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.040+0000 7fd28affd640 1 -- 172.21.15.37:0/3169368026 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 68) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd29c027020 con 0x55ef91062fd0 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.040+0000 7fd2b4584740 10 client.26230 fetch_fsmap finished waiting for FSMap version 68 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.040+0000 7fd2b4584740 10 client.26230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.040+0000 7fd2b4584740 20 client.26230 populate_metadata read hostname 'smithi037' 2024-03-26T06:39:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.040+0000 7fd2b4584740 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ef9109ea70 0x55ef910a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.040+0000 7fd2b4584740 4 client.26230 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:39:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.040+0000 7fd2b4584740 1 -- 172.21.15.37:0/3169368026 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef90fd4b30 con 0x55ef9109ea70 2024-03-26T06:39:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.040+0000 7fd2a27fc640 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ef9109ea70 0x55ef910a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.041+0000 7fd28affd640 10 client.26230 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.041+0000 7fd2a27fc640 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ef9109ea70 0x55ef910a0e60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ffeb30bf54144b7 server_cookie=5c011ae8a9186753 in_seq=0 out_seq=0 2024-03-26T06:39:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.041+0000 7fd28affd640 1 -- 172.21.15.37:0/3169368026 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef90fd4b30 con 0x55ef9109ea70 2024-03-26T06:39:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.041+0000 7fd28affd640 10 client.26230 handle_command_reply: tid=0 2024-03-26T06:39:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 1 client.26230 shutdown 2024-03-26T06:39:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 1 -- 172.21.15.37:0/3169368026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 msgr2=0x55ef91069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 0x55ef91069530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd29c004620 tx=0x7fd29c004730 comp rx=0 tx=0).stop 2024-03-26T06:39:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 1 -- 172.21.15.37:0/3169368026 shutdown_connections 2024-03-26T06:39:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ef9109ea70 0x55ef910a0e60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef9106c550 0x55ef91069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd28affd640 0 client.26230 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 0x55ef91069a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 1 --2- 172.21.15.37:0/3169368026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 0x55ef91069530 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 1 -- 172.21.15.37:0/3169368026 >> 172.21.15.37:0/3169368026 conn(0x55ef9102a5b0 msgr2=0x55ef9105fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 1 -- 172.21.15.37:0/3169368026 shutdown_connections 2024-03-26T06:39:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 1 -- 172.21.15.37:0/3169368026 wait complete. 2024-03-26T06:39:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.042+0000 7fd2b4584740 20 client.26230 trim_cache size 0 max 16384 2024-03-26T06:39:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2b4584740 1 Processor -- start 2024-03-26T06:39:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2b4584740 1 -- start start 2024-03-26T06:39:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 0x55ef9106bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91063d40 0x55ef91067e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef9106c550 0x55ef91068340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2b4584740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac06ef50 con 0x55ef9106c550 2024-03-26T06:39:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2b4584740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0708d0 con 0x55ef91063d40 2024-03-26T06:39:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2b4584740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac06f5b0 con 0x55ef91062fd0 2024-03-26T06:39:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2ab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef9106c550 0x55ef91068340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2aaffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 0x55ef9106bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2ab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef9106c550 0x55ef91068340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44384/0 (socket says 172.21.15.37:44384) 2024-03-26T06:39:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2ab7fe640 1 -- 172.21.15.37:0/3996677719 learned_addr learned my addr 172.21.15.37:0/3996677719 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.059+0000 7fd2a27fc640 1 --2- 172.21.15.37:0/3996677719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91063d40 0x55ef91067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.060+0000 7fd2ab7fe640 1 -- 172.21.15.37:0/3996677719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 msgr2=0x55ef9106bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.060+0000 7fd2ab7fe640 1 --2- 172.21.15.37:0/3996677719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 0x55ef9106bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.060+0000 7fd2ab7fe640 1 -- 172.21.15.37:0/3996677719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91063d40 msgr2=0x55ef91067e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.060+0000 7fd2ab7fe640 1 --2- 172.21.15.37:0/3996677719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91063d40 0x55ef91067e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.060+0000 7fd2ab7fe640 1 -- 172.21.15.37:0/3996677719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef9106ecb0 con 0x55ef9106c550 2024-03-26T06:39:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.060+0000 7fd2ab7fe640 1 --2- 172.21.15.37:0/3996677719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef9106c550 0x55ef91068340 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009f40 tx=0x7fd2a40099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40eb7e4c45b4af28 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.061+0000 7fd2a17fa640 1 -- 172.21.15.37:0/3996677719 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a401f040 con 0x55ef9106c550 2024-03-26T06:39:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.061+0000 7fd2a17fa640 1 -- 172.21.15.37:0/3996677719 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2a40052b0 con 0x55ef9106c550 2024-03-26T06:39:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.061+0000 7fd2a17fa640 1 -- 172.21.15.37:0/3996677719 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40128d0 con 0x55ef9106c550 2024-03-26T06:39:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.062+0000 7fd2b4584740 1 -- 172.21.15.37:0/3996677719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef9106c550 msgr2=0x55ef91068340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.062+0000 7fd2b4584740 1 --2- 172.21.15.37:0/3996677719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef9106c550 0x55ef91068340 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009f40 tx=0x7fd2a40099c0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.062+0000 7fd2b4584740 1 -- 172.21.15.37:0/3996677719 shutdown_connections 2024-03-26T06:39:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.062+0000 7fd2b4584740 1 --2- 172.21.15.37:0/3996677719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef9106c550 0x55ef91068340 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.062+0000 7fd2b4584740 1 --2- 172.21.15.37:0/3996677719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91063d40 0x55ef91067e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.062+0000 7fd2b4584740 1 --2- 172.21.15.37:0/3996677719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef91062fd0 0x55ef9106bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.062+0000 7fd2b4584740 1 -- 172.21.15.37:0/3996677719 >> 172.21.15.37:0/3996677719 conn(0x55ef9102a5b0 msgr2=0x55ef910a5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.062+0000 7fd2b4584740 1 -- 172.21.15.37:0/3996677719 shutdown_connections 2024-03-26T06:39:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.062+0000 7fd2b4584740 1 -- 172.21.15.37:0/3996677719 wait complete. 2024-03-26T06:39:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.062+0000 7fd2b4584740 1 -- start start 2024-03-26T06:39:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.062+0000 7fd2b4584740 1 Processor -- start 2024-03-26T06:39:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 0x55ef91095a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 0x55ef91093dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2b4584740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef9106c550 0x55ef91094310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2b4584740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac070270 con 0x55ef91063d40 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2b4584740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac06fc10 con 0x55ef91062fd0 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2b4584740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac06d230 con 0x55ef9106c550 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2b4584740 10 client.0 _ll_register_callbacks cb 0x7fd2ac06ac40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2ab7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef9106c550 0x55ef91094310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2a27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 0x55ef91093dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2aaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 0x55ef91095a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2a27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 0x55ef91093dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44392/0 (socket says 172.21.15.37:44392) 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2ab7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef9106c550 0x55ef91094310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:58516/0 (socket says 172.21.15.37:58516) 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2a27fc640 1 -- 172.21.15.37:0/661187682 learned_addr learned my addr 172.21.15.37:0/661187682 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2aaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 0x55ef91095a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:36908/0 (socket says 172.21.15.37:36908) 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2aaffd640 1 -- 172.21.15.37:0/661187682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef9106c550 msgr2=0x55ef91094310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2aaffd640 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef9106c550 0x55ef91094310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2aaffd640 1 -- 172.21.15.37:0/661187682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 msgr2=0x55ef91093dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2aaffd640 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 0x55ef91093dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.063+0000 7fd2aaffd640 1 -- 172.21.15.37:0/661187682 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef91094b40 con 0x55ef91062fd0 2024-03-26T06:39:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.064+0000 7fd2ab7fe640 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef9106c550 0x55ef91094310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:39:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.064+0000 7fd2a27fc640 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 0x55ef91093dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:39:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.064+0000 7fd2aaffd640 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 0x55ef91095a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd29c019040 tx=0x7fd29c01fe10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7ce986859b85170 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.064+0000 7fd28affd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:39:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.064+0000 7fd28affd640 1 -- 172.21.15.37:0/661187682 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c041060 con 0x55ef91062fd0 2024-03-26T06:39:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.064+0000 7fd2b4584740 1 -- 172.21.15.37:0/661187682 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2a40052b0 con 0x55ef91062fd0 2024-03-26T06:39:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.064+0000 7fd28affd640 1 -- 172.21.15.37:0/661187682 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd29c029c50 con 0x55ef91062fd0 2024-03-26T06:39:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.064+0000 7fd28affd640 1 -- 172.21.15.37:0/661187682 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd29c029e50 con 0x55ef91062fd0 2024-03-26T06:39:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.064+0000 7fd2b4584740 10 client.26348 fetch_fsmap learned FSMap version 68 2024-03-26T06:39:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.064+0000 7fd2b4584740 1 -- 172.21.15.37:0/661187682 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55ef910f5eb0 con 0x55ef91062fd0 2024-03-26T06:39:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.065+0000 7fd28affd640 1 -- 172.21.15.37:0/661187682 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2a40052b0 con 0x55ef91062fd0 2024-03-26T06:39:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.065+0000 7fd28affd640 1 -- 172.21.15.37:0/661187682 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 68) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd29c032960 con 0x55ef91062fd0 2024-03-26T06:39:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.065+0000 7fd2b4584740 10 client.26348 fetch_fsmap finished waiting for FSMap version 68 2024-03-26T06:39:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.065+0000 7fd2b4584740 10 client.26348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:39:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.065+0000 7fd2b4584740 20 client.26348 populate_metadata read hostname 'smithi037' 2024-03-26T06:39:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.065+0000 7fd2b4584740 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ef910f6520 0x55ef910f8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.065+0000 7fd2b4584740 4 client.26348 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:39:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.065+0000 7fd2b4584740 1 -- 172.21.15.37:0/661187682 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ef90fcee80 con 0x55ef910f6520 2024-03-26T06:39:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.066+0000 7fd2a27fc640 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ef910f6520 0x55ef910f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.066+0000 7fd28affd640 10 client.26348 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.066+0000 7fd2a27fc640 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ef910f6520 0x55ef910f8910 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d69f7e1327a1d8c7 server_cookie=9e878f30b177e552 in_seq=0 out_seq=0 2024-03-26T06:39:31.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:30 smithi066 ceph-mon[36504]: pgmap v522: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.11k op/s 2024-03-26T06:39:31.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:30 smithi149 ceph-mon[36681]: pgmap v522: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.11k op/s 2024-03-26T06:39:31.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:30 smithi037 ceph-mon[31974]: pgmap v522: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.11k op/s 2024-03-26T06:39:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.272+0000 7fd28affd640 1 -- 172.21.15.37:0/661187682 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ef90fcee80 con 0x55ef910f6520 2024-03-26T06:39:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.272+0000 7fd28affd640 10 client.26348 handle_command_reply: tid=0 2024-03-26T06:39:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.272+0000 7fd2b4584740 1 client.26348 shutdown 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.272+0000 7fd2b4584740 1 -- 172.21.15.37:0/661187682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 msgr2=0x55ef91095a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.272+0000 7fd2b4584740 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 0x55ef91095a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd29c019040 tx=0x7fd29c01fe10 comp rx=0 tx=0).stop 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.273+0000 7fd2b4584740 1 -- 172.21.15.37:0/661187682 shutdown_connections 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.273+0000 7fd2b4584740 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55ef910f6520 0x55ef910f8910 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.273+0000 7fd28affd640 0 client.26348 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.273+0000 7fd2b4584740 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef9106c550 0x55ef91094310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.273+0000 7fd2b4584740 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef91063d40 0x55ef91093dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.273+0000 7fd2b4584740 1 --2- 172.21.15.37:0/661187682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef91062fd0 0x55ef91095a30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.273+0000 7fd2b4584740 1 -- 172.21.15.37:0/661187682 >> 172.21.15.37:0/661187682 conn(0x55ef9102a5b0 msgr2=0x55ef9105c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.273+0000 7fd2b4584740 1 -- 172.21.15.37:0/661187682 shutdown_connections 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.273+0000 7fd2b4584740 1 -- 172.21.15.37:0/661187682 wait complete. 2024-03-26T06:39:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.274+0000 7fd2b4584740 20 client.26348 trim_cache size 0 max 16384 2024-03-26T06:39:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:39:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:39:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:39:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.274+0000 7fd2b19a9640 1 -- 172.21.15.37:0/2485380645 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fd2980a2a10 msgr2=0x7fd2980a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.274+0000 7fd2b19a9640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fd2980a2a10 0x7fd2980a4ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55ef910644a0 tx=0x7fd29c005f90 comp rx=0 tx=0).stop 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.274+0000 7fd2b19a9640 1 -- 172.21.15.37:0/2485380645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac069310 msgr2=0x7fd2ac19fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.274+0000 7fd2b19a9640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac069310 0x7fd2ac19fc10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd29402efd0 tx=0x7fd29402cec0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.274+0000 7fd2b19a9640 1 -- 172.21.15.37:0/2485380645 shutdown_connections 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.274+0000 7fd2b19a9640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fd2980a2a10 0x7fd2980a4ed0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.274+0000 7fd2b19a9640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2ac1073e0 0x7fd2ac199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.274+0000 7fd2b19a9640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac069310 0x7fd2ac19fc10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.275+0000 7fd2b19a9640 1 --2- 172.21.15.37:0/2485380645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ac068970 0x7fd2ac19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.275+0000 7fd2b19a9640 1 -- 172.21.15.37:0/2485380645 >> 172.21.15.37:0/2485380645 conn(0x7fd2ac077040 msgr2=0x7fd2ac10df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.275+0000 7fd2b19a9640 1 -- 172.21.15.37:0/2485380645 shutdown_connections 2024-03-26T06:39:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:31.275+0000 7fd2b19a9640 1 -- 172.21.15.37:0/2485380645 wait complete. 2024-03-26T06:39:31.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 155383 38.60 MB/sec execute 66 sec latency 106.617 ms 2024-03-26T06:39:32.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 156236 38.43 MB/sec execute 67 sec latency 64.462 ms 2024-03-26T06:39:33.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:32 smithi066 ceph-mon[36504]: pgmap v523: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 26 MiB/s wr, 1.38k op/s 2024-03-26T06:39:33.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:32 smithi149 ceph-mon[36681]: pgmap v523: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 26 MiB/s wr, 1.38k op/s 2024-03-26T06:39:33.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:32 smithi037 ceph-mon[31974]: pgmap v523: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 26 MiB/s wr, 1.38k op/s 2024-03-26T06:39:33.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 157289 38.76 MB/sec execute 68 sec latency 147.668 ms 2024-03-26T06:39:34.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 158270 39.14 MB/sec execute 69 sec latency 96.638 ms 2024-03-26T06:39:35.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:34 smithi066 ceph-mon[36504]: pgmap v524: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 936 op/s 2024-03-26T06:39:35.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:34 smithi149 ceph-mon[36681]: pgmap v524: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 936 op/s 2024-03-26T06:39:35.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:34 smithi037 ceph-mon[31974]: pgmap v524: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 936 op/s 2024-03-26T06:39:35.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 158963 39.08 MB/sec execute 70 sec latency 79.505 ms 2024-03-26T06:39:36.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 159880 39.13 MB/sec execute 71 sec latency 71.275 ms 2024-03-26T06:39:37.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:36 smithi066 ceph-mon[36504]: pgmap v525: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 23 MiB/s wr, 1.10k op/s 2024-03-26T06:39:37.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:36 smithi149 ceph-mon[36681]: pgmap v525: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 23 MiB/s wr, 1.10k op/s 2024-03-26T06:39:37.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:36 smithi037 ceph-mon[31974]: pgmap v525: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 23 MiB/s wr, 1.10k op/s 2024-03-26T06:39:37.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 161082 39.49 MB/sec execute 72 sec latency 53.968 ms 2024-03-26T06:39:38.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 161945 39.78 MB/sec execute 73 sec latency 124.468 ms 2024-03-26T06:39:39.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:38 smithi066 ceph-mon[36504]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-03-26T06:39:39.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:38 smithi149 ceph-mon[36681]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-03-26T06:39:39.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:38 smithi037 ceph-mon[31974]: pgmap v526: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-03-26T06:39:39.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 162610 39.68 MB/sec execute 74 sec latency 98.791 ms 2024-03-26T06:39:40.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 163462 39.60 MB/sec execute 75 sec latency 62.746 ms 2024-03-26T06:39:41.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:40 smithi066 ceph-mon[36504]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.3 MiB/s wr, 734 op/s 2024-03-26T06:39:41.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:40 smithi149 ceph-mon[36681]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.3 MiB/s wr, 734 op/s 2024-03-26T06:39:41.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:40 smithi037 ceph-mon[31974]: pgmap v527: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.3 MiB/s wr, 734 op/s 2024-03-26T06:39:41.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 164586 39.83 MB/sec execute 76 sec latency 67.501 ms 2024-03-26T06:39:42.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:41 smithi066 ceph-mon[36504]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-03-26T06:39:42.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:41 smithi149 ceph-mon[36681]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-03-26T06:39:42.202 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:41 smithi037 ceph-mon[31974]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-03-26T06:39:42.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 165483 40.18 MB/sec execute 77 sec latency 53.530 ms 2024-03-26T06:39:43.852 INFO:tasks.workunit.client.0.smithi037.stdout: 10 165897 39.99 MB/sec execute 78 sec latency 659.780 ms 2024-03-26T06:39:44.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:44 smithi066 ceph-mon[36504]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.4 MiB/s wr, 668 op/s 2024-03-26T06:39:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.407+0000 7fe101281640 1 -- 172.21.15.37:0/2390371643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0fc075820 msgr2=0x7fe0fc075c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.407+0000 7fe101281640 1 --2- 172.21.15.37:0/2390371643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0fc075820 0x7fe0fc075c00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009900 tx=0x7fe0ec02bc30 comp rx=0 tx=0).stop 2024-03-26T06:39:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.407+0000 7fe101281640 1 -- 172.21.15.37:0/2390371643 shutdown_connections 2024-03-26T06:39:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.407+0000 7fe101281640 1 --2- 172.21.15.37:0/2390371643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0fc112170 0x7fe0fc114560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.407+0000 7fe101281640 1 --2- 172.21.15.37:0/2390371643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0fc076140 0x7fe0fc111c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.407+0000 7fe101281640 1 --2- 172.21.15.37:0/2390371643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0fc075820 0x7fe0fc075c00 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.407+0000 7fe101281640 1 -- 172.21.15.37:0/2390371643 >> 172.21.15.37:0/2390371643 conn(0x7fe0fc0fefa0 msgr2=0x7fe0fc1013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.407+0000 7fe101281640 1 -- 172.21.15.37:0/2390371643 shutdown_connections 2024-03-26T06:39:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.407+0000 7fe101281640 1 -- 172.21.15.37:0/2390371643 wait complete. 2024-03-26T06:39:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.408+0000 7fe101281640 1 Processor -- start 2024-03-26T06:39:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.408+0000 7fe101281640 1 -- start start 2024-03-26T06:39:44.413 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:44 smithi149 ceph-mon[36681]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.4 MiB/s wr, 668 op/s 2024-03-26T06:39:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe101281640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0fc075820 0x7fe0fc19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe101281640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0fc076140 0x7fe0fc19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe101281640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0fc112170 0x7fe0fc1a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe101281640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc117cd0 con 0x7fe0fc075820 2024-03-26T06:39:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe101281640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc117b50 con 0x7fe0fc112170 2024-03-26T06:39:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe101281640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc117e50 con 0x7fe0fc076140 2024-03-26T06:39:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0fc075820 0x7fe0fc19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0fc075820 0x7fe0fc19e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55938/0 (socket says 172.21.15.37:55938) 2024-03-26T06:39:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe0faffd640 1 -- 172.21.15.37:0/4216960605 learned_addr learned my addr 172.21.15.37:0/4216960605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0fc076140 0x7fe0fc19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe0faffd640 1 -- 172.21.15.37:0/4216960605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0fc076140 msgr2=0x7fe0fc19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe0faffd640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0fc076140 0x7fe0fc19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe0faffd640 1 -- 172.21.15.37:0/4216960605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0fc112170 msgr2=0x7fe0fc1a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe0faffd640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0fc112170 0x7fe0fc1a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.409+0000 7fe0faffd640 1 -- 172.21.15.37:0/4216960605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc1a3100 con 0x7fe0fc075820 2024-03-26T06:39:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.410+0000 7fe0faffd640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0fc075820 0x7fe0fc19e1b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec005940 tx=0x7fe0ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.410+0000 7fe0e3fff640 1 -- 172.21.15.37:0/4216960605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec041070 con 0x7fe0fc075820 2024-03-26T06:39:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.410+0000 7fe0e3fff640 1 -- 172.21.15.37:0/4216960605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0ec0054e0 con 0x7fe0fc075820 2024-03-26T06:39:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.410+0000 7fe101281640 1 -- 172.21.15.37:0/4216960605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc1a3390 con 0x7fe0fc075820 2024-03-26T06:39:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.411+0000 7fe0e3fff640 1 -- 172.21.15.37:0/4216960605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec003d00 con 0x7fe0fc075820 2024-03-26T06:39:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.411+0000 7fe101281640 1 -- 172.21.15.37:0/4216960605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc1aac80 con 0x7fe0fc075820 2024-03-26T06:39:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.414+0000 7fe0e3fff640 1 -- 172.21.15.37:0/4216960605 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fe0ec044070 con 0x7fe0fc075820 2024-03-26T06:39:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.415+0000 7fe0e3fff640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe0e40a2900 0x7fe0e40a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.415+0000 7fe0e3fff640 1 -- 172.21.15.37:0/4216960605 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe0ec0f0330 con 0x7fe0fc075820 2024-03-26T06:39:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.415+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe0e40a2900 0x7fe0e40a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.415+0000 7fe103e5e740 1 Processor -- start 2024-03-26T06:39:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.415+0000 7fe103e5e740 1 -- start start 2024-03-26T06:39:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 0x55bd76acca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 0x55bd76acd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 0x55bd76ad8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe103e5e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd76a315f0 con 0x55bd76accfd0 2024-03-26T06:39:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe103e5e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd769da930 con 0x55bd76acdd40 2024-03-26T06:39:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe103e5e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd76a2d690 con 0x55bd76ad6550 2024-03-26T06:39:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 0x55bd76acca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 0x55bd76acca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:38382/0 (socket says 172.21.15.37:38382) 2024-03-26T06:39:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0faffd640 1 -- 172.21.15.37:0/3924017630 learned_addr learned my addr 172.21.15.37:0/3924017630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0fb7fe640 1 --2- 172.21.15.37:0/3924017630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 0x55bd76ad8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0faffd640 1 -- 172.21.15.37:0/3924017630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 msgr2=0x55bd76ad8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0faffd640 1 --2- 172.21.15.37:0/3924017630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 0x55bd76ad8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe0e40a2900 0x7fe0e40a4dc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc102b30 tx=0x7fe0f0005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0faffd640 1 -- 172.21.15.37:0/3924017630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 msgr2=0x55bd76acd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/3924017630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 0x55bd76acd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0faffd640 1 --2- 172.21.15.37:0/3924017630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 0x55bd76acd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0faffd640 1 -- 172.21.15.37:0/3924017630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd76746210 con 0x55bd76acdd40 2024-03-26T06:39:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.416+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/3924017630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 0x55bd76acd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:39:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.417+0000 7fe0faffd640 1 --2- 172.21.15.37:0/3924017630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 0x55bd76acca90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0c4cb0 tx=0x7fe0ec0bc650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=febe415ffe76dd30 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.417+0000 7fe0e17fa640 1 -- 172.21.15.37:0/3924017630 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0ef070 con 0x55bd76acdd40 2024-03-26T06:39:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.417+0000 7fe0e17fa640 1 -- 172.21.15.37:0/3924017630 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0ec0bd140 con 0x55bd76acdd40 2024-03-26T06:39:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.417+0000 7fe0e17fa640 1 -- 172.21.15.37:0/3924017630 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec04b040 con 0x55bd76acdd40 2024-03-26T06:39:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.418+0000 7fe103e5e740 1 -- 172.21.15.37:0/3924017630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 msgr2=0x55bd76acca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.418+0000 7fe103e5e740 1 --2- 172.21.15.37:0/3924017630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 0x55bd76acca90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0c4cb0 tx=0x7fe0ec0bc650 comp rx=0 tx=0).stop 2024-03-26T06:39:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.418+0000 7fe103e5e740 1 -- 172.21.15.37:0/3924017630 shutdown_connections 2024-03-26T06:39:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.418+0000 7fe103e5e740 1 --2- 172.21.15.37:0/3924017630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 0x55bd76ad8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.418+0000 7fe103e5e740 1 --2- 172.21.15.37:0/3924017630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 0x55bd76acd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.418+0000 7fe103e5e740 1 --2- 172.21.15.37:0/3924017630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 0x55bd76acca90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.418+0000 7fe103e5e740 1 -- 172.21.15.37:0/3924017630 >> 172.21.15.37:0/3924017630 conn(0x55bd76a945b0 msgr2=0x55bd76ac7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.419+0000 7fe103e5e740 1 -- 172.21.15.37:0/3924017630 shutdown_connections 2024-03-26T06:39:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.419+0000 7fe103e5e740 1 -- 172.21.15.37:0/3924017630 wait complete. 2024-03-26T06:39:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.419+0000 7fe103e5e740 1 -- start start 2024-03-26T06:39:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.419+0000 7fe103e5e740 1 Processor -- start 2024-03-26T06:39:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76accfd0 0x55bd76ad3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76acdd40 0x55bd76ad3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76ad6550 0x55bd76ad4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe103e5e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd76ad9040 con 0x55bd76acdd40 2024-03-26T06:39:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe103e5e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd769edd90 con 0x55bd76ad6550 2024-03-26T06:39:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe103e5e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd76ad91c0 con 0x55bd76accfd0 2024-03-26T06:39:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe103e5e740 10 client.0 _ll_register_callbacks cb 0x55bd76745a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:39:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76accfd0 0x55bd76ad3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76accfd0 0x55bd76ad3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:56722/0 (socket says 172.21.15.37:56722) 2024-03-26T06:39:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe0faffd640 1 -- 172.21.15.37:0/502616308 learned_addr learned my addr 172.21.15.37:0/502616308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76acdd40 0x55bd76ad3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe0fb7fe640 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76ad6550 0x55bd76ad4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe0faffd640 1 -- 172.21.15.37:0/502616308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76ad6550 msgr2=0x55bd76ad4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe0faffd640 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76ad6550 0x55bd76ad4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe0faffd640 1 -- 172.21.15.37:0/502616308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76acdd40 msgr2=0x55bd76ad3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe0faffd640 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76acdd40 0x55bd76ad3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.420+0000 7fe0faffd640 1 -- 172.21.15.37:0/502616308 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd76812620 con 0x55bd76accfd0 2024-03-26T06:39:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.421+0000 7fe0faffd640 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76accfd0 0x55bd76ad3610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0bc740 tx=0x7fe0ec0c0c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a27bc8eacf9ac724 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.421+0000 7fe0deff5640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:39:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.421+0000 7fe0deff5640 1 -- 172.21.15.37:0/502616308 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0d4070 con 0x55bd76accfd0 2024-03-26T06:39:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.421+0000 7fe0deff5640 1 -- 172.21.15.37:0/502616308 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe0ec0c4d50 con 0x55bd76accfd0 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.421+0000 7fe103e5e740 1 -- 172.21.15.37:0/502616308 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0ec0bd140 con 0x55bd76accfd0 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.421+0000 7fe0deff5640 1 -- 172.21.15.37:0/502616308 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0ec0ef070 con 0x55bd76accfd0 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.421+0000 7fe103e5e740 10 client.26248 fetch_fsmap learned FSMap version 68 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.421+0000 7fe103e5e740 1 -- 172.21.15.37:0/502616308 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55bd76ad9040 con 0x55bd76accfd0 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.421+0000 7fe0deff5640 1 -- 172.21.15.37:0/502616308 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0ec0bd140 con 0x55bd76accfd0 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.421+0000 7fe0deff5640 1 -- 172.21.15.37:0/502616308 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 68) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe0ec0bbf20 con 0x55bd76accfd0 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.422+0000 7fe103e5e740 10 client.26248 fetch_fsmap finished waiting for FSMap version 68 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.422+0000 7fe103e5e740 10 client.26248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.422+0000 7fe103e5e740 20 client.26248 populate_metadata read hostname 'smithi037' 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.422+0000 7fe103e5e740 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bd76ae6180 0x55bd76ae8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.422+0000 7fe103e5e740 4 client.26248 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.422+0000 7fe103e5e740 1 -- 172.21.15.37:0/502616308 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd76a3eb30 con 0x55bd76ae6180 2024-03-26T06:39:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.422+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bd76ae6180 0x55bd76ae8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.422+0000 7fe0deff5640 10 client.26248 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.422+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bd76ae6180 0x55bd76ae8570 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9940c1c56fe27c55 server_cookie=3b7050c8a7cef55f in_seq=0 out_seq=0 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.424+0000 7fe0deff5640 1 -- 172.21.15.37:0/502616308 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd76a3eb30 con 0x55bd76ae6180 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.424+0000 7fe0deff5640 10 client.26248 handle_command_reply: tid=0 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.424+0000 7fe103e5e740 1 client.26248 shutdown 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe103e5e740 1 -- 172.21.15.37:0/502616308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76accfd0 msgr2=0x55bd76ad3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe103e5e740 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76accfd0 0x55bd76ad3610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0bc740 tx=0x7fe0ec0c0c00 comp rx=0 tx=0).stop 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe103e5e740 1 -- 172.21.15.37:0/502616308 shutdown_connections 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe103e5e740 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bd76ae6180 0x55bd76ae8570 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe103e5e740 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76ad6550 0x55bd76ad4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe103e5e740 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76acdd40 0x55bd76ad3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe0deff5640 0 client.26248 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe103e5e740 1 --2- 172.21.15.37:0/502616308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76accfd0 0x55bd76ad3610 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe103e5e740 1 -- 172.21.15.37:0/502616308 >> 172.21.15.37:0/502616308 conn(0x55bd76a945b0 msgr2=0x55bd76ac9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe103e5e740 1 -- 172.21.15.37:0/502616308 shutdown_connections 2024-03-26T06:39:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe103e5e740 1 -- 172.21.15.37:0/502616308 wait complete. 2024-03-26T06:39:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.425+0000 7fe103e5e740 20 client.26248 trim_cache size 0 max 16384 2024-03-26T06:39:44.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:44 smithi037 ceph-mon[31974]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.4 MiB/s wr, 668 op/s 2024-03-26T06:39:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.450+0000 7fe103e5e740 1 Processor -- start 2024-03-26T06:39:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.450+0000 7fe103e5e740 1 -- start start 2024-03-26T06:39:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.450+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 0x55bd76ad5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.450+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76acdd40 0x55bd76add2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.450+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76ad6550 0x55bd76add830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.450+0000 7fe103e5e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1b2ff0 con 0x55bd76accfd0 2024-03-26T06:39:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.450+0000 7fe103e5e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1b4970 con 0x55bd76ad6550 2024-03-26T06:39:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.450+0000 7fe103e5e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1b3650 con 0x55bd76acdd40 2024-03-26T06:39:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.450+0000 7fe0fb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76ad6550 0x55bd76add830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.450+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 0x55bd76ad5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.450+0000 7fe0fb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76ad6550 0x55bd76add830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:38404/0 (socket says 172.21.15.37:38404) 2024-03-26T06:39:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.451+0000 7fe0fb7fe640 1 -- 172.21.15.37:0/817703113 learned_addr learned my addr 172.21.15.37:0/817703113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:44.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.451+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/817703113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76acdd40 0x55bd76add2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.451+0000 7fe0faffd640 1 -- 172.21.15.37:0/817703113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76acdd40 msgr2=0x55bd76add2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.451+0000 7fe0faffd640 1 --2- 172.21.15.37:0/817703113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76acdd40 0x55bd76add2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.451+0000 7fe0faffd640 1 -- 172.21.15.37:0/817703113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76ad6550 msgr2=0x55bd76add830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.451+0000 7fe0faffd640 1 --2- 172.21.15.37:0/817703113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76ad6550 0x55bd76add830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.451+0000 7fe0faffd640 1 -- 172.21.15.37:0/817703113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd76acc2a0 con 0x55bd76accfd0 2024-03-26T06:39:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.451+0000 7fe0faffd640 1 --2- 172.21.15.37:0/817703113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 0x55bd76ad5d40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0c53a0 tx=0x7fe0ec0a7230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b882845a1f8dab9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.451+0000 7fe0e0ff9640 1 -- 172.21.15.37:0/817703113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0d4070 con 0x55bd76accfd0 2024-03-26T06:39:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.452+0000 7fe0e0ff9640 1 -- 172.21.15.37:0/817703113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0ec009520 con 0x55bd76accfd0 2024-03-26T06:39:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.452+0000 7fe0e0ff9640 1 -- 172.21.15.37:0/817703113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec04b050 con 0x55bd76accfd0 2024-03-26T06:39:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.453+0000 7fe103e5e740 1 -- 172.21.15.37:0/817703113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 msgr2=0x55bd76ad5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.453+0000 7fe103e5e740 1 --2- 172.21.15.37:0/817703113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 0x55bd76ad5d40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0c53a0 tx=0x7fe0ec0a7230 comp rx=0 tx=0).stop 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.453+0000 7fe103e5e740 1 -- 172.21.15.37:0/817703113 shutdown_connections 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.453+0000 7fe103e5e740 1 --2- 172.21.15.37:0/817703113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76ad6550 0x55bd76add830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.453+0000 7fe103e5e740 1 --2- 172.21.15.37:0/817703113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76acdd40 0x55bd76add2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.453+0000 7fe103e5e740 1 --2- 172.21.15.37:0/817703113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76accfd0 0x55bd76ad5d40 secure :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0c53a0 tx=0x7fe0ec0a7230 comp rx=0 tx=0).stop 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.453+0000 7fe103e5e740 1 -- 172.21.15.37:0/817703113 >> 172.21.15.37:0/817703113 conn(0x55bd76a945b0 msgr2=0x55bd76ad1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.453+0000 7fe103e5e740 1 -- 172.21.15.37:0/817703113 shutdown_connections 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.453+0000 7fe103e5e740 1 -- 172.21.15.37:0/817703113 wait complete. 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.454+0000 7fe103e5e740 1 -- start start 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.454+0000 7fe103e5e740 1 Processor -- start 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.454+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 0x55bd76ad0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 0x55bd76ad08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 0x55bd76ad0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe0faffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 0x55bd76ad0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:38420/0 (socket says 172.21.15.37:38420) 2024-03-26T06:39:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe103e5e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76ae6180 0x55bd76ad0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe0fa7fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 0x55bd76ad08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe0fa7fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 0x55bd76ad08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:56742/0 (socket says 172.21.15.37:56742) 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe0fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76ae6180 0x55bd76ad0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe0fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76ae6180 0x55bd76ad0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55988/0 (socket says 172.21.15.37:55988) 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe0faffd640 1 -- 172.21.15.37:0/2964000006 learned_addr learned my addr 172.21.15.37:0/2964000006 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe103e5e740 1 -- 172.21.15.37:0/2964000006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1b4310 con 0x55bd76ae6180 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe103e5e740 1 -- 172.21.15.37:0/2964000006 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1b3cb0 con 0x55bd76acdd40 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.455+0000 7fe103e5e740 1 -- 172.21.15.37:0/2964000006 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1b12b0 con 0x55bd76ad6550 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.456+0000 7fe103e5e740 10 client.0 _ll_register_callbacks cb 0x7fe0ec003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.456+0000 7fe0fa7fc640 1 -- 172.21.15.37:0/2964000006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 msgr2=0x55bd76ad0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.456+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/2964000006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 0x55bd76ad0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.456+0000 7fe0fa7fc640 1 -- 172.21.15.37:0/2964000006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76ae6180 msgr2=0x55bd76ad0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.456+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/2964000006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76ae6180 0x55bd76ad0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.456+0000 7fe0fa7fc640 1 -- 172.21.15.37:0/2964000006 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd76b3c800 con 0x55bd76ad6550 2024-03-26T06:39:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.456+0000 7fe103e5e740 1 -- 172.21.15.37:0/2964000006 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0ec009520 con 0x55bd76ad6550 2024-03-26T06:39:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.457+0000 7fe0fa7fc640 1 --2- 172.21.15.37:0/2964000006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 0x55bd76ad08c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55bd76ae8e50 tx=0x7fe0f0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f6c4a4fff56da32 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.457+0000 7fe0deff5640 10 client.26254 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:39:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.457+0000 7fe0deff5640 1 -- 172.21.15.37:0/2964000006 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0036050 con 0x55bd76ad6550 2024-03-26T06:39:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.458+0000 7fe0deff5640 1 -- 172.21.15.37:0/2964000006 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe0f001cce0 con 0x55bd76ad6550 2024-03-26T06:39:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.458+0000 7fe0deff5640 1 -- 172.21.15.37:0/2964000006 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0f0031420 con 0x55bd76ad6550 2024-03-26T06:39:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.458+0000 7fe103e5e740 10 client.26254 fetch_fsmap learned FSMap version 68 2024-03-26T06:39:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.458+0000 7fe103e5e740 1 -- 172.21.15.37:0/2964000006 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({fsmap=68}) v3 -- 0x55bd76b3da30 con 0x55bd76ad6550 2024-03-26T06:39:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.458+0000 7fe0deff5640 1 -- 172.21.15.37:0/2964000006 <== mon.1 v2:172.21.15.149:3300/0 4 ==== fsmap(e 68) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe0f0039020 con 0x55bd76ad6550 2024-03-26T06:39:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.458+0000 7fe103e5e740 10 client.26254 fetch_fsmap finished waiting for FSMap version 68 2024-03-26T06:39:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.458+0000 7fe103e5e740 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:39:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.458+0000 7fe103e5e740 20 client.26254 populate_metadata read hostname 'smithi037' 2024-03-26T06:39:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.459+0000 7fe103e5e740 1 --2- 172.21.15.37:0/2964000006 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bd76b3e050 0x55bd76b40440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.459+0000 7fe103e5e740 4 client.26254 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:39:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.459+0000 7fe103e5e740 1 -- 172.21.15.37:0/2964000006 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fe0ec009520 con 0x55bd76b3e050 2024-03-26T06:39:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.459+0000 7fe0faffd640 1 --2- 172.21.15.37:0/2964000006 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bd76b3e050 0x55bd76b40440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.459+0000 7fe0deff5640 10 client.26254 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.459+0000 7fe0faffd640 1 --2- 172.21.15.37:0/2964000006 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bd76b3e050 0x55bd76b40440 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84431c9e130d72e2 server_cookie=45cbbaa66d6875d2 in_seq=0 out_seq=0 2024-03-26T06:39:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.717+0000 7fe0deff5640 1 -- 172.21.15.37:0/2964000006 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fe0ec009520 con 0x55bd76b3e050 2024-03-26T06:39:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.717+0000 7fe0deff5640 10 client.26254 handle_command_reply: tid=0 2024-03-26T06:39:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.717+0000 7fe103e5e740 1 client.26254 shutdown 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe103e5e740 1 -- 172.21.15.37:0/2964000006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 msgr2=0x55bd76ad08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe103e5e740 1 --2- 172.21.15.37:0/2964000006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 0x55bd76ad08c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55bd76ae8e50 tx=0x7fe0f0004730 comp rx=0 tx=0).stop 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe103e5e740 1 -- 172.21.15.37:0/2964000006 shutdown_connections 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe103e5e740 1 --2- 172.21.15.37:0/2964000006 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55bd76b3e050 0x55bd76b40440 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe103e5e740 1 --2- 172.21.15.37:0/2964000006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd76ae6180 0x55bd76ad0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe103e5e740 1 --2- 172.21.15.37:0/2964000006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd76ad6550 0x55bd76ad08c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe0deff5640 0 client.26254 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe103e5e740 1 --2- 172.21.15.37:0/2964000006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bd76acdd40 0x55bd76ad0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe103e5e740 1 -- 172.21.15.37:0/2964000006 >> 172.21.15.37:0/2964000006 conn(0x55bd76a945b0 msgr2=0x55bd76ac65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe103e5e740 1 -- 172.21.15.37:0/2964000006 shutdown_connections 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe103e5e740 1 -- 172.21.15.37:0/2964000006 wait complete. 2024-03-26T06:39:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.718+0000 7fe103e5e740 20 client.26254 trim_cache size 0 max 16384 2024-03-26T06:39:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:39:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:39:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:39:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:39:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 -- 172.21.15.37:0/4216960605 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe0e40a2900 msgr2=0x7fe0e40a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe0e40a2900 0x7fe0e40a4dc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc102b30 tx=0x7fe0f0005f90 comp rx=0 tx=0).stop 2024-03-26T06:39:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 -- 172.21.15.37:0/4216960605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0fc075820 msgr2=0x7fe0fc19e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0fc075820 0x7fe0fc19e1b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec005940 tx=0x7fe0ec004270 comp rx=0 tx=0).stop 2024-03-26T06:39:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 -- 172.21.15.37:0/4216960605 shutdown_connections 2024-03-26T06:39:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fe0e40a2900 0x7fe0e40a4dc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0fc112170 0x7fe0fc1a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0fc076140 0x7fe0fc19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 --2- 172.21.15.37:0/4216960605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0fc075820 0x7fe0fc19e1b0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 -- 172.21.15.37:0/4216960605 >> 172.21.15.37:0/4216960605 conn(0x7fe0fc0fefa0 msgr2=0x7fe0fc101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 -- 172.21.15.37:0/4216960605 shutdown_connections 2024-03-26T06:39:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:44.719+0000 7fe101281640 1 -- 172.21.15.37:0/4216960605 wait complete. 2024-03-26T06:39:44.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 166505 39.72 MB/sec execute 79 sec latency 152.858 ms 2024-03-26T06:39:45.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 167446 39.92 MB/sec execute 80 sec latency 54.307 ms 2024-03-26T06:39:46.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:46 smithi066 ceph-mon[36504]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.7 MiB/s wr, 798 op/s 2024-03-26T06:39:46.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:46 smithi149 ceph-mon[36681]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.7 MiB/s wr, 798 op/s 2024-03-26T06:39:46.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:46 smithi037 ceph-mon[31974]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.7 MiB/s wr, 798 op/s 2024-03-26T06:39:46.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 168385 39.98 MB/sec execute 81 sec latency 102.092 ms 2024-03-26T06:39:47.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 169205 40.15 MB/sec execute 82 sec latency 150.076 ms 2024-03-26T06:39:48.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:48 smithi066 ceph-mon[36504]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.5 MiB/s wr, 828 op/s 2024-03-26T06:39:48.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:48 smithi149 ceph-mon[36681]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.5 MiB/s wr, 828 op/s 2024-03-26T06:39:48.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:48 smithi037 ceph-mon[31974]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.5 MiB/s wr, 828 op/s 2024-03-26T06:39:48.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 170010 40.01 MB/sec execute 83 sec latency 70.721 ms 2024-03-26T06:39:49.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 170932 40.18 MB/sec execute 84 sec latency 90.171 ms 2024-03-26T06:39:50.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:50 smithi066 ceph-mon[36504]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 628 op/s 2024-03-26T06:39:50.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:50 smithi149 ceph-mon[36681]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 628 op/s 2024-03-26T06:39:50.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:50 smithi037 ceph-mon[31974]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 628 op/s 2024-03-26T06:39:50.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 171915 40.40 MB/sec execute 85 sec latency 45.147 ms 2024-03-26T06:39:51.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 172771 40.47 MB/sec execute 86 sec latency 60.487 ms 2024-03-26T06:39:52.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:52 smithi066 ceph-mon[36504]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 11 MiB/s wr, 933 op/s 2024-03-26T06:39:52.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:52 smithi149 ceph-mon[36681]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 11 MiB/s wr, 933 op/s 2024-03-26T06:39:52.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:52 smithi037 ceph-mon[31974]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 11 MiB/s wr, 933 op/s 2024-03-26T06:39:52.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 173609 40.47 MB/sec execute 87 sec latency 58.002 ms 2024-03-26T06:39:53.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 174421 40.53 MB/sec execute 88 sec latency 140.558 ms 2024-03-26T06:39:54.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:54 smithi066 ceph-mon[36504]: pgmap v534: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.0 MiB/s wr, 630 op/s 2024-03-26T06:39:54.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:54 smithi149 ceph-mon[36681]: pgmap v534: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.0 MiB/s wr, 630 op/s 2024-03-26T06:39:54.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:54 smithi037 ceph-mon[31974]: pgmap v534: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.0 MiB/s wr, 630 op/s 2024-03-26T06:39:54.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 175386 40.61 MB/sec execute 89 sec latency 101.660 ms 2024-03-26T06:39:55.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 176375 40.80 MB/sec execute 90 sec latency 67.174 ms 2024-03-26T06:39:56.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:56 smithi066 ceph-mon[36504]: pgmap v535: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 8.9 MiB/s wr, 793 op/s 2024-03-26T06:39:56.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:56 smithi149 ceph-mon[36681]: pgmap v535: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 8.9 MiB/s wr, 793 op/s 2024-03-26T06:39:56.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:56 smithi037 ceph-mon[31974]: pgmap v535: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 8.9 MiB/s wr, 793 op/s 2024-03-26T06:39:56.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 177249 40.92 MB/sec execute 91 sec latency 46.607 ms 2024-03-26T06:39:57.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 178078 40.87 MB/sec execute 92 sec latency 60.268 ms 2024-03-26T06:39:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.853+0000 7fb28835b640 1 -- 172.21.15.37:0/2798621899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2801034e0 msgr2=0x7fb280103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.853+0000 7fb28835b640 1 --2- 172.21.15.37:0/2798621899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2801034e0 0x7fb280103960 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb270009900 tx=0x7fb27002bc30 comp rx=0 tx=0).stop 2024-03-26T06:39:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.854+0000 7fb28835b640 1 -- 172.21.15.37:0/2798621899 shutdown_connections 2024-03-26T06:39:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.854+0000 7fb28835b640 1 --2- 172.21.15.37:0/2798621899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb280103ea0 0x7fb280110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.854+0000 7fb28835b640 1 --2- 172.21.15.37:0/2798621899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2801034e0 0x7fb280103960 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.854+0000 7fb28835b640 1 --2- 172.21.15.37:0/2798621899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2801095a0 0x7fb280109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.854+0000 7fb28835b640 1 -- 172.21.15.37:0/2798621899 >> 172.21.15.37:0/2798621899 conn(0x7fb2800fefb0 msgr2=0x7fb2801013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.854+0000 7fb28835b640 1 -- 172.21.15.37:0/2798621899 shutdown_connections 2024-03-26T06:39:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.854+0000 7fb28835b640 1 -- 172.21.15.37:0/2798621899 wait complete. 2024-03-26T06:39:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.855+0000 7fb28835b640 1 Processor -- start 2024-03-26T06:39:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.855+0000 7fb28835b640 1 -- start start 2024-03-26T06:39:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.855+0000 7fb28835b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2801034e0 0x7fb280110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.855+0000 7fb28835b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb280103ea0 0x7fb28010d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb28835b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2801095a0 0x7fb28010d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb28835b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb280113880 con 0x7fb2801095a0 2024-03-26T06:39:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb28835b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb280113700 con 0x7fb2801034e0 2024-03-26T06:39:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb28835b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb280113a00 con 0x7fb280103ea0 2024-03-26T06:39:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2858f6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb280103ea0 0x7fb28010d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2858f6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb280103ea0 0x7fb28010d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:40244/0 (socket says 172.21.15.37:40244) 2024-03-26T06:39:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2858f6640 1 -- 172.21.15.37:0/348619945 learned_addr learned my addr 172.21.15.37:0/348619945 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2868f8640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2801095a0 0x7fb28010d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2860f7640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2801034e0 0x7fb280110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2858f6640 1 -- 172.21.15.37:0/348619945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2801034e0 msgr2=0x7fb280110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2858f6640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2801034e0 0x7fb280110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2858f6640 1 -- 172.21.15.37:0/348619945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2801095a0 msgr2=0x7fb28010d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2858f6640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2801095a0 0x7fb28010d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2858f6640 1 -- 172.21.15.37:0/348619945 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28010df70 con 0x7fb280103ea0 2024-03-26T06:39:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2868f8640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2801095a0 0x7fb28010d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:39:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.856+0000 7fb2860f7640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2801034e0 0x7fb280110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:39:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.857+0000 7fb2858f6640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb280103ea0 0x7fb28010d1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb270005940 tx=0x7fb2700042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.857+0000 7fb2777fe640 1 -- 172.21.15.37:0/348619945 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb270041070 con 0x7fb280103ea0 2024-03-26T06:39:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.857+0000 7fb28835b640 1 -- 172.21.15.37:0/348619945 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2801aa5c0 con 0x7fb280103ea0 2024-03-26T06:39:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.857+0000 7fb28835b640 1 -- 172.21.15.37:0/348619945 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2801aab60 con 0x7fb280103ea0 2024-03-26T06:39:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.857+0000 7fb2777fe640 1 -- 172.21.15.37:0/348619945 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb27002ccc0 con 0x7fb280103ea0 2024-03-26T06:39:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.857+0000 7fb2777fe640 1 -- 172.21.15.37:0/348619945 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27004b060 con 0x7fb280103ea0 2024-03-26T06:39:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.861+0000 7fb28b01b740 1 Processor -- start 2024-03-26T06:39:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.861+0000 7fb28b01b740 1 -- start start 2024-03-26T06:39:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.861+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3357d40 0x55feb3356a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.861+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 0x55feb3357450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3360550 0x55feb3362940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2868f8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3357d40 0x55feb3356a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2777fe640 1 -- 172.21.15.37:0/348619945 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb270044070 con 0x7fb280103ea0 2024-03-26T06:39:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2868f8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3357d40 0x55feb3356a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:58694/0 (socket says 172.21.15.37:58694) 2024-03-26T06:39:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2868f8640 1 -- 172.21.15.37:0/1380422480 learned_addr learned my addr 172.21.15.37:0/1380422480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb28b01b740 1 -- 172.21.15.37:0/1380422480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb32bb5f0 con 0x55feb3360550 2024-03-26T06:39:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb28b01b740 1 -- 172.21.15.37:0/1380422480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb3264930 con 0x55feb3357d40 2024-03-26T06:39:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2860f7640 1 --2- 172.21.15.37:0/1380422480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 0x55feb3357450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb28b01b740 1 -- 172.21.15.37:0/1380422480 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb32b7690 con 0x55feb3356fd0 2024-03-26T06:39:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2858f6640 1 --2- 172.21.15.37:0/1380422480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3360550 0x55feb3362940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2860f7640 1 -- 172.21.15.37:0/1380422480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3357d40 msgr2=0x55feb3356a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2860f7640 1 --2- 172.21.15.37:0/1380422480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3357d40 0x55feb3356a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2860f7640 1 -- 172.21.15.37:0/1380422480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3360550 msgr2=0x55feb3362940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2860f7640 1 --2- 172.21.15.37:0/1380422480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3360550 0x55feb3362940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2860f7640 1 -- 172.21.15.37:0/1380422480 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55feb2fd0210 con 0x55feb3356fd0 2024-03-26T06:39:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.862+0000 7fb2858f6640 1 --2- 172.21.15.37:0/1380422480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3360550 0x55feb3362940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:39:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.863+0000 7fb2777fe640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb26c0a2a10 0x7fb26c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.863+0000 7fb2868f8640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb26c0a2a10 0x7fb26c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.863+0000 7fb2860f7640 1 --2- 172.21.15.37:0/1380422480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 0x55feb3357450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009c40 tx=0x7fb27c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4e6c2696033fc57 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.863+0000 7fb2777fe640 1 -- 172.21.15.37:0/348619945 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb2700f0590 con 0x7fb280103ea0 2024-03-26T06:39:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.863+0000 7fb274bfe640 1 -- 172.21.15.37:0/1380422480 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c004030 con 0x55feb3356fd0 2024-03-26T06:39:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.863+0000 7fb274bfe640 1 -- 172.21.15.37:0/1380422480 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb27c0041d0 con 0x55feb3356fd0 2024-03-26T06:39:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.863+0000 7fb274bfe640 1 -- 172.21.15.37:0/1380422480 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c012650 con 0x55feb3356fd0 2024-03-26T06:39:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.864+0000 7fb2868f8640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb26c0a2a10 0x7fb26c0a4ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb2780098b0 tx=0x7fb278009600 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.864+0000 7fb28b01b740 1 -- 172.21.15.37:0/1380422480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 msgr2=0x55feb3357450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.864+0000 7fb28b01b740 1 --2- 172.21.15.37:0/1380422480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 0x55feb3357450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009c40 tx=0x7fb27c010740 comp rx=0 tx=0).stop 2024-03-26T06:39:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.864+0000 7fb28b01b740 1 -- 172.21.15.37:0/1380422480 shutdown_connections 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.864+0000 7fb28b01b740 1 --2- 172.21.15.37:0/1380422480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3360550 0x55feb3362940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.864+0000 7fb28b01b740 1 --2- 172.21.15.37:0/1380422480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 0x55feb3357450 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.864+0000 7fb28b01b740 1 --2- 172.21.15.37:0/1380422480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3357d40 0x55feb3356a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.864+0000 7fb28b01b740 1 -- 172.21.15.37:0/1380422480 >> 172.21.15.37:0/1380422480 conn(0x55feb331e5b0 msgr2=0x55feb3351cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.864+0000 7fb28b01b740 1 -- 172.21.15.37:0/1380422480 shutdown_connections 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.865+0000 7fb28b01b740 1 -- 172.21.15.37:0/1380422480 wait complete. 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.865+0000 7fb28b01b740 1 -- start start 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.865+0000 7fb28b01b740 1 Processor -- start 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 0x55feb335d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb335db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 0x55feb335e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb28b01b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb3363280 con 0x55feb3357d40 2024-03-26T06:39:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb28b01b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb3277d90 con 0x55feb3360550 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb28b01b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feb3363400 con 0x55feb3356fd0 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb2860f7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 0x55feb335d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb2858f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb335db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb28b01b740 10 client.0 _ll_register_callbacks cb 0x55feb2fcfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb2858f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb335db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45138/0 (socket says 172.21.15.37:45138) 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb2858f6640 1 -- 172.21.15.37:0/2024642487 learned_addr learned my addr 172.21.15.37:0/2024642487 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb2860f7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 0x55feb335d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:40262/0 (socket says 172.21.15.37:40262) 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.866+0000 7fb2868f8640 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 0x55feb335e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb2858f6640 1 -- 172.21.15.37:0/2024642487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 msgr2=0x55feb335d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb2858f6640 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 0x55feb335d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb2858f6640 1 -- 172.21.15.37:0/2024642487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 msgr2=0x55feb335e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb2858f6640 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 0x55feb335e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb2858f6640 1 -- 172.21.15.37:0/2024642487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55feb309c620 con 0x55feb3357d40 2024-03-26T06:39:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb2860f7640 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 0x55feb335d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:39:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb2868f8640 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 0x55feb335e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:39:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb2858f6640 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb335db10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb2700bca00 tx=0x7fb2700056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b50dd97c6c63a34 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb269ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:39:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb269ffb640 1 -- 172.21.15.37:0/2024642487 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2700b8fa0 con 0x55feb3357d40 2024-03-26T06:39:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb28b01b740 1 -- 172.21.15.37:0/2024642487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb27c0041d0 con 0x55feb3357d40 2024-03-26T06:39:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.867+0000 7fb269ffb640 1 -- 172.21.15.37:0/2024642487 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb2700b9370 con 0x55feb3357d40 2024-03-26T06:39:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.868+0000 7fb269ffb640 1 -- 172.21.15.37:0/2024642487 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2700bc660 con 0x55feb3357d40 2024-03-26T06:39:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.868+0000 7fb28b01b740 10 client.16872 fetch_fsmap learned FSMap version 68 2024-03-26T06:39:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.868+0000 7fb28b01b740 1 -- 172.21.15.37:0/2024642487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55feb336f5b0 con 0x55feb3357d40 2024-03-26T06:39:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.868+0000 7fb269ffb640 1 -- 172.21.15.37:0/2024642487 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb27c0041d0 con 0x55feb3357d40 2024-03-26T06:39:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.868+0000 7fb269ffb640 1 -- 172.21.15.37:0/2024642487 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 68) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb27004e060 con 0x55feb3357d40 2024-03-26T06:39:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.868+0000 7fb28b01b740 10 client.16872 fetch_fsmap finished waiting for FSMap version 68 2024-03-26T06:39:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.868+0000 7fb28b01b740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:39:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.868+0000 7fb28b01b740 20 client.16872 populate_metadata read hostname 'smithi037' 2024-03-26T06:39:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.868+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55feb3370a10 0x55feb3372e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.868+0000 7fb28b01b740 4 client.16872 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:39:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.868+0000 7fb28b01b740 1 -- 172.21.15.37:0/2024642487 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55feb32c8b30 con 0x55feb3370a10 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.869+0000 7fb2860f7640 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55feb3370a10 0x55feb3372e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.869+0000 7fb269ffb640 10 client.16872 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.869+0000 7fb2860f7640 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55feb3370a10 0x55feb3372e70 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e1404a7b02c138a server_cookie=35531446ed64eb85 in_seq=0 out_seq=0 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.870+0000 7fb269ffb640 1 -- 172.21.15.37:0/2024642487 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55feb32c8b30 con 0x55feb3370a10 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.870+0000 7fb269ffb640 10 client.16872 handle_command_reply: tid=0 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.870+0000 7fb28b01b740 1 client.16872 shutdown 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb28b01b740 1 -- 172.21.15.37:0/2024642487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 msgr2=0x55feb335db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb335db10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb2700bca00 tx=0x7fb2700056c0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb28b01b740 1 -- 172.21.15.37:0/2024642487 shutdown_connections 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55feb3370a10 0x55feb3372e70 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 0x55feb335e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb335db10 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb269ffb640 0 client.16872 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2024642487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3356fd0 0x55feb335d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb28b01b740 1 -- 172.21.15.37:0/2024642487 >> 172.21.15.37:0/2024642487 conn(0x55feb331e5b0 msgr2=0x55feb3353a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb28b01b740 1 -- 172.21.15.37:0/2024642487 shutdown_connections 2024-03-26T06:39:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb28b01b740 1 -- 172.21.15.37:0/2024642487 wait complete. 2024-03-26T06:39:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.871+0000 7fb28b01b740 20 client.16872 trim_cache size 0 max 16384 2024-03-26T06:39:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb28b01b740 1 Processor -- start 2024-03-26T06:39:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb28b01b740 1 -- start start 2024-03-26T06:39:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3356fd0 0x55feb3367670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3357d40 0x55feb3367bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 0x55feb33517f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb28b01b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2801b3140 con 0x55feb3356fd0 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb28b01b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2801b4ac0 con 0x55feb3360550 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb28b01b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2801b37a0 con 0x55feb3357d40 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb2860f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3356fd0 0x55feb3367670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb2860f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3356fd0 0x55feb3367670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45148/0 (socket says 172.21.15.37:45148) 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb2858f6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3357d40 0x55feb3367bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.891+0000 7fb2860f7640 1 -- 172.21.15.37:0/2498243226 learned_addr learned my addr 172.21.15.37:0/2498243226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.892+0000 7fb2868f8640 1 --2- 172.21.15.37:0/2498243226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 0x55feb33517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.892+0000 7fb2868f8640 1 -- 172.21.15.37:0/2498243226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3357d40 msgr2=0x55feb3367bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.892+0000 7fb2868f8640 1 --2- 172.21.15.37:0/2498243226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3357d40 0x55feb3367bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.892+0000 7fb2868f8640 1 -- 172.21.15.37:0/2498243226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3356fd0 msgr2=0x55feb3367670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.892+0000 7fb2868f8640 1 --2- 172.21.15.37:0/2498243226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3356fd0 0x55feb3367670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.892+0000 7fb2868f8640 1 -- 172.21.15.37:0/2498243226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55feb33584d0 con 0x55feb3360550 2024-03-26T06:39:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.892+0000 7fb2860f7640 1 --2- 172.21.15.37:0/2498243226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3356fd0 0x55feb3367670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:39:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.892+0000 7fb2858f6640 1 --2- 172.21.15.37:0/2498243226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3357d40 0x55feb3367bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:39:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.892+0000 7fb2868f8640 1 --2- 172.21.15.37:0/2498243226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 0x55feb33517f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb278005fd0 tx=0x7fb27800b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76168d982cfbb8f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.892+0000 7fb26bfff640 1 -- 172.21.15.37:0/2498243226 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27802c070 con 0x55feb3360550 2024-03-26T06:39:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.893+0000 7fb26bfff640 1 -- 172.21.15.37:0/2498243226 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb27801c940 con 0x55feb3360550 2024-03-26T06:39:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.893+0000 7fb26bfff640 1 -- 172.21.15.37:0/2498243226 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278027b00 con 0x55feb3360550 2024-03-26T06:39:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.893+0000 7fb28b01b740 1 -- 172.21.15.37:0/2498243226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 msgr2=0x55feb33517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.893+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2498243226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 0x55feb33517f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb278005fd0 tx=0x7fb27800b000 comp rx=0 tx=0).stop 2024-03-26T06:39:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.893+0000 7fb28b01b740 1 -- 172.21.15.37:0/2498243226 shutdown_connections 2024-03-26T06:39:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.893+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2498243226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3360550 0x55feb33517f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.893+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2498243226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3357d40 0x55feb3367bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.893+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2498243226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3356fd0 0x55feb3367670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.893+0000 7fb28b01b740 1 -- 172.21.15.37:0/2498243226 >> 172.21.15.37:0/2498243226 conn(0x55feb331e5b0 msgr2=0x55feb334cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.893+0000 7fb28b01b740 1 -- 172.21.15.37:0/2498243226 shutdown_connections 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb28b01b740 1 -- 172.21.15.37:0/2498243226 wait complete. 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb28b01b740 1 -- start start 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb28b01b740 1 Processor -- start 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3356fd0 0x55feb3367950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb3359f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb28b01b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3360550 0x55feb335a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb2858f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb3359f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb28b01b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2801b4460 con 0x55feb3357d40 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb2858f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb3359f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45152/0 (socket says 172.21.15.37:45152) 2024-03-26T06:39:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb28b01b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2801b3e00 con 0x55feb3356fd0 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb2860f7640 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3356fd0 0x55feb3367950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb2858f6640 1 -- 172.21.15.37:0/2940354751 learned_addr learned my addr 172.21.15.37:0/2940354751 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.894+0000 7fb28b01b740 1 -- 172.21.15.37:0/2940354751 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2801b1400 con 0x55feb3360550 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.895+0000 7fb2868f8640 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3360550 0x55feb335a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.895+0000 7fb2860f7640 1 -- 172.21.15.37:0/2940354751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3360550 msgr2=0x55feb335a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.895+0000 7fb2860f7640 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3360550 0x55feb335a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.895+0000 7fb2860f7640 1 -- 172.21.15.37:0/2940354751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 msgr2=0x55feb3359f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.895+0000 7fb2860f7640 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb3359f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.895+0000 7fb2860f7640 1 -- 172.21.15.37:0/2940354751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55feb335ac70 con 0x55feb3356fd0 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.895+0000 7fb2858f6640 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb3359f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.895+0000 7fb28b01b740 10 client.0 _ll_register_callbacks cb 0x7fb27c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.895+0000 7fb28b01b740 1 -- 172.21.15.37:0/2940354751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb27801c940 con 0x55feb3356fd0 2024-03-26T06:39:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.895+0000 7fb2860f7640 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3356fd0 0x55feb3367950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb27c010c40 tx=0x7fb27c010cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15fbbb922bc0880d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.895+0000 7fb269ffb640 10 client.26378 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb269ffb640 1 -- 172.21.15.37:0/2940354751 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c02e060 con 0x55feb3356fd0 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb269ffb640 1 -- 172.21.15.37:0/2940354751 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb27c0037f0 con 0x55feb3356fd0 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb269ffb640 1 -- 172.21.15.37:0/2940354751 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb27c003a90 con 0x55feb3356fd0 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb28b01b740 10 client.26378 fetch_fsmap learned FSMap version 68 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb28b01b740 1 -- 172.21.15.37:0/2940354751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=68}) v3 -- 0x55feb33c7b50 con 0x55feb3356fd0 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb269ffb640 1 -- 172.21.15.37:0/2940354751 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 68) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb27c029ce0 con 0x55feb3356fd0 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb28b01b740 10 client.26378 fetch_fsmap finished waiting for FSMap version 68 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb28b01b740 10 client.26378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb28b01b740 20 client.26378 populate_metadata read hostname 'smithi037' 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55feb33c7f50 0x55feb33ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb28b01b740 4 client.26378 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb28b01b740 1 -- 172.21.15.37:0/2940354751 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55feb32c2e80 con 0x55feb33c7f50 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.896+0000 7fb2858f6640 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55feb33c7f50 0x55feb33ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:39:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.897+0000 7fb2858f6640 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55feb33c7f50 0x55feb33ca410 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d81702f03fa379 server_cookie=2ccf650614a5f6ff in_seq=0 out_seq=0 2024-03-26T06:39:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:57.897+0000 7fb269ffb640 10 client.26378 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:58.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:39:58 smithi066 ceph-mon[36504]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 9.3 MiB/s wr, 789 op/s 2024-03-26T06:39:58.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:39:58 smithi149 ceph-mon[36681]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 9.3 MiB/s wr, 789 op/s 2024-03-26T06:39:58.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:39:58 smithi037 ceph-mon[31974]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 9.3 MiB/s wr, 789 op/s 2024-03-26T06:39:58.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 178217 40.52 MB/sec execute 93 sec latency 883.604 ms 2024-03-26T06:39:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.478+0000 7fb269ffb640 1 -- 172.21.15.37:0/2940354751 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55feb32c2e80 con 0x55feb33c7f50 2024-03-26T06:39:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.478+0000 7fb269ffb640 10 client.26378 handle_command_reply: tid=0 2024-03-26T06:39:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.478+0000 7fb28b01b740 1 client.26378 shutdown 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.478+0000 7fb28b01b740 1 -- 172.21.15.37:0/2940354751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3356fd0 msgr2=0x55feb3367950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.478+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3356fd0 0x55feb3367950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb27c010c40 tx=0x7fb27c010cf0 comp rx=0 tx=0).stop 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.478+0000 7fb28b01b740 1 -- 172.21.15.37:0/2940354751 shutdown_connections 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.478+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55feb33c7f50 0x55feb33ca410 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.478+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55feb3360550 0x55feb335a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.478+0000 7fb269ffb640 0 client.26378 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.478+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55feb3357d40 0x55feb3359f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.478+0000 7fb28b01b740 1 --2- 172.21.15.37:0/2940354751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55feb3356fd0 0x55feb3367950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28b01b740 1 -- 172.21.15.37:0/2940354751 >> 172.21.15.37:0/2940354751 conn(0x55feb331e5b0 msgr2=0x55feb33517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28b01b740 1 -- 172.21.15.37:0/2940354751 shutdown_connections 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28b01b740 1 -- 172.21.15.37:0/2940354751 wait complete. 2024-03-26T06:39:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28b01b740 20 client.26378 trim_cache size 0 max 16384 2024-03-26T06:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:39:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 -- 172.21.15.37:0/348619945 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb26c0a2a10 msgr2=0x7fb26c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb26c0a2a10 0x7fb26c0a4ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb2780098b0 tx=0x7fb278009600 comp rx=0 tx=0).stop 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 -- 172.21.15.37:0/348619945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb280103ea0 msgr2=0x7fb28010d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb280103ea0 0x7fb28010d1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb270005940 tx=0x7fb2700042f0 comp rx=0 tx=0).stop 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 -- 172.21.15.37:0/348619945 shutdown_connections 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb26c0a2a10 0x7fb26c0a4ed0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2801095a0 0x7fb28010d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb280103ea0 0x7fb28010d1a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 --2- 172.21.15.37:0/348619945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2801034e0 0x7fb280110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 -- 172.21.15.37:0/348619945 >> 172.21.15.37:0/348619945 conn(0x7fb2800fefb0 msgr2=0x7fb280100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 -- 172.21.15.37:0/348619945 shutdown_connections 2024-03-26T06:39:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:39:59.479+0000 7fb28835b640 1 -- 172.21.15.37:0/348619945 wait complete. 2024-03-26T06:39:59.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 178245 40.10 MB/sec execute 94 sec latency 1489.518 ms 2024-03-26T06:40:00.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:00 smithi066 ceph-mon[36504]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.5 MiB/s wr, 593 op/s 2024-03-26T06:40:00.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:00 smithi066 ceph-mon[36504]: overall HEALTH_OK 2024-03-26T06:40:00.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:00 smithi149 ceph-mon[36681]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.5 MiB/s wr, 593 op/s 2024-03-26T06:40:00.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:00 smithi149 ceph-mon[36681]: overall HEALTH_OK 2024-03-26T06:40:00.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:00 smithi037 ceph-mon[31974]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.5 MiB/s wr, 593 op/s 2024-03-26T06:40:00.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:00 smithi037 ceph-mon[31974]: overall HEALTH_OK 2024-03-26T06:40:00.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 178654 39.92 MB/sec execute 95 sec latency 1147.620 ms 2024-03-26T06:40:01.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 178843 39.55 MB/sec execute 96 sec latency 435.151 ms 2024-03-26T06:40:02.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:02 smithi066 ceph-mon[36504]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 26 MiB/s wr, 1.17k op/s 2024-03-26T06:40:02.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:02 smithi149 ceph-mon[36681]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 26 MiB/s wr, 1.17k op/s 2024-03-26T06:40:02.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:02 smithi037 ceph-mon[31974]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 26 MiB/s wr, 1.17k op/s 2024-03-26T06:40:02.853 INFO:tasks.workunit.client.0.smithi037.stdout: 10 178843 39.15 MB/sec execute 97 sec latency 1435.260 ms 2024-03-26T06:40:03.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 178843 38.75 MB/sec execute 98 sec latency 2435.350 ms 2024-03-26T06:40:04.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:04 smithi149 ceph-mon[36681]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 863 op/s 2024-03-26T06:40:04.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:04 smithi037 ceph-mon[31974]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 863 op/s 2024-03-26T06:40:04.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:04 smithi066 ceph-mon[36504]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 863 op/s 2024-03-26T06:40:04.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 178843 38.35 MB/sec execute 99 sec latency 3435.455 ms 2024-03-26T06:40:05.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 178843 37.97 MB/sec execute 100 sec latency 4435.531 ms 2024-03-26T06:40:06.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:06 smithi149 ceph-mon[36681]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 23 MiB/s wr, 977 op/s 2024-03-26T06:40:06.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:06 smithi037 ceph-mon[31974]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 23 MiB/s wr, 977 op/s 2024-03-26T06:40:06.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:06 smithi066 ceph-mon[36504]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 23 MiB/s wr, 977 op/s 2024-03-26T06:40:06.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 179358 37.88 MB/sec execute 101 sec latency 4937.935 ms 2024-03-26T06:40:07.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 180112 37.99 MB/sec execute 102 sec latency 82.897 ms 2024-03-26T06:40:08.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:08 smithi149 ceph-mon[36681]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 986 op/s 2024-03-26T06:40:08.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:08 smithi037 ceph-mon[31974]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 986 op/s 2024-03-26T06:40:08.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:08 smithi066 ceph-mon[36504]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 21 MiB/s wr, 986 op/s 2024-03-26T06:40:08.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 180808 38.00 MB/sec execute 103 sec latency 110.207 ms 2024-03-26T06:40:09.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 181596 37.93 MB/sec execute 104 sec latency 65.356 ms 2024-03-26T06:40:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:10 smithi149 ceph-mon[36681]: pgmap v542: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 860 op/s 2024-03-26T06:40:10.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:10 smithi037 ceph-mon[31974]: pgmap v542: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 860 op/s 2024-03-26T06:40:10.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:10 smithi066 ceph-mon[36504]: pgmap v542: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 860 op/s 2024-03-26T06:40:10.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 182244 37.91 MB/sec execute 105 sec latency 92.748 ms 2024-03-26T06:40:11.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:11 smithi149 ceph-mon[36681]: osdmap e113: 12 total, 12 up, 12 in 2024-03-26T06:40:11.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:11 smithi037 ceph-mon[31974]: osdmap e113: 12 total, 12 up, 12 in 2024-03-26T06:40:11.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:11 smithi066 ceph-mon[36504]: osdmap e113: 12 total, 12 up, 12 in 2024-03-26T06:40:11.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 182768 37.93 MB/sec execute 106 sec latency 88.566 ms 2024-03-26T06:40:12.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:12 smithi149 ceph-mon[36681]: pgmap v544: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.3 MiB/s wr, 649 op/s 2024-03-26T06:40:12.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:12 smithi037 ceph-mon[31974]: pgmap v544: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.3 MiB/s wr, 649 op/s 2024-03-26T06:40:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.590+0000 7f3608661640 1 -- 172.21.15.37:0/311403239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600105910 msgr2=0x7f3600105cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.590+0000 7f3608661640 1 --2- 172.21.15.37:0/311403239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600105910 0x7f3600105cf0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00da30 tx=0x7f35fc02dcb0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.590+0000 7f3608661640 1 -- 172.21.15.37:0/311403239 shutdown_connections 2024-03-26T06:40:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.590+0000 7f3608661640 1 --2- 172.21.15.37:0/311403239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36001002c0 0x7f360010c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.590+0000 7f3608661640 1 --2- 172.21.15.37:0/311403239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36000ff900 0x7f36000ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.590+0000 7f3608661640 1 --2- 172.21.15.37:0/311403239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600105910 0x7f3600105cf0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.590+0000 7f3608661640 1 -- 172.21.15.37:0/311403239 >> 172.21.15.37:0/311403239 conn(0x7f3600076780 msgr2=0x7f36000fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.590+0000 7f3608661640 1 -- 172.21.15.37:0/311403239 shutdown_connections 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.590+0000 7f3608661640 1 -- 172.21.15.37:0/311403239 wait complete. 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.590+0000 7f3608661640 1 Processor -- start 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3608661640 1 -- start start 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3608661640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36000ff900 0x7f360019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3608661640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36001002c0 0x7f360019ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3608661640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600105910 0x7f3600199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3608661640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360010f4a0 con 0x7f3600105910 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3608661640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360010f320 con 0x7f36001002c0 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3608661640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360010f620 con 0x7f36000ff900 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3606bfe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600105910 0x7f3600199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f36063fd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36000ff900 0x7f360019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3606bfe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600105910 0x7f3600199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56354/0 (socket says 172.21.15.37:56354) 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3606bfe640 1 -- 172.21.15.37:0/2011599553 learned_addr learned my addr 172.21.15.37:0/2011599553 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3605bfc640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36001002c0 0x7f360019ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f36063fd640 1 -- 172.21.15.37:0/2011599553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36001002c0 msgr2=0x7f360019ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f36063fd640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36001002c0 0x7f360019ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f36063fd640 1 -- 172.21.15.37:0/2011599553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600105910 msgr2=0x7f3600199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f36063fd640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600105910 0x7f3600199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f36063fd640 1 -- 172.21.15.37:0/2011599553 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360019a3d0 con 0x7f36000ff900 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.591+0000 7f3605bfc640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36001002c0 0x7f360019ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.592+0000 7f36063fd640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36000ff900 0x7f360019fa50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f35fc000c00 tx=0x7f35fc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.592+0000 7f35f77fe640 1 -- 172.21.15.37:0/2011599553 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc02ea60 con 0x7f36000ff900 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.592+0000 7f3608661640 1 -- 172.21.15.37:0/2011599553 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360019a660 con 0x7f36000ff900 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.592+0000 7f35f77fe640 1 -- 172.21.15.37:0/2011599553 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35fc036c70 con 0x7f36000ff900 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.592+0000 7f35f77fe640 1 -- 172.21.15.37:0/2011599553 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc005180 con 0x7f36000ff900 2024-03-26T06:40:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.592+0000 7f3608661640 1 -- 172.21.15.37:0/2011599553 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36001a69d0 con 0x7f36000ff900 2024-03-26T06:40:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.595+0000 7f35f77fe640 1 -- 172.21.15.37:0/2011599553 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f35fc005320 con 0x7f36000ff900 2024-03-26T06:40:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.595+0000 7f35f77fe640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35ec0a2a10 0x7f35ec0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.595+0000 7f360b243740 1 Processor -- start 2024-03-26T06:40:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f3605bfc640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35ec0a2a10 0x7f35ec0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f35f77fe640 1 -- 172.21.15.37:0/2011599553 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f35fc0f09d0 con 0x7f36000ff900 2024-03-26T06:40:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f360b243740 1 -- start start 2024-03-26T06:40:12.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:12 smithi066 ceph-mon[36504]: pgmap v544: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.3 MiB/s wr, 649 op/s 2024-03-26T06:40:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f360b243740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f21a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f360b243740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f22450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f360b243740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f33d60 0x564b39f36150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f360b243740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b39e865f0 con 0x564b39f21fd0 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f360b243740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b39e2f930 con 0x564b39f22d40 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f360b243740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b39e82690 con 0x564b39f33d60 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f3605bfc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f22450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f3605bfc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f22450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56364/0 (socket says 172.21.15.37:56364) 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f3605bfc640 1 -- 172.21.15.37:0/1308223443 learned_addr learned my addr 172.21.15.37:0/1308223443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f3606bfe640 1 --2- 172.21.15.37:0/1308223443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f33d60 0x564b39f36150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f36063fd640 1 --2- 172.21.15.37:0/1308223443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f21a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f3605bfc640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35ec0a2a10 0x7f35ec0a4ed0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f360019b2f0 tx=0x7f35f0005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f3605bfc640 1 -- 172.21.15.37:0/1308223443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f33d60 msgr2=0x564b39f36150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f3605bfc640 1 --2- 172.21.15.37:0/1308223443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f33d60 0x564b39f36150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f3605bfc640 1 -- 172.21.15.37:0/1308223443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 msgr2=0x564b39f21a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.596+0000 7f3605bfc640 1 --2- 172.21.15.37:0/1308223443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f21a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.597+0000 7f3605bfc640 1 -- 172.21.15.37:0/1308223443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b39b9b210 con 0x564b39f21fd0 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.597+0000 7f3605bfc640 1 --2- 172.21.15.37:0/1308223443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f22450 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f35f00042b0 tx=0x7f35f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee9b47f26ebdd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.597+0000 7f35f4bfe640 1 -- 172.21.15.37:0/1308223443 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f00044e0 con 0x564b39f21fd0 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.597+0000 7f35f4bfe640 1 -- 172.21.15.37:0/1308223443 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35f000f960 con 0x564b39f21fd0 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.597+0000 7f35f4bfe640 1 -- 172.21.15.37:0/1308223443 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f001dad0 con 0x564b39f21fd0 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.598+0000 7f360b243740 1 -- 172.21.15.37:0/1308223443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 msgr2=0x564b39f22450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.598+0000 7f360b243740 1 --2- 172.21.15.37:0/1308223443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f22450 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f35f00042b0 tx=0x7f35f0014000 comp rx=0 tx=0).stop 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.598+0000 7f360b243740 1 -- 172.21.15.37:0/1308223443 shutdown_connections 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.598+0000 7f360b243740 1 --2- 172.21.15.37:0/1308223443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f33d60 0x564b39f36150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.598+0000 7f360b243740 1 --2- 172.21.15.37:0/1308223443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f22450 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.598+0000 7f360b243740 1 --2- 172.21.15.37:0/1308223443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f21a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.598+0000 7f360b243740 1 -- 172.21.15.37:0/1308223443 >> 172.21.15.37:0/1308223443 conn(0x564b39ee95b0 msgr2=0x564b39f1ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.598+0000 7f360b243740 1 -- 172.21.15.37:0/1308223443 shutdown_connections 2024-03-26T06:40:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.598+0000 7f360b243740 1 -- 172.21.15.37:0/1308223443 wait complete. 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.598+0000 7f360b243740 1 -- start start 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.598+0000 7f360b243740 1 Processor -- start 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f360b243740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f30d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f360b243740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f31290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f360b243740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f33d60 0x564b39f317d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f360b243740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b39f36a90 con 0x564b39f21fd0 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f360b243740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b39e42d90 con 0x564b39f22d40 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f36063fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f30d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f360b243740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b39f36c10 con 0x564b39f33d60 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f36063fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f30d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56366/0 (socket says 172.21.15.37:56366) 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f36063fd640 1 -- 172.21.15.37:0/1326025526 learned_addr learned my addr 172.21.15.37:0/1326025526 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f360b243740 10 client.0 _ll_register_callbacks cb 0x564b39b9aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f3606bfe640 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f33d60 0x564b39f317d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f3605bfc640 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f31290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f36063fd640 1 -- 172.21.15.37:0/1326025526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f33d60 msgr2=0x564b39f317d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f36063fd640 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f33d60 0x564b39f317d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f36063fd640 1 -- 172.21.15.37:0/1326025526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 msgr2=0x564b39f31290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f36063fd640 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f31290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f36063fd640 1 -- 172.21.15.37:0/1326025526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b39c67620 con 0x564b39f21fd0 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.599+0000 7f3605bfc640 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f31290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f36063fd640 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f30d50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0ee2c0 tx=0x7f35fc00d4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d4746608fb22b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f35e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f35e9ffb640 1 -- 172.21.15.37:0/1326025526 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0bca80 con 0x564b39f21fd0 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f360b243740 1 -- 172.21.15.37:0/1326025526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35f000f960 con 0x564b39f21fd0 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f35e9ffb640 1 -- 172.21.15.37:0/1326025526 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f35fc0c5630 con 0x564b39f21fd0 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f35e9ffb640 1 -- 172.21.15.37:0/1326025526 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35fc02ee10 con 0x564b39f21fd0 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f360b243740 10 client.16890 fetch_fsmap learned FSMap version 68 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f360b243740 1 -- 172.21.15.37:0/1326025526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x564b39f42db0 con 0x564b39f21fd0 2024-03-26T06:40:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f35e9ffb640 1 -- 172.21.15.37:0/1326025526 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35f000f960 con 0x564b39f21fd0 2024-03-26T06:40:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f35e9ffb640 1 -- 172.21.15.37:0/1326025526 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 68) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f35fc00b040 con 0x564b39f21fd0 2024-03-26T06:40:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f360b243740 10 client.16890 fetch_fsmap finished waiting for FSMap version 68 2024-03-26T06:40:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.600+0000 7f360b243740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:40:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.601+0000 7f360b243740 20 client.16890 populate_metadata read hostname 'smithi037' 2024-03-26T06:40:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.601+0000 7f360b243740 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x564b39f441c0 0x564b39f46610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.601+0000 7f360b243740 4 client.16890 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:40:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.601+0000 7f360b243740 1 -- 172.21.15.37:0/1326025526 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b39e93b30 con 0x564b39f441c0 2024-03-26T06:40:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.601+0000 7f3605bfc640 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x564b39f441c0 0x564b39f46610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.603+0000 7f35e9ffb640 10 client.16890 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.603+0000 7f3605bfc640 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x564b39f441c0 0x564b39f46610 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f452990ce45956b server_cookie=4d8a2fb7205dc608 in_seq=0 out_seq=0 2024-03-26T06:40:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f35e9ffb640 1 -- 172.21.15.37:0/1326025526 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b39e93b30 con 0x564b39f441c0 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f35e9ffb640 10 client.16890 handle_command_reply: tid=0 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 1 client.16890 shutdown 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 1 -- 172.21.15.37:0/1326025526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 msgr2=0x564b39f30d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f30d50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0ee2c0 tx=0x7f35fc00d4c0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 1 -- 172.21.15.37:0/1326025526 shutdown_connections 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x564b39f441c0 0x564b39f46610 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f33d60 0x564b39f317d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f35e9ffb640 0 client.16890 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f31290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 1 --2- 172.21.15.37:0/1326025526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f21fd0 0x564b39f30d50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 1 -- 172.21.15.37:0/1326025526 >> 172.21.15.37:0/1326025526 conn(0x564b39ee95b0 msgr2=0x564b39f1ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 1 -- 172.21.15.37:0/1326025526 shutdown_connections 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 1 -- 172.21.15.37:0/1326025526 wait complete. 2024-03-26T06:40:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.604+0000 7f360b243740 20 client.16890 trim_cache size 0 max 16384 2024-03-26T06:40:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.623+0000 7f360b243740 1 Processor -- start 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.623+0000 7f360b243740 1 -- start start 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f360b243740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f21fd0 0x564b39f33480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f360b243740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f2f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f360b243740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 0x564b39f2fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f360b243740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e400e490 con 0x564b39f33d60 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f360b243740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e400fe10 con 0x564b39f22d40 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f360b243740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e400eaf0 con 0x564b39f21fd0 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f3606bfe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 0x564b39f2fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f3606bfe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 0x564b39f2fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56376/0 (socket says 172.21.15.37:56376) 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f36063fd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f21fd0 0x564b39f33480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f3606bfe640 1 -- 172.21.15.37:0/646087133 learned_addr learned my addr 172.21.15.37:0/646087133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f3605bfc640 1 --2- 172.21.15.37:0/646087133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f2f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f3606bfe640 1 -- 172.21.15.37:0/646087133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f21fd0 msgr2=0x564b39f33480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f3606bfe640 1 --2- 172.21.15.37:0/646087133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f21fd0 0x564b39f33480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f3606bfe640 1 -- 172.21.15.37:0/646087133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 msgr2=0x564b39f2f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f3606bfe640 1 --2- 172.21.15.37:0/646087133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f2f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f3606bfe640 1 -- 172.21.15.37:0/646087133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b39f212a0 con 0x564b39f33d60 2024-03-26T06:40:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.624+0000 7f3606bfe640 1 --2- 172.21.15.37:0/646087133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 0x564b39f2fb60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f35f8012770 tx=0x7f35f8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27692ad6b552d13a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.625+0000 7f35ebfff640 1 -- 172.21.15.37:0/646087133 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f80120d0 con 0x564b39f33d60 2024-03-26T06:40:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.625+0000 7f35ebfff640 1 -- 172.21.15.37:0/646087133 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35f8021070 con 0x564b39f33d60 2024-03-26T06:40:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.625+0000 7f35ebfff640 1 -- 172.21.15.37:0/646087133 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8010ba0 con 0x564b39f33d60 2024-03-26T06:40:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.625+0000 7f360b243740 1 -- 172.21.15.37:0/646087133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 msgr2=0x564b39f2fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.625+0000 7f360b243740 1 --2- 172.21.15.37:0/646087133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 0x564b39f2fb60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f35f8012770 tx=0x7f35f8009500 comp rx=0 tx=0).stop 2024-03-26T06:40:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.625+0000 7f360b243740 1 -- 172.21.15.37:0/646087133 shutdown_connections 2024-03-26T06:40:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.625+0000 7f360b243740 1 --2- 172.21.15.37:0/646087133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 0x564b39f2fb60 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.625+0000 7f360b243740 1 --2- 172.21.15.37:0/646087133 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f22d40 0x564b39f2f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.625+0000 7f360b243740 1 --2- 172.21.15.37:0/646087133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f21fd0 0x564b39f33480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.625+0000 7f360b243740 1 -- 172.21.15.37:0/646087133 >> 172.21.15.37:0/646087133 conn(0x564b39ee95b0 msgr2=0x564b39f2c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f360b243740 1 -- 172.21.15.37:0/646087133 shutdown_connections 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f360b243740 1 -- 172.21.15.37:0/646087133 wait complete. 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f360b243740 1 -- start start 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f360b243740 1 Processor -- start 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f360b243740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f21fd0 0x564b39f39830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f360b243740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f22d40 0x564b39f39d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f360b243740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 0x564b39f3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f360b243740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e400f7b0 con 0x564b39f33d60 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f360b243740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e400f150 con 0x564b39f21fd0 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f360b243740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e400c750 con 0x564b39f22d40 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f360b243740 10 client.0 _ll_register_callbacks cb 0x7f35f0009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f3605bfc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f22d40 0x564b39f39d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f3606bfe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 0x564b39f3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f36063fd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f21fd0 0x564b39f39830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f36063fd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f21fd0 0x564b39f39830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:42230/0 (socket says 172.21.15.37:42230) 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f3606bfe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 0x564b39f3a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56386/0 (socket says 172.21.15.37:56386) 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f3605bfc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f22d40 0x564b39f39d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:60050/0 (socket says 172.21.15.37:60050) 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.626+0000 7f36063fd640 1 -- 172.21.15.37:0/3833944742 learned_addr learned my addr 172.21.15.37:0/3833944742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f3605bfc640 1 -- 172.21.15.37:0/3833944742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f21fd0 msgr2=0x564b39f39830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f3605bfc640 1 --2- 172.21.15.37:0/3833944742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f21fd0 0x564b39f39830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f3605bfc640 1 -- 172.21.15.37:0/3833944742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 msgr2=0x564b39f3a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f3605bfc640 1 --2- 172.21.15.37:0/3833944742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 0x564b39f3a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f3605bfc640 1 -- 172.21.15.37:0/3833944742 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b39f9a740 con 0x564b39f22d40 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f3605bfc640 1 --2- 172.21.15.37:0/3833944742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f22d40 0x564b39f39d70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f35f0011ca0 tx=0x7f35f0011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e044f8001ee2d80 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f35e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f35e9ffb640 1 -- 172.21.15.37:0/3833944742 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f00320f0 con 0x564b39f22d40 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f360b243740 1 -- 172.21.15.37:0/3833944742 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35f8021070 con 0x564b39f22d40 2024-03-26T06:40:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f35e9ffb640 1 -- 172.21.15.37:0/3833944742 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f35f0009040 con 0x564b39f22d40 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f35e9ffb640 1 -- 172.21.15.37:0/3833944742 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35f0032290 con 0x564b39f22d40 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f360b243740 10 client.26290 fetch_fsmap learned FSMap version 68 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.627+0000 7f360b243740 1 -- 172.21.15.37:0/3833944742 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x564b39f9b9b0 con 0x564b39f22d40 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.628+0000 7f35e9ffb640 1 -- 172.21.15.37:0/3833944742 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35f8021070 con 0x564b39f22d40 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.628+0000 7f35e9ffb640 1 -- 172.21.15.37:0/3833944742 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 68) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f35f001a070 con 0x564b39f22d40 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.628+0000 7f360b243740 10 client.26290 fetch_fsmap finished waiting for FSMap version 68 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.628+0000 7f360b243740 10 client.26290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.628+0000 7f360b243740 20 client.26290 populate_metadata read hostname 'smithi037' 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.628+0000 7f360b243740 1 --2- 172.21.15.37:0/3833944742 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x564b39f441c0 0x564b39f9de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.628+0000 7f360b243740 4 client.26290 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.628+0000 7f360b243740 1 -- 172.21.15.37:0/3833944742 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564b39e8de80 con 0x564b39f441c0 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.628+0000 7f36063fd640 1 --2- 172.21.15.37:0/3833944742 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x564b39f441c0 0x564b39f9de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.629+0000 7f35e9ffb640 10 client.26290 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:12.629+0000 7f36063fd640 1 --2- 172.21.15.37:0/3833944742 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x564b39f441c0 0x564b39f9de10 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b17450cbd427f0 server_cookie=a08a7e15468f5fb in_seq=0 out_seq=0 2024-03-26T06:40:12.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 183383 37.95 MB/sec execute 107 sec latency 991.591 ms 2024-03-26T06:40:13.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 183887 37.74 MB/sec execute 108 sec latency 1991.625 ms 2024-03-26T06:40:14.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:14 smithi149 ceph-mon[36681]: pgmap v545: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.3 MiB/s wr, 649 op/s 2024-03-26T06:40:14.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:14 smithi037 ceph-mon[31974]: pgmap v545: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.3 MiB/s wr, 649 op/s 2024-03-26T06:40:14.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:14 smithi066 ceph-mon[36504]: pgmap v545: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.3 MiB/s wr, 649 op/s 2024-03-26T06:40:14.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 184592 37.73 MB/sec execute 109 sec latency 2991.659 ms 2024-03-26T06:40:15.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 185288 37.74 MB/sec execute 110 sec latency 3991.688 ms 2024-03-26T06:40:16.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:16 smithi149 ceph-mon[36681]: pgmap v546: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 653 op/s 2024-03-26T06:40:16.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:16 smithi037 ceph-mon[31974]: pgmap v546: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 653 op/s 2024-03-26T06:40:16.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:16 smithi066 ceph-mon[36504]: pgmap v546: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 653 op/s 2024-03-26T06:40:16.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 185696 37.67 MB/sec execute 111 sec latency 4991.720 ms 2024-03-26T06:40:17.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 185897 37.39 MB/sec execute 112 sec latency 5991.748 ms 2024-03-26T06:40:18.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:18 smithi149 ceph-mon[36681]: pgmap v547: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 624 op/s 2024-03-26T06:40:18.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:18 smithi037 ceph-mon[31974]: pgmap v547: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 624 op/s 2024-03-26T06:40:18.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:18 smithi066 ceph-mon[36504]: pgmap v547: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 624 op/s 2024-03-26T06:40:18.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 186093 37.17 MB/sec execute 113 sec latency 6991.779 ms 2024-03-26T06:40:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f35e9ffb640 1 -- 172.21.15.37:0/3833944742 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564b39e8de80 con 0x564b39f441c0 2024-03-26T06:40:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f35e9ffb640 10 client.26290 handle_command_reply: tid=0 2024-03-26T06:40:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 1 client.26290 shutdown 2024-03-26T06:40:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 1 -- 172.21.15.37:0/3833944742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f22d40 msgr2=0x564b39f39d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 1 --2- 172.21.15.37:0/3833944742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f22d40 0x564b39f39d70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f35f0011ca0 tx=0x7f35f0011d70 comp rx=0 tx=0).stop 2024-03-26T06:40:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 1 -- 172.21.15.37:0/3833944742 shutdown_connections 2024-03-26T06:40:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 1 --2- 172.21.15.37:0/3833944742 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x564b39f441c0 0x564b39f9de10 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 1 --2- 172.21.15.37:0/3833944742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b39f33d60 0x564b39f3a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 1 --2- 172.21.15.37:0/3833944742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b39f22d40 0x564b39f39d70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f35e9ffb640 0 client.26290 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 1 --2- 172.21.15.37:0/3833944742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564b39f21fd0 0x564b39f39830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 1 -- 172.21.15.37:0/3833944742 >> 172.21.15.37:0/3833944742 conn(0x564b39ee95b0 msgr2=0x564b39f2f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 1 -- 172.21.15.37:0/3833944742 shutdown_connections 2024-03-26T06:40:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 1 -- 172.21.15.37:0/3833944742 wait complete. 2024-03-26T06:40:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.294+0000 7f360b243740 20 client.26290 trim_cache size 0 max 16384 2024-03-26T06:40:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:40:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:40:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:40:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:40:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 -- 172.21.15.37:0/2011599553 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35ec0a2a10 msgr2=0x7f35ec0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35ec0a2a10 0x7f35ec0a4ed0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f360019b2f0 tx=0x7f35f0005f90 comp rx=0 tx=0).stop 2024-03-26T06:40:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 -- 172.21.15.37:0/2011599553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36000ff900 msgr2=0x7f360019fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36000ff900 0x7f360019fa50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f35fc000c00 tx=0x7f35fc004270 comp rx=0 tx=0).stop 2024-03-26T06:40:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 -- 172.21.15.37:0/2011599553 shutdown_connections 2024-03-26T06:40:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f35ec0a2a10 0x7f35ec0a4ed0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3600105910 0x7f3600199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36001002c0 0x7f360019ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 --2- 172.21.15.37:0/2011599553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36000ff900 0x7f360019fa50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 -- 172.21.15.37:0/2011599553 >> 172.21.15.37:0/2011599553 conn(0x7f3600076780 msgr2=0x7f3600102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 -- 172.21.15.37:0/2011599553 shutdown_connections 2024-03-26T06:40:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:19.295+0000 7f3608661640 1 -- 172.21.15.37:0/2011599553 wait complete. 2024-03-26T06:40:19.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 186351 36.95 MB/sec execute 114 sec latency 7991.812 ms 2024-03-26T06:40:20.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 186698 36.76 MB/sec execute 115 sec latency 8991.861 ms 2024-03-26T06:40:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:20 smithi066 ceph-mon[36504]: pgmap v548: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 624 op/s 2024-03-26T06:40:21.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:20 smithi066 ceph-mon[36504]: osdmap e114: 12 total, 12 up, 12 in 2024-03-26T06:40:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:20 smithi149 ceph-mon[36681]: pgmap v548: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 624 op/s 2024-03-26T06:40:21.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:20 smithi149 ceph-mon[36681]: osdmap e114: 12 total, 12 up, 12 in 2024-03-26T06:40:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:20 smithi037 ceph-mon[31974]: pgmap v548: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 624 op/s 2024-03-26T06:40:21.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:20 smithi037 ceph-mon[31974]: osdmap e114: 12 total, 12 up, 12 in 2024-03-26T06:40:21.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 187155 36.67 MB/sec execute 116 sec latency 9991.891 ms 2024-03-26T06:40:22.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 187602 36.59 MB/sec execute 117 sec latency 10991.924 ms 2024-03-26T06:40:23.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:22 smithi066 ceph-mon[36504]: pgmap v550: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-03-26T06:40:23.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:22 smithi149 ceph-mon[36681]: pgmap v550: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-03-26T06:40:23.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:22 smithi037 ceph-mon[31974]: pgmap v550: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-03-26T06:40:23.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 187929 36.38 MB/sec execute 118 sec latency 11529.594 ms 2024-03-26T06:40:24.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 188557 36.36 MB/sec execute 119 sec latency 10693.824 ms 2024-03-26T06:40:25.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:24 smithi066 ceph-mon[36504]: pgmap v551: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-03-26T06:40:25.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:24 smithi149 ceph-mon[36681]: pgmap v551: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-03-26T06:40:25.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:24 smithi037 ceph-mon[31974]: pgmap v551: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-03-26T06:40:25.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 189213 36.34 MB/sec execute 120 sec latency 11280.049 ms 2024-03-26T06:40:26.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:25 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:40:26.188 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:25 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:40:26.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:25 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:40:26.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 189858 36.30 MB/sec execute 121 sec latency 338.756 ms 2024-03-26T06:40:27.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:26 smithi066 ceph-mon[36504]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.4 MiB/s wr, 456 op/s 2024-03-26T06:40:27.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:26 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:40:27.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:26 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:40:27.100 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:26 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:40:27.167 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:26 smithi149 ceph-mon[36681]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.4 MiB/s wr, 456 op/s 2024-03-26T06:40:27.167 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:26 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:40:27.167 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:26 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:40:27.167 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:26 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:40:27.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:26 smithi037 ceph-mon[31974]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.4 MiB/s wr, 456 op/s 2024-03-26T06:40:27.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:26 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:40:27.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:26 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:40:27.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:26 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:40:27.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 190663 36.39 MB/sec execute 122 sec latency 81.402 ms 2024-03-26T06:40:28.182 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:27 smithi149 ceph-mon[36681]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.9 MiB/s wr, 464 op/s 2024-03-26T06:40:28.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:27 smithi037 ceph-mon[31974]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.9 MiB/s wr, 464 op/s 2024-03-26T06:40:28.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:27 smithi066 ceph-mon[36504]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.9 MiB/s wr, 464 op/s 2024-03-26T06:40:28.854 INFO:tasks.workunit.client.0.smithi037.stdout: 10 191272 36.42 MB/sec execute 123 sec latency 225.532 ms 2024-03-26T06:40:29.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 191742 36.29 MB/sec execute 124 sec latency 330.418 ms 2024-03-26T06:40:30.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:30 smithi066 ceph-mon[36504]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.9 MiB/s wr, 464 op/s 2024-03-26T06:40:30.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:30 smithi149 ceph-mon[36681]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.9 MiB/s wr, 464 op/s 2024-03-26T06:40:30.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:30 smithi037 ceph-mon[31974]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.9 MiB/s wr, 464 op/s 2024-03-26T06:40:30.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 192517 36.37 MB/sec execute 125 sec latency 96.264 ms 2024-03-26T06:40:31.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 193135 36.29 MB/sec execute 126 sec latency 64.903 ms 2024-03-26T06:40:32.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:32 smithi066 ceph-mon[36504]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 696 op/s 2024-03-26T06:40:32.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:32 smithi149 ceph-mon[36681]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 696 op/s 2024-03-26T06:40:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.418+0000 7f85f41a9640 1 -- 172.21.15.37:0/194114122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ec100df0 msgr2=0x7f85ec104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.418+0000 7f85f41a9640 1 --2- 172.21.15.37:0/194114122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ec100df0 0x7f85ec104e30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009900 tx=0x7f85dc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:40:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.418+0000 7f85f41a9640 1 -- 172.21.15.37:0/194114122 shutdown_connections 2024-03-26T06:40:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.418+0000 7f85f41a9640 1 --2- 172.21.15.37:0/194114122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85ec105510 0x7f85ec10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.418+0000 7f85f41a9640 1 --2- 172.21.15.37:0/194114122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ec100df0 0x7f85ec104e30 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.418+0000 7f85f41a9640 1 --2- 172.21.15.37:0/194114122 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec100440 0x7f85ec100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.418+0000 7f85f41a9640 1 -- 172.21.15.37:0/194114122 >> 172.21.15.37:0/194114122 conn(0x7f85ec077350 msgr2=0x7f85ec077760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.418+0000 7f85f41a9640 1 -- 172.21.15.37:0/194114122 shutdown_connections 2024-03-26T06:40:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.418+0000 7f85f41a9640 1 -- 172.21.15.37:0/194114122 wait complete. 2024-03-26T06:40:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.419+0000 7f85f41a9640 1 Processor -- start 2024-03-26T06:40:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.419+0000 7f85f41a9640 1 -- start start 2024-03-26T06:40:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f41a9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85ec100440 0x7f85ec0fe020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f41a9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec105510 0x7f85ec0fe560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f41a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ec0feaa0 0x7f85ec1a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f41a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1105d0 con 0x7f85ec0feaa0 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f41a9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec110450 con 0x7f85ec105510 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f41a9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec110750 con 0x7f85ec100440 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f2746640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ec0feaa0 0x7f85ec1a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f1744640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec105510 0x7f85ec0fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f1f45640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85ec100440 0x7f85ec0fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f2746640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ec0feaa0 0x7f85ec1a5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53090/0 (socket says 172.21.15.37:53090) 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f1744640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec105510 0x7f85ec0fe560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:54750/0 (socket says 172.21.15.37:54750) 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f1f45640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85ec100440 0x7f85ec0fe020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:45236/0 (socket says 172.21.15.37:45236) 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f2746640 1 -- 172.21.15.37:0/4071560479 learned_addr learned my addr 172.21.15.37:0/4071560479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f2746640 1 -- 172.21.15.37:0/4071560479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85ec100440 msgr2=0x7f85ec0fe020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f2746640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85ec100440 0x7f85ec0fe020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f2746640 1 -- 172.21.15.37:0/4071560479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec105510 msgr2=0x7f85ec0fe560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f2746640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec105510 0x7f85ec0fe560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f2746640 1 -- 172.21.15.37:0/4071560479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec1a6380 con 0x7f85ec0feaa0 2024-03-26T06:40:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.420+0000 7f85f2746640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ec0feaa0 0x7f85ec1a5cb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f85e400eb10 tx=0x7f85e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.421+0000 7f85e2ffd640 1 -- 172.21.15.37:0/4071560479 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4004b20 con 0x7f85ec0feaa0 2024-03-26T06:40:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.421+0000 7f85f41a9640 1 -- 172.21.15.37:0/4071560479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ec1a6670 con 0x7f85ec0feaa0 2024-03-26T06:40:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.421+0000 7f85e2ffd640 1 -- 172.21.15.37:0/4071560479 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85e4004cc0 con 0x7f85ec0feaa0 2024-03-26T06:40:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.421+0000 7f85e2ffd640 1 -- 172.21.15.37:0/4071560479 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4010720 con 0x7f85ec0feaa0 2024-03-26T06:40:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.421+0000 7f85f41a9640 1 -- 172.21.15.37:0/4071560479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec1a6b90 con 0x7f85ec0feaa0 2024-03-26T06:40:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.424+0000 7f85e2ffd640 1 -- 172.21.15.37:0/4071560479 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f85e40108c0 con 0x7f85ec0feaa0 2024-03-26T06:40:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.425+0000 7f85f6d86740 1 Processor -- start 2024-03-26T06:40:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.425+0000 7f85f6d86740 1 -- start start 2024-03-26T06:40:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.425+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b49d40 0x55e216b48a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.425+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b48fd0 0x55e216b49450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b5ad60 0x55e216b5d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85f6d86740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e216aad5f0 con 0x55e216b48fd0 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85f6d86740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e216a56930 con 0x55e216b49d40 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85f6d86740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e216aa9690 con 0x55e216b5ad60 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85f1744640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b49d40 0x55e216b48a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85f1744640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b49d40 0x55e216b48a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:54756/0 (socket says 172.21.15.37:54756) 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85f1744640 1 -- 172.21.15.37:0/4084347535 learned_addr learned my addr 172.21.15.37:0/4084347535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85e2ffd640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85d80a2b60 0x7f85d80a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85f1744640 1 --2- 172.21.15.37:0/4084347535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b5ad60 0x55e216b5d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85e2ffd640 1 -- 172.21.15.37:0/4071560479 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f85e40cf310 con 0x7f85ec0feaa0 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85f1744640 1 -- 172.21.15.37:0/4084347535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b5ad60 msgr2=0x55e216b5d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85f1744640 1 --2- 172.21.15.37:0/4084347535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b5ad60 0x55e216b5d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.426+0000 7f85f1744640 1 -- 172.21.15.37:0/4084347535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b48fd0 msgr2=0x55e216b49450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.427+0000 7f85f1f45640 1 --2- 172.21.15.37:0/4084347535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b48fd0 0x55e216b49450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.427+0000 7f85f1744640 1 --2- 172.21.15.37:0/4084347535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b48fd0 0x55e216b49450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.427+0000 7f85f1744640 1 -- 172.21.15.37:0/4084347535 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2167c2210 con 0x55e216b49d40 2024-03-26T06:40:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.427+0000 7f85f2746640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85d80a2b60 0x7f85d80a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.427+0000 7f85f1f45640 1 --2- 172.21.15.37:0/4084347535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b48fd0 0x55e216b49450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:40:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.427+0000 7f85f1744640 1 --2- 172.21.15.37:0/4084347535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b49d40 0x55e216b48a90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f85dc006010 tx=0x7f85dc02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce0b5ca3dfb4173f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.427+0000 7f85d7fff640 1 -- 172.21.15.37:0/4084347535 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc02c880 con 0x55e216b49d40 2024-03-26T06:40:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.427+0000 7f85d7fff640 1 -- 172.21.15.37:0/4084347535 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85dc005530 con 0x55e216b49d40 2024-03-26T06:40:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.427+0000 7f85d7fff640 1 -- 172.21.15.37:0/4084347535 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0049b0 con 0x55e216b49d40 2024-03-26T06:40:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.428+0000 7f85f2746640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85d80a2b60 0x7f85d80a5020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009270 tx=0x7f85e4009200 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.428+0000 7f85f6d86740 1 -- 172.21.15.37:0/4084347535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b49d40 msgr2=0x55e216b48a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.428+0000 7f85f6d86740 1 --2- 172.21.15.37:0/4084347535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b49d40 0x55e216b48a90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f85dc006010 tx=0x7f85dc02bc30 comp rx=0 tx=0).stop 2024-03-26T06:40:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.429+0000 7f85f6d86740 1 -- 172.21.15.37:0/4084347535 shutdown_connections 2024-03-26T06:40:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.429+0000 7f85f6d86740 1 --2- 172.21.15.37:0/4084347535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b5ad60 0x55e216b5d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.429+0000 7f85f6d86740 1 --2- 172.21.15.37:0/4084347535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b48fd0 0x55e216b49450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.429+0000 7f85f6d86740 1 --2- 172.21.15.37:0/4084347535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b49d40 0x55e216b48a90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.429+0000 7f85f6d86740 1 -- 172.21.15.37:0/4084347535 >> 172.21.15.37:0/4084347535 conn(0x55e216b105b0 msgr2=0x55e216b43cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.429+0000 7f85f6d86740 1 -- 172.21.15.37:0/4084347535 shutdown_connections 2024-03-26T06:40:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.429+0000 7f85f6d86740 1 -- 172.21.15.37:0/4084347535 wait complete. 2024-03-26T06:40:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.429+0000 7f85f6d86740 1 -- start start 2024-03-26T06:40:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.429+0000 7f85f6d86740 1 Processor -- start 2024-03-26T06:40:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b48fd0 0x55e216b57e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b49d40 0x55e216b583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b5ad60 0x55e216b588e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f6d86740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e216b5da30 con 0x55e216b5ad60 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f6d86740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e216a69d90 con 0x55e216b48fd0 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f6d86740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e216b5dbb0 con 0x55e216b49d40 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f6d86740 10 client.0 _ll_register_callbacks cb 0x55e2167c1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f1744640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b5ad60 0x55e216b588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f1f45640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b49d40 0x55e216b583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f1744640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b48fd0 0x55e216b57e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f1744640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b48fd0 0x55e216b57e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:54770/0 (socket says 172.21.15.37:54770) 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f1f45640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b49d40 0x55e216b583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:45260/0 (socket says 172.21.15.37:45260) 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f1744640 1 -- 172.21.15.37:0/1850367646 learned_addr learned my addr 172.21.15.37:0/1850367646 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f1744640 1 -- 172.21.15.37:0/1850367646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b49d40 msgr2=0x55e216b583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f1744640 1 --2- 172.21.15.37:0/1850367646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b49d40 0x55e216b583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f1744640 1 -- 172.21.15.37:0/1850367646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b5ad60 msgr2=0x55e216b588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.430+0000 7f85f1744640 1 --2- 172.21.15.37:0/1850367646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b5ad60 0x55e216b588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.431+0000 7f85f1744640 1 -- 172.21.15.37:0/1850367646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e21688e620 con 0x55e216b48fd0 2024-03-26T06:40:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.431+0000 7f85f1744640 1 --2- 172.21.15.37:0/1850367646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b48fd0 0x55e216b57e60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f85dc036040 tx=0x7f85dc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95e5f22fff6b461 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.431+0000 7f85d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-03-26T06:40:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.431+0000 7f85d57fa640 1 -- 172.21.15.37:0/1850367646 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc003cf0 con 0x55e216b48fd0 2024-03-26T06:40:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.431+0000 7f85f6d86740 1 -- 172.21.15.37:0/1850367646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85dc005530 con 0x55e216b48fd0 2024-03-26T06:40:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.431+0000 7f85d57fa640 1 -- 172.21.15.37:0/1850367646 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f85dc003e90 con 0x55e216b48fd0 2024-03-26T06:40:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.431+0000 7f85d57fa640 1 -- 172.21.15.37:0/1850367646 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85dc04f450 con 0x55e216b48fd0 2024-03-26T06:40:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.431+0000 7f85f6d86740 10 client.26408 fetch_fsmap learned FSMap version 68 2024-03-26T06:40:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.431+0000 7f85f6d86740 1 -- 172.21.15.37:0/1850367646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55e216b69d60 con 0x55e216b48fd0 2024-03-26T06:40:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.431+0000 7f85d57fa640 1 -- 172.21.15.37:0/1850367646 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85dc005530 con 0x55e216b48fd0 2024-03-26T06:40:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.432+0000 7f85d57fa640 1 -- 172.21.15.37:0/1850367646 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 68) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f85dc043680 con 0x55e216b48fd0 2024-03-26T06:40:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.432+0000 7f85f6d86740 10 client.26408 fetch_fsmap finished waiting for FSMap version 68 2024-03-26T06:40:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.432+0000 7f85f6d86740 10 client.26408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:40:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.432+0000 7f85f6d86740 20 client.26408 populate_metadata read hostname 'smithi037' 2024-03-26T06:40:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.432+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1850367646 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e216b6b0b0 0x55e216b6d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.432+0000 7f85f6d86740 4 client.26408 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:40:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.432+0000 7f85f6d86740 1 -- 172.21.15.37:0/1850367646 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e216abab30 con 0x55e216b6b0b0 2024-03-26T06:40:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.432+0000 7f85f1f45640 1 --2- 172.21.15.37:0/1850367646 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e216b6b0b0 0x55e216b6d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.433+0000 7f85f1f45640 1 --2- 172.21.15.37:0/1850367646 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e216b6b0b0 0x55e216b6d4a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f9ad5231dfb8c8 server_cookie=6be933f80702a49f in_seq=0 out_seq=0 2024-03-26T06:40:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.433+0000 7f85d57fa640 10 client.26408 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.438+0000 7f85d57fa640 1 -- 172.21.15.37:0/1850367646 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e216abab30 con 0x55e216b6b0b0 2024-03-26T06:40:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.438+0000 7f85d57fa640 10 client.26408 handle_command_reply: tid=0 2024-03-26T06:40:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 1 client.26408 shutdown 2024-03-26T06:40:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 1 -- 172.21.15.37:0/1850367646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b48fd0 msgr2=0x55e216b57e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1850367646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b48fd0 0x55e216b57e60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f85dc036040 tx=0x7f85dc000bc0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 1 -- 172.21.15.37:0/1850367646 shutdown_connections 2024-03-26T06:40:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1850367646 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e216b6b0b0 0x55e216b6d4a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1850367646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b5ad60 0x55e216b588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1850367646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b49d40 0x55e216b583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1850367646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b48fd0 0x55e216b57e60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85d57fa640 0 client.26408 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 1 -- 172.21.15.37:0/1850367646 >> 172.21.15.37:0/1850367646 conn(0x55e216b105b0 msgr2=0x55e216b45a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 1 -- 172.21.15.37:0/1850367646 shutdown_connections 2024-03-26T06:40:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 1 -- 172.21.15.37:0/1850367646 wait complete. 2024-03-26T06:40:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.439+0000 7f85f6d86740 20 client.26408 trim_cache size 0 max 16384 2024-03-26T06:40:32.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:32 smithi037 ceph-mon[31974]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 696 op/s 2024-03-26T06:40:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f6d86740 1 Processor -- start 2024-03-26T06:40:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f6d86740 1 -- start start 2024-03-26T06:40:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b105b0 0x55e216b45820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 0x55e216b6f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b49d40 0x55e216b6f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f6d86740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1aaa60 con 0x55e216b49d40 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f6d86740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1ac3e0 con 0x55e216b105b0 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f6d86740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1ab0c0 con 0x55e216b48fd0 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f1744640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b49d40 0x55e216b6f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f1744640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b49d40 0x55e216b6f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53122/0 (socket says 172.21.15.37:53122) 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f1744640 1 -- 172.21.15.37:0/1068453368 learned_addr learned my addr 172.21.15.37:0/1068453368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f1f45640 1 --2- 172.21.15.37:0/1068453368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 0x55e216b6f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.460+0000 7f85f1744640 1 --2- 172.21.15.37:0/1068453368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b105b0 0x55e216b45820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.461+0000 7f85f1f45640 1 -- 172.21.15.37:0/1068453368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b105b0 msgr2=0x55e216b45820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.461+0000 7f85f1f45640 1 --2- 172.21.15.37:0/1068453368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b105b0 0x55e216b45820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.461+0000 7f85f1f45640 1 -- 172.21.15.37:0/1068453368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b49d40 msgr2=0x55e216b6f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.461+0000 7f85f1f45640 1 --2- 172.21.15.37:0/1068453368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b49d40 0x55e216b6f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.461+0000 7f85f1f45640 1 -- 172.21.15.37:0/1068453368 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e216b482a0 con 0x55e216b48fd0 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.461+0000 7f85f1f45640 1 --2- 172.21.15.37:0/1068453368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 0x55e216b6f4b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55e216b5dbb0 tx=0x7f85e80128e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ebb250ba647ed9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.461+0000 7f85d77fe640 1 -- 172.21.15.37:0/1068453368 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8021030 con 0x55e216b48fd0 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.461+0000 7f85d77fe640 1 -- 172.21.15.37:0/1068453368 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85e8010430 con 0x55e216b48fd0 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.461+0000 7f85d77fe640 1 -- 172.21.15.37:0/1068453368 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800f620 con 0x55e216b48fd0 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.462+0000 7f85f6d86740 1 -- 172.21.15.37:0/1068453368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 msgr2=0x55e216b6f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.462+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1068453368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 0x55e216b6f4b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55e216b5dbb0 tx=0x7f85e80128e0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.462+0000 7f85f6d86740 1 -- 172.21.15.37:0/1068453368 shutdown_connections 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.462+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1068453368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b49d40 0x55e216b6f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.462+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1068453368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 0x55e216b6f4b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.462+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1068453368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b105b0 0x55e216b45820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.462+0000 7f85f6d86740 1 -- 172.21.15.37:0/1068453368 >> 172.21.15.37:0/1068453368 conn(0x55e216b5d330 msgr2=0x55e216b45110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.462+0000 7f85f6d86740 1 -- 172.21.15.37:0/1068453368 shutdown_connections 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.462+0000 7f85f6d86740 1 -- 172.21.15.37:0/1068453368 wait complete. 2024-03-26T06:40:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f6d86740 1 -- start start 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f6d86740 1 Processor -- start 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b105b0 0x55e216b54c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 0x55e216b55190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f6d86740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b49d40 0x55e216b556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f6d86740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1abd80 con 0x55e216b49d40 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f6d86740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1ab720 con 0x55e216b105b0 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f6d86740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1a8d40 con 0x55e216b48fd0 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f1744640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b105b0 0x55e216b54c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f1f45640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 0x55e216b55190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f6d86740 10 client.0 _ll_register_callbacks cb 0x7f85e40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f1f45640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 0x55e216b55190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:45270/0 (socket says 172.21.15.37:45270) 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f1744640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b105b0 0x55e216b54c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.37:54790/0 (socket says 172.21.15.37:54790) 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.463+0000 7f85f1f45640 1 -- 172.21.15.37:0/1288451097 learned_addr learned my addr 172.21.15.37:0/1288451097 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85f1744640 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b49d40 0x55e216b556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85f1f45640 1 -- 172.21.15.37:0/1288451097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b105b0 msgr2=0x55e216b54c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85f1f45640 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b105b0 0x55e216b54c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85f1f45640 1 -- 172.21.15.37:0/1288451097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b49d40 msgr2=0x55e216b556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85f1f45640 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b49d40 0x55e216b556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85f1f45640 1 -- 172.21.15.37:0/1288451097 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e216bc1350 con 0x55e216b48fd0 2024-03-26T06:40:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85f1f45640 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 0x55e216b55190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f85e80099e0 tx=0x7f85e801d7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eadae50f6680e805 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:40:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85d57fa640 1 -- 172.21.15.37:0/1288451097 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8021040 con 0x55e216b48fd0 2024-03-26T06:40:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85d57fa640 1 -- 172.21.15.37:0/1288451097 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f85e8005240 con 0x55e216b48fd0 2024-03-26T06:40:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85f6d86740 1 -- 172.21.15.37:0/1288451097 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85e8010430 con 0x55e216b48fd0 2024-03-26T06:40:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.464+0000 7f85d57fa640 1 -- 172.21.15.37:0/1288451097 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e8005440 con 0x55e216b48fd0 2024-03-26T06:40:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.465+0000 7f85f6d86740 10 client.26308 fetch_fsmap learned FSMap version 68 2024-03-26T06:40:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.465+0000 7f85f6d86740 1 -- 172.21.15.37:0/1288451097 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55e216bc2620 con 0x55e216b48fd0 2024-03-26T06:40:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.465+0000 7f85d57fa640 1 -- 172.21.15.37:0/1288451097 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85e8010430 con 0x55e216b48fd0 2024-03-26T06:40:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.465+0000 7f85d57fa640 1 -- 172.21.15.37:0/1288451097 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 68) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f85e8010970 con 0x55e216b48fd0 2024-03-26T06:40:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.465+0000 7f85f6d86740 10 client.26308 fetch_fsmap finished waiting for FSMap version 68 2024-03-26T06:40:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.465+0000 7f85f6d86740 10 client.26308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:40:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.465+0000 7f85f6d86740 20 client.26308 populate_metadata read hostname 'smithi037' 2024-03-26T06:40:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.465+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e216b5ad60 0x55e216bc4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.465+0000 7f85f6d86740 4 client.26308 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:40:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.465+0000 7f85f6d86740 1 -- 172.21.15.37:0/1288451097 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e216ab4e80 con 0x55e216b5ad60 2024-03-26T06:40:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.465+0000 7f85f1744640 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e216b5ad60 0x55e216bc4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.466+0000 7f85f1744640 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e216b5ad60 0x55e216bc4cc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a29f1e136e651e7d server_cookie=51349d70e5c7af8f in_seq=0 out_seq=0 2024-03-26T06:40:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.466+0000 7f85d57fa640 10 client.26308 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.797+0000 7f85d57fa640 1 -- 172.21.15.37:0/1288451097 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e216ab4e80 con 0x55e216b5ad60 2024-03-26T06:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.797+0000 7f85d57fa640 10 client.26308 handle_command_reply: tid=0 2024-03-26T06:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.797+0000 7f85f6d86740 1 client.26308 shutdown 2024-03-26T06:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.797+0000 7f85f6d86740 1 -- 172.21.15.37:0/1288451097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 msgr2=0x55e216b55190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.797+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 0x55e216b55190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f85e80099e0 tx=0x7f85e801d7b0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.797+0000 7f85f6d86740 1 -- 172.21.15.37:0/1288451097 shutdown_connections 2024-03-26T06:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.797+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x55e216b5ad60 0x55e216bc4cc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.798+0000 7f85d57fa640 0 client.26308 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.798+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e216b49d40 0x55e216b556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.798+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e216b48fd0 0x55e216b55190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.798+0000 7f85f6d86740 1 --2- 172.21.15.37:0/1288451097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e216b105b0 0x55e216b54c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.798+0000 7f85f6d86740 1 -- 172.21.15.37:0/1288451097 >> 172.21.15.37:0/1288451097 conn(0x55e216b5d330 msgr2=0x55e216b42e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.798+0000 7f85f6d86740 1 -- 172.21.15.37:0/1288451097 shutdown_connections 2024-03-26T06:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.798+0000 7f85f6d86740 1 -- 172.21.15.37:0/1288451097 wait complete. 2024-03-26T06:40:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.798+0000 7f85f6d86740 20 client.26308 trim_cache size 0 max 16384 2024-03-26T06:40:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:40:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:40:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:40:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 -- 172.21.15.37:0/4071560479 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85d80a2b60 msgr2=0x7f85d80a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85d80a2b60 0x7f85d80a5020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009270 tx=0x7f85e4009200 comp rx=0 tx=0).stop 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 -- 172.21.15.37:0/4071560479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ec0feaa0 msgr2=0x7f85ec1a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ec0feaa0 0x7f85ec1a5cb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f85e400eb10 tx=0x7f85e400c2d0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 -- 172.21.15.37:0/4071560479 shutdown_connections 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f85d80a2b60 0x7f85d80a5020 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ec0feaa0 0x7f85ec1a5cb0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85ec105510 0x7f85ec0fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 --2- 172.21.15.37:0/4071560479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85ec100440 0x7f85ec0fe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 -- 172.21.15.37:0/4071560479 >> 172.21.15.37:0/4071560479 conn(0x7f85ec077350 msgr2=0x7f85ec069740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 -- 172.21.15.37:0/4071560479 shutdown_connections 2024-03-26T06:40:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:32.799+0000 7f85f41a9640 1 -- 172.21.15.37:0/4071560479 wait complete. 2024-03-26T06:40:32.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 193797 36.26 MB/sec execute 127 sec latency 217.451 ms 2024-03-26T06:40:33.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 194585 36.36 MB/sec execute 128 sec latency 182.932 ms 2024-03-26T06:40:34.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:34 smithi066 ceph-mon[36504]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 543 op/s 2024-03-26T06:40:34.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:34 smithi037 ceph-mon[31974]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 543 op/s 2024-03-26T06:40:34.770 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:34 smithi149 ceph-mon[36681]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 543 op/s 2024-03-26T06:40:34.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 195227 36.36 MB/sec execute 129 sec latency 91.044 ms 2024-03-26T06:40:35.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 195879 36.34 MB/sec execute 130 sec latency 102.558 ms 2024-03-26T06:40:36.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:36 smithi066 ceph-mon[36504]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 739 op/s 2024-03-26T06:40:36.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:36 smithi149 ceph-mon[36681]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 739 op/s 2024-03-26T06:40:36.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:36 smithi037 ceph-mon[31974]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 739 op/s 2024-03-26T06:40:36.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 196495 36.35 MB/sec execute 131 sec latency 221.361 ms 2024-03-26T06:40:37.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 197043 36.21 MB/sec execute 132 sec latency 141.147 ms 2024-03-26T06:40:38.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:38 smithi149 ceph-mon[36681]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 823 op/s 2024-03-26T06:40:38.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:38 smithi037 ceph-mon[31974]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 823 op/s 2024-03-26T06:40:38.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:38 smithi066 ceph-mon[36504]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 823 op/s 2024-03-26T06:40:38.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 197805 36.27 MB/sec execute 133 sec latency 180.428 ms 2024-03-26T06:40:39.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 197955 36.10 MB/sec execute 134 sec latency 853.524 ms 2024-03-26T06:40:40.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 198001 35.85 MB/sec execute 135 sec latency 1619.502 ms 2024-03-26T06:40:41.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:40 smithi066 ceph-mon[36504]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.9 MiB/s wr, 669 op/s 2024-03-26T06:40:41.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:40 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:40:41.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:40 smithi149 ceph-mon[36681]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.9 MiB/s wr, 669 op/s 2024-03-26T06:40:41.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:40 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:40:41.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:40 smithi037 ceph-mon[31974]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.9 MiB/s wr, 669 op/s 2024-03-26T06:40:41.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:40 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:40:41.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 198450 35.79 MB/sec execute 136 sec latency 167.028 ms 2024-03-26T06:40:42.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 198947 35.73 MB/sec execute 137 sec latency 176.159 ms 2024-03-26T06:40:43.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:42 smithi066 ceph-mon[36504]: pgmap v560: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.07k op/s 2024-03-26T06:40:43.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:42 smithi149 ceph-mon[36681]: pgmap v560: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.07k op/s 2024-03-26T06:40:43.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:42 smithi037 ceph-mon[31974]: pgmap v560: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.07k op/s 2024-03-26T06:40:43.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 199531 35.66 MB/sec execute 138 sec latency 127.216 ms 2024-03-26T06:40:44.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:43 smithi149 ceph-mon[36681]: pgmap v561: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 775 op/s 2024-03-26T06:40:44.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:43 smithi037 ceph-mon[31974]: pgmap v561: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 775 op/s 2024-03-26T06:40:44.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:43 smithi066 ceph-mon[36504]: pgmap v561: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 775 op/s 2024-03-26T06:40:44.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 200136 35.60 MB/sec execute 139 sec latency 99.336 ms 2024-03-26T06:40:45.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 200837 35.65 MB/sec execute 140 sec latency 72.598 ms 2024-03-26T06:40:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.926+0000 7ff658b59640 1 -- 172.21.15.37:0/2001258740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6541002c0 msgr2=0x7ff65410c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.926+0000 7ff658b59640 1 --2- 172.21.15.37:0/2001258740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6541002c0 0x7ff65410c7d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff64c00ade0 tx=0x7ff64c02bee0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.926+0000 7ff658b59640 1 -- 172.21.15.37:0/2001258740 shutdown_connections 2024-03-26T06:40:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.926+0000 7ff658b59640 1 --2- 172.21.15.37:0/2001258740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6541002c0 0x7ff65410c7d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.926+0000 7ff658b59640 1 --2- 172.21.15.37:0/2001258740 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6540ff900 0x7ff6540ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.926+0000 7ff658b59640 1 --2- 172.21.15.37:0/2001258740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff654105910 0x7ff654105cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.926+0000 7ff658b59640 1 -- 172.21.15.37:0/2001258740 >> 172.21.15.37:0/2001258740 conn(0x7ff654076780 msgr2=0x7ff6540fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.926+0000 7ff658b59640 1 -- 172.21.15.37:0/2001258740 shutdown_connections 2024-03-26T06:40:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.926+0000 7ff658b59640 1 -- 172.21.15.37:0/2001258740 wait complete. 2024-03-26T06:40:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff658b59640 1 Processor -- start 2024-03-26T06:40:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff658b59640 1 -- start start 2024-03-26T06:40:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff658b59640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6540ff900 0x7ff65419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff658b59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6541002c0 0x7ff65419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff658b59640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff654105910 0x7ff654199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff658b59640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65410f4a0 con 0x7ff6541002c0 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff658b59640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65410f320 con 0x7ff6540ff900 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff658b59640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65410f620 con 0x7ff654105910 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff651d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6541002c0 0x7ff65419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff65259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6540ff900 0x7ff65419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff651d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6541002c0 0x7ff65419fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55876/0 (socket says 172.21.15.37:55876) 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff651d9b640 1 -- 172.21.15.37:0/3220568995 learned_addr learned my addr 172.21.15.37:0/3220568995 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.927+0000 7ff652d9d640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff654105910 0x7ff654199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.928+0000 7ff65259c640 1 -- 172.21.15.37:0/3220568995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff654105910 msgr2=0x7ff654199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.928+0000 7ff65259c640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff654105910 0x7ff654199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.928+0000 7ff65259c640 1 -- 172.21.15.37:0/3220568995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6541002c0 msgr2=0x7ff65419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.928+0000 7ff65259c640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6541002c0 0x7ff65419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.928+0000 7ff65259c640 1 -- 172.21.15.37:0/3220568995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65419a270 con 0x7ff6540ff900 2024-03-26T06:40:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.928+0000 7ff651d9b640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6541002c0 0x7ff65419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:40:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.928+0000 7ff65259c640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6540ff900 0x7ff65419f8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff64400e9f0 tx=0x7ff64400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.928+0000 7ff6437fe640 1 -- 172.21.15.37:0/3220568995 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64400ccc0 con 0x7ff6540ff900 2024-03-26T06:40:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.928+0000 7ff658b59640 1 -- 172.21.15.37:0/3220568995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65419a560 con 0x7ff6540ff900 2024-03-26T06:40:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.928+0000 7ff6437fe640 1 -- 172.21.15.37:0/3220568995 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff644010910 con 0x7ff6540ff900 2024-03-26T06:40:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.928+0000 7ff658b59640 1 -- 172.21.15.37:0/3220568995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6541a67b0 con 0x7ff6540ff900 2024-03-26T06:40:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.929+0000 7ff6437fe640 1 -- 172.21.15.37:0/3220568995 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644018960 con 0x7ff6540ff900 2024-03-26T06:40:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.932+0000 7ff65b807740 1 Processor -- start 2024-03-26T06:40:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.932+0000 7ff65b807740 1 -- start start 2024-03-26T06:40:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.932+0000 7ff65b807740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e56a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.932+0000 7ff65b807740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e56fd0 0x557560e57450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.932+0000 7ff65b807740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e62940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.932+0000 7ff65b807740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557560dbb5f0 con 0x557560e60550 2024-03-26T06:40:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.932+0000 7ff65b807740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557560d64930 con 0x557560e57d40 2024-03-26T06:40:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.932+0000 7ff65b807740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557560db7690 con 0x557560e56fd0 2024-03-26T06:40:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.932+0000 7ff6437fe640 1 -- 172.21.15.37:0/3220568995 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7ff644018b00 con 0x7ff6540ff900 2024-03-26T06:40:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.932+0000 7ff651d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e62940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.933+0000 7ff651d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e62940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55878/0 (socket says 172.21.15.37:55878) 2024-03-26T06:40:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.933+0000 7ff651d9b640 1 -- 172.21.15.37:0/490639540 learned_addr learned my addr 172.21.15.37:0/490639540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.933+0000 7ff652d9d640 1 --2- 172.21.15.37:0/490639540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e56a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.933+0000 7ff65259c640 1 --2- 172.21.15.37:0/490639540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e56fd0 0x557560e57450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.933+0000 7ff651d9b640 1 -- 172.21.15.37:0/490639540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e56fd0 msgr2=0x557560e57450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.933+0000 7ff651d9b640 1 --2- 172.21.15.37:0/490639540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e56fd0 0x557560e57450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.933+0000 7ff651d9b640 1 -- 172.21.15.37:0/490639540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 msgr2=0x557560e56a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.933+0000 7ff651d9b640 1 --2- 172.21.15.37:0/490639540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e56a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.933+0000 7ff651d9b640 1 -- 172.21.15.37:0/490639540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557560ad0210 con 0x557560e60550 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.933+0000 7ff651d9b640 1 --2- 172.21.15.37:0/490639540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e62940 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff6480093f0 tx=0x7ff648005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292fdb0de4ee2998 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.933+0000 7ff640ff9640 1 -- 172.21.15.37:0/490639540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64801b030 con 0x557560e60550 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.934+0000 7ff640ff9640 1 -- 172.21.15.37:0/490639540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff648005500 con 0x557560e60550 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.934+0000 7ff6437fe640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff63c0a2b10 0x7ff63c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.934+0000 7ff652d9d640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff63c0a2b10 0x7ff63c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.934+0000 7ff640ff9640 1 -- 172.21.15.37:0/490639540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648017720 con 0x557560e60550 2024-03-26T06:40:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.934+0000 7ff6437fe640 1 -- 172.21.15.37:0/3220568995 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff6440cf100 con 0x7ff6540ff900 2024-03-26T06:40:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.935+0000 7ff65b807740 1 -- 172.21.15.37:0/490639540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 msgr2=0x557560e62940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.935+0000 7ff65b807740 1 --2- 172.21.15.37:0/490639540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e62940 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff6480093f0 tx=0x7ff648005f90 comp rx=0 tx=0).stop 2024-03-26T06:40:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.935+0000 7ff65b807740 1 -- 172.21.15.37:0/490639540 shutdown_connections 2024-03-26T06:40:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.935+0000 7ff65b807740 1 --2- 172.21.15.37:0/490639540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e62940 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.935+0000 7ff65b807740 1 --2- 172.21.15.37:0/490639540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e56fd0 0x557560e57450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.935+0000 7ff65b807740 1 --2- 172.21.15.37:0/490639540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e56a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.935+0000 7ff65b807740 1 -- 172.21.15.37:0/490639540 >> 172.21.15.37:0/490639540 conn(0x557560e1e5b0 msgr2=0x557560e51cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.935+0000 7ff65b807740 1 -- 172.21.15.37:0/490639540 shutdown_connections 2024-03-26T06:40:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.935+0000 7ff65b807740 1 -- 172.21.15.37:0/490639540 wait complete. 2024-03-26T06:40:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.935+0000 7ff652d9d640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff63c0a2b10 0x7ff63c0a4fd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff64c005cc0 tx=0x7ff64c005c50 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff65b807740 1 -- start start 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff65b807740 1 Processor -- start 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff65b807740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e56fd0 0x557560e5d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff65b807740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e5d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff65b807740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff65b807740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557560e632d0 con 0x557560e60550 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff65b807740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557560d77d90 con 0x557560e57d40 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff65b807740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557560e63450 con 0x557560e56fd0 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff65b807740 10 client.0 _ll_register_callbacks cb 0x557560acfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff652d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff652d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55892/0 (socket says 172.21.15.37:55892) 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff652d9d640 1 -- 172.21.15.37:0/268109865 learned_addr learned my addr 172.21.15.37:0/268109865 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff651d9b640 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e5d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.936+0000 7ff65259c640 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e56fd0 0x557560e5d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.937+0000 7ff652d9d640 1 -- 172.21.15.37:0/268109865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e56fd0 msgr2=0x557560e5d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.937+0000 7ff652d9d640 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e56fd0 0x557560e5d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.937+0000 7ff652d9d640 1 -- 172.21.15.37:0/268109865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 msgr2=0x557560e5d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.937+0000 7ff652d9d640 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e5d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.937+0000 7ff652d9d640 1 -- 172.21.15.37:0/268109865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557560b9c620 con 0x557560e60550 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.937+0000 7ff652d9d640 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5df20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff64c005e30 tx=0x7ff64c00a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befa1425ac9c1612 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.937+0000 7ff639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.937+0000 7ff639ffb640 1 -- 172.21.15.37:0/268109865 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c039040 con 0x557560e60550 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.937+0000 7ff65b807740 1 -- 172.21.15.37:0/268109865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff648005500 con 0x557560e60550 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.937+0000 7ff639ffb640 1 -- 172.21.15.37:0/268109865 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff64c003b70 con 0x557560e60550 2024-03-26T06:40:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.938+0000 7ff639ffb640 1 -- 172.21.15.37:0/268109865 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff64c05b050 con 0x557560e60550 2024-03-26T06:40:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.938+0000 7ff65b807740 10 client.16923 fetch_fsmap learned FSMap version 70 2024-03-26T06:40:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.938+0000 7ff65b807740 1 -- 172.21.15.37:0/268109865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x557560e6f600 con 0x557560e60550 2024-03-26T06:40:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.938+0000 7ff639ffb640 1 -- 172.21.15.37:0/268109865 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff648005500 con 0x557560e60550 2024-03-26T06:40:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.938+0000 7ff639ffb640 1 -- 172.21.15.37:0/268109865 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 70) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff64c04d480 con 0x557560e60550 2024-03-26T06:40:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.938+0000 7ff65b807740 10 client.16923 fetch_fsmap finished waiting for FSMap version 70 2024-03-26T06:40:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.938+0000 7ff65b807740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:40:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.938+0000 7ff65b807740 20 client.16923 populate_metadata read hostname 'smithi037' 2024-03-26T06:40:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.938+0000 7ff65b807740 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557560e70ac0 0x557560e72f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.938+0000 7ff65b807740 4 client.16923 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:40:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.938+0000 7ff65b807740 1 -- 172.21.15.37:0/268109865 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557560dc8b30 con 0x557560e70ac0 2024-03-26T06:40:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.939+0000 7ff65259c640 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557560e70ac0 0x557560e72f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.939+0000 7ff639ffb640 10 client.16923 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.939+0000 7ff65259c640 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557560e70ac0 0x557560e72f40 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68cb19acb8da6519 server_cookie=738ba797dd22f8b4 in_seq=0 out_seq=0 2024-03-26T06:40:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.944+0000 7ff639ffb640 1 -- 172.21.15.37:0/268109865 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557560dc8b30 con 0x557560e70ac0 2024-03-26T06:40:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.944+0000 7ff639ffb640 10 client.16923 handle_command_reply: tid=0 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.944+0000 7ff65b807740 1 client.16923 shutdown 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff65b807740 1 -- 172.21.15.37:0/268109865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 msgr2=0x557560e5df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff65b807740 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5df20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff64c005e30 tx=0x7ff64c00a6d0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff65b807740 1 -- 172.21.15.37:0/268109865 shutdown_connections 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff65b807740 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557560e70ac0 0x557560e72f40 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff65b807740 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5df20 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff65b807740 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e5d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff65b807740 1 --2- 172.21.15.37:0/268109865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e56fd0 0x557560e5d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff65b807740 1 -- 172.21.15.37:0/268109865 >> 172.21.15.37:0/268109865 conn(0x557560e1e5b0 msgr2=0x557560e53a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff639ffb640 0 client.16923 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff65b807740 1 -- 172.21.15.37:0/268109865 shutdown_connections 2024-03-26T06:40:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff65b807740 1 -- 172.21.15.37:0/268109865 wait complete. 2024-03-26T06:40:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.945+0000 7ff65b807740 20 client.16923 trim_cache size 0 max 16384 2024-03-26T06:40:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.962+0000 7ff65b807740 1 Processor -- start 2024-03-26T06:40:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.962+0000 7ff65b807740 1 -- start start 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.962+0000 7ff65b807740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e56fd0 0x557560e5fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.962+0000 7ff65b807740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e57d40 0x557560e5a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.962+0000 7ff65b807740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff65b807740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63400e450 con 0x557560e60550 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff65b807740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63400fe30 con 0x557560e56fd0 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff65b807740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63400eab0 con 0x557560e57d40 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff652d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff652d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55908/0 (socket says 172.21.15.37:55908) 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff652d9d640 1 -- 172.21.15.37:0/2992217666 learned_addr learned my addr 172.21.15.37:0/2992217666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff65259c640 1 --2- 172.21.15.37:0/2992217666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e56fd0 0x557560e5fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff651d9b640 1 --2- 172.21.15.37:0/2992217666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e57d40 0x557560e5a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff652d9d640 1 -- 172.21.15.37:0/2992217666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e57d40 msgr2=0x557560e5a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff652d9d640 1 --2- 172.21.15.37:0/2992217666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e57d40 0x557560e5a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff652d9d640 1 -- 172.21.15.37:0/2992217666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e56fd0 msgr2=0x557560e5fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff652d9d640 1 --2- 172.21.15.37:0/2992217666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e56fd0 0x557560e5fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff652d9d640 1 -- 172.21.15.37:0/2992217666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557560e59080 con 0x557560e60550 2024-03-26T06:40:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff65259c640 1 --2- 172.21.15.37:0/2992217666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e56fd0 0x557560e5fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:40:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.963+0000 7ff652d9d640 1 --2- 172.21.15.37:0/2992217666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5ac40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff64c02c4f0 tx=0x7ff64c02ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35873577a9ad921 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.964+0000 7ff63bfff640 1 -- 172.21.15.37:0/2992217666 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c039070 con 0x557560e60550 2024-03-26T06:40:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.964+0000 7ff63bfff640 1 -- 172.21.15.37:0/2992217666 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff64c003e50 con 0x557560e60550 2024-03-26T06:40:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.964+0000 7ff63bfff640 1 -- 172.21.15.37:0/2992217666 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c044040 con 0x557560e60550 2024-03-26T06:40:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.964+0000 7ff65b807740 1 -- 172.21.15.37:0/2992217666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 msgr2=0x557560e5ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.964+0000 7ff65b807740 1 --2- 172.21.15.37:0/2992217666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5ac40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff64c02c4f0 tx=0x7ff64c02ca90 comp rx=0 tx=0).stop 2024-03-26T06:40:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.964+0000 7ff65b807740 1 -- 172.21.15.37:0/2992217666 shutdown_connections 2024-03-26T06:40:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.964+0000 7ff65b807740 1 --2- 172.21.15.37:0/2992217666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e60550 0x557560e5ac40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.964+0000 7ff65b807740 1 --2- 172.21.15.37:0/2992217666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e57d40 0x557560e5a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.964+0000 7ff65b807740 1 --2- 172.21.15.37:0/2992217666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e56fd0 0x557560e5fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.964+0000 7ff65b807740 1 -- 172.21.15.37:0/2992217666 >> 172.21.15.37:0/2992217666 conn(0x557560e1e5b0 msgr2=0x557560e778e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.965+0000 7ff65b807740 1 -- 172.21.15.37:0/2992217666 shutdown_connections 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.965+0000 7ff65b807740 1 -- 172.21.15.37:0/2992217666 wait complete. 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.965+0000 7ff65b807740 1 -- start start 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.965+0000 7ff65b807740 1 Processor -- start 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.965+0000 7ff65b807740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e56fd0 0x557560e65550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.965+0000 7ff65b807740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e65a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.965+0000 7ff65b807740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e60550 0x557560ec6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.965+0000 7ff65259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e56fd0 0x557560e65550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.965+0000 7ff65b807740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63400f7d0 con 0x557560e56fd0 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.965+0000 7ff65259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e56fd0 0x557560e65550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55918/0 (socket says 172.21.15.37:55918) 2024-03-26T06:40:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff65259c640 1 -- 172.21.15.37:0/3263401871 learned_addr learned my addr 172.21.15.37:0/3263401871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff65b807740 1 -- 172.21.15.37:0/3263401871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63400f110 con 0x557560e57d40 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff65b807740 1 -- 172.21.15.37:0/3263401871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63400c710 con 0x557560e60550 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff651d9b640 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e65a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff65b807740 10 client.0 _ll_register_callbacks cb 0x7ff64800d650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff652d9d640 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e60550 0x557560ec6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff652d9d640 1 -- 172.21.15.37:0/3263401871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 msgr2=0x557560e65a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff652d9d640 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e65a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff652d9d640 1 -- 172.21.15.37:0/3263401871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e56fd0 msgr2=0x557560e65550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff652d9d640 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e56fd0 0x557560e65550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff652d9d640 1 -- 172.21.15.37:0/3263401871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557560e66360 con 0x557560e60550 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff652d9d640 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e60550 0x557560ec6060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff64c04d920 tx=0x7ff64c02cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c14cd899b3107fa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff639ffb640 1 -- 172.21.15.37:0/3263401871 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c044070 con 0x557560e60550 2024-03-26T06:40:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff639ffb640 1 -- 172.21.15.37:0/3263401871 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff64c004180 con 0x557560e60550 2024-03-26T06:40:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.966+0000 7ff65b807740 1 -- 172.21.15.37:0/3263401871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff64c003e50 con 0x557560e60550 2024-03-26T06:40:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.967+0000 7ff639ffb640 1 -- 172.21.15.37:0/3263401871 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff64c039040 con 0x557560e60550 2024-03-26T06:40:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.967+0000 7ff65b807740 10 client.26326 fetch_fsmap learned FSMap version 70 2024-03-26T06:40:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.967+0000 7ff65b807740 1 -- 172.21.15.37:0/3263401871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x557560ec7690 con 0x557560e60550 2024-03-26T06:40:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.967+0000 7ff639ffb640 1 -- 172.21.15.37:0/3263401871 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff64c003e50 con 0x557560e60550 2024-03-26T06:40:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.967+0000 7ff639ffb640 1 -- 172.21.15.37:0/3263401871 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 70) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff64c0347e0 con 0x557560e60550 2024-03-26T06:40:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.967+0000 7ff65b807740 10 client.26326 fetch_fsmap finished waiting for FSMap version 70 2024-03-26T06:40:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.967+0000 7ff65b807740 10 client.26326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:40:45.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.967+0000 7ff65b807740 20 client.26326 populate_metadata read hostname 'smithi037' 2024-03-26T06:40:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.967+0000 7ff65b807740 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557560ec7c00 0x557560ec9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.967+0000 7ff65b807740 4 client.26326 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:40:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.967+0000 7ff65b807740 1 -- 172.21.15.37:0/3263401871 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557560dc2e80 con 0x557560ec7c00 2024-03-26T06:40:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.968+0000 7ff65259c640 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557560ec7c00 0x557560ec9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.968+0000 7ff65259c640 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557560ec7c00 0x557560ec9ff0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=279720f8478f1b98 server_cookie=9f412d98e32ee820 in_seq=0 out_seq=0 2024-03-26T06:40:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:45.968+0000 7ff639ffb640 10 client.26326 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:46.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:45 smithi149 ceph-mon[36681]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:40:46.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:45 smithi149 ceph-mon[36681]: pgmap v562: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 979 op/s 2024-03-26T06:40:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff639ffb640 1 -- 172.21.15.37:0/3263401871 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557560dc2e80 con 0x557560ec7c00 2024-03-26T06:40:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff639ffb640 10 client.26326 handle_command_reply: tid=0 2024-03-26T06:40:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff65b807740 1 client.26326 shutdown 2024-03-26T06:40:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff65b807740 1 -- 172.21.15.37:0/3263401871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e60550 msgr2=0x557560ec6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff65b807740 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e60550 0x557560ec6060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff64c04d920 tx=0x7ff64c02cdd0 comp rx=0 tx=0).stop 2024-03-26T06:40:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff65b807740 1 -- 172.21.15.37:0/3263401871 shutdown_connections 2024-03-26T06:40:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff65b807740 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x557560ec7c00 0x557560ec9ff0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff639ffb640 0 client.26326 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff65b807740 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557560e60550 0x557560ec6060 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff65b807740 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557560e57d40 0x557560e65a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff65b807740 1 --2- 172.21.15.37:0/3263401871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557560e56fd0 0x557560e65550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.192+0000 7ff65b807740 1 -- 172.21.15.37:0/3263401871 >> 172.21.15.37:0/3263401871 conn(0x557560e1e5b0 msgr2=0x557560e5bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.193+0000 7ff65b807740 1 -- 172.21.15.37:0/3263401871 shutdown_connections 2024-03-26T06:40:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.193+0000 7ff65b807740 1 -- 172.21.15.37:0/3263401871 wait complete. 2024-03-26T06:40:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.193+0000 7ff65b807740 20 client.26326 trim_cache size 0 max 16384 2024-03-26T06:40:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:40:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:40:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:40:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:40:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 -- 172.21.15.37:0/3220568995 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff63c0a2b10 msgr2=0x7ff63c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff63c0a2b10 0x7ff63c0a4fd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff64c005cc0 tx=0x7ff64c005c50 comp rx=0 tx=0).stop 2024-03-26T06:40:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 -- 172.21.15.37:0/3220568995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6540ff900 msgr2=0x7ff65419f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6540ff900 0x7ff65419f8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff64400e9f0 tx=0x7ff64400eeb0 comp rx=0 tx=0).stop 2024-03-26T06:40:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 -- 172.21.15.37:0/3220568995 shutdown_connections 2024-03-26T06:40:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7ff63c0a2b10 0x7ff63c0a4fd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff654105910 0x7ff654199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6541002c0 0x7ff65419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 --2- 172.21.15.37:0/3220568995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6540ff900 0x7ff65419f8f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 -- 172.21.15.37:0/3220568995 >> 172.21.15.37:0/3220568995 conn(0x7ff654076780 msgr2=0x7ff6541026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 -- 172.21.15.37:0/3220568995 shutdown_connections 2024-03-26T06:40:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:46.194+0000 7ff658b59640 1 -- 172.21.15.37:0/3220568995 wait complete. 2024-03-26T06:40:46.202 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:45 smithi037 ceph-mon[31974]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:40:46.202 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:45 smithi037 ceph-mon[31974]: pgmap v562: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 979 op/s 2024-03-26T06:40:46.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:45 smithi066 ceph-mon[36504]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T06:40:46.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:45 smithi066 ceph-mon[36504]: pgmap v562: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 21 MiB/s wr, 979 op/s 2024-03-26T06:40:46.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 201342 35.60 MB/sec execute 141 sec latency 136.445 ms 2024-03-26T06:40:47.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 202113 35.61 MB/sec execute 142 sec latency 111.239 ms 2024-03-26T06:40:48.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:48 smithi066 ceph-mon[36504]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 1.03k op/s 2024-03-26T06:40:48.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:48 smithi149 ceph-mon[36681]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 1.03k op/s 2024-03-26T06:40:48.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:48 smithi037 ceph-mon[31974]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 21 MiB/s wr, 1.03k op/s 2024-03-26T06:40:48.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 202768 35.63 MB/sec execute 143 sec latency 81.896 ms 2024-03-26T06:40:49.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 203545 35.65 MB/sec execute 144 sec latency 106.257 ms 2024-03-26T06:40:50.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:50 smithi066 ceph-mon[36504]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 847 op/s 2024-03-26T06:40:50.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:50 smithi149 ceph-mon[36681]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 847 op/s 2024-03-26T06:40:50.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:50 smithi037 ceph-mon[31974]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 847 op/s 2024-03-26T06:40:50.856 INFO:tasks.workunit.client.0.smithi037.stdout: 10 204209 35.66 MB/sec execute 145 sec latency 88.511 ms 2024-03-26T06:40:51.856 INFO:tasks.workunit.client.0.smithi037.stdout: 10 205011 35.71 MB/sec execute 146 sec latency 81.093 ms 2024-03-26T06:40:52.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:52 smithi066 ceph-mon[36504]: pgmap v565: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 23 MiB/s wr, 1.13k op/s 2024-03-26T06:40:52.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:52 smithi149 ceph-mon[36681]: pgmap v565: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 23 MiB/s wr, 1.13k op/s 2024-03-26T06:40:52.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:52 smithi037 ceph-mon[31974]: pgmap v565: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 23 MiB/s wr, 1.13k op/s 2024-03-26T06:40:52.855 INFO:tasks.workunit.client.0.smithi037.stdout: 10 205713 35.75 MB/sec execute 147 sec latency 87.307 ms 2024-03-26T06:40:53.856 INFO:tasks.workunit.client.0.smithi037.stdout: 10 206344 35.71 MB/sec execute 148 sec latency 105.441 ms 2024-03-26T06:40:54.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:54 smithi066 ceph-mon[36504]: pgmap v566: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.2 MiB/s wr, 733 op/s 2024-03-26T06:40:54.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:54 smithi149 ceph-mon[36681]: pgmap v566: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.2 MiB/s wr, 733 op/s 2024-03-26T06:40:54.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:54 smithi037 ceph-mon[31974]: pgmap v566: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.2 MiB/s wr, 733 op/s 2024-03-26T06:40:54.856 INFO:tasks.workunit.client.0.smithi037.stdout: 10 207146 35.81 MB/sec execute 149 sec latency 100.704 ms 2024-03-26T06:40:55.856 INFO:tasks.workunit.client.0.smithi037.stdout: 10 207725 35.74 MB/sec execute 150 sec latency 96.700 ms 2024-03-26T06:40:56.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:56 smithi066 ceph-mon[36504]: pgmap v567: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 858 op/s 2024-03-26T06:40:56.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:56 smithi149 ceph-mon[36681]: pgmap v567: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 858 op/s 2024-03-26T06:40:56.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:56 smithi037 ceph-mon[31974]: pgmap v567: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 858 op/s 2024-03-26T06:40:56.856 INFO:tasks.workunit.client.0.smithi037.stdout: 10 208391 35.72 MB/sec execute 151 sec latency 167.614 ms 2024-03-26T06:40:57.856 INFO:tasks.workunit.client.0.smithi037.stdout: 10 209159 35.80 MB/sec execute 152 sec latency 78.216 ms 2024-03-26T06:40:58.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:40:58 smithi066 ceph-mon[36504]: pgmap v568: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.1 MiB/s wr, 782 op/s 2024-03-26T06:40:58.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:40:58 smithi149 ceph-mon[36681]: pgmap v568: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.1 MiB/s wr, 782 op/s 2024-03-26T06:40:58.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:40:58 smithi037 ceph-mon[31974]: pgmap v568: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.1 MiB/s wr, 782 op/s 2024-03-26T06:40:58.856 INFO:tasks.workunit.client.0.smithi037.stdout: 10 209838 35.77 MB/sec execute 153 sec latency 113.484 ms 2024-03-26T06:40:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.322+0000 7f53db8c6640 1 -- 172.21.15.37:0/1851505595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d4069d00 msgr2=0x7f53d406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.322+0000 7f53db8c6640 1 --2- 172.21.15.37:0/1851505595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d4069d00 0x7f53d406a0e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009900 tx=0x7f53d002bc30 comp rx=0 tx=0).stop 2024-03-26T06:40:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.322+0000 7f53db8c6640 1 -- 172.21.15.37:0/1851505595 shutdown_connections 2024-03-26T06:40:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.322+0000 7f53db8c6640 1 --2- 172.21.15.37:0/1851505595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53d41065e0 0x7f53d410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.322+0000 7f53db8c6640 1 --2- 172.21.15.37:0/1851505595 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53d406a6b0 0x7f53d4105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.322+0000 7f53db8c6640 1 --2- 172.21.15.37:0/1851505595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d4069d00 0x7f53d406a0e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.322+0000 7f53db8c6640 1 -- 172.21.15.37:0/1851505595 >> 172.21.15.37:0/1851505595 conn(0x7f53d40771d0 msgr2=0x7f53d40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.322+0000 7f53db8c6640 1 -- 172.21.15.37:0/1851505595 shutdown_connections 2024-03-26T06:40:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.322+0000 7f53db8c6640 1 -- 172.21.15.37:0/1851505595 wait complete. 2024-03-26T06:40:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53db8c6640 1 Processor -- start 2024-03-26T06:40:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53db8c6640 1 -- start start 2024-03-26T06:40:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53db8c6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53d4069d00 0x7f53d4072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53db8c6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53d41065e0 0x7f53d406bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53db8c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d4073130 0x7f53d406c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53db8c6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4111690 con 0x7f53d4073130 2024-03-26T06:40:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53d9e63640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d4073130 0x7f53d406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53d9e63640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d4073130 0x7f53d406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46490/0 (socket says 172.21.15.37:46490) 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53d9e63640 1 -- 172.21.15.37:0/166230686 learned_addr learned my addr 172.21.15.37:0/166230686 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53d9662640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53d4069d00 0x7f53d4072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53d8e61640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53d41065e0 0x7f53d406bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53db8c6640 1 -- 172.21.15.37:0/166230686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4111510 con 0x7f53d41065e0 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.323+0000 7f53db8c6640 1 -- 172.21.15.37:0/166230686 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d4111810 con 0x7f53d4069d00 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.324+0000 7f53d9e63640 1 -- 172.21.15.37:0/166230686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53d4069d00 msgr2=0x7f53d4072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.324+0000 7f53d9e63640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53d4069d00 0x7f53d4072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.324+0000 7f53d9e63640 1 -- 172.21.15.37:0/166230686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53d41065e0 msgr2=0x7f53d406bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.324+0000 7f53d9e63640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53d41065e0 0x7f53d406bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.324+0000 7f53d9e63640 1 -- 172.21.15.37:0/166230686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4070600 con 0x7f53d4073130 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.324+0000 7f53d8e61640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53d41065e0 0x7f53d406bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.324+0000 7f53d9e63640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d4073130 0x7f53d406c170 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00eb10 tx=0x7f53cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.325+0000 7f53ca7fc640 1 -- 172.21.15.37:0/166230686 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc004b20 con 0x7f53d4073130 2024-03-26T06:40:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.325+0000 7f53ca7fc640 1 -- 172.21.15.37:0/166230686 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53cc004cc0 con 0x7f53d4073130 2024-03-26T06:40:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.325+0000 7f53ca7fc640 1 -- 172.21.15.37:0/166230686 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc010760 con 0x7f53d4073130 2024-03-26T06:40:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.326+0000 7f53db8c6640 1 -- 172.21.15.37:0/166230686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d40708f0 con 0x7f53d4073130 2024-03-26T06:40:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.326+0000 7f53db8c6640 1 -- 172.21.15.37:0/166230686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d4070d90 con 0x7f53d4073130 2024-03-26T06:40:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.328+0000 7f53ca7fc640 1 -- 172.21.15.37:0/166230686 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f53cc005020 con 0x7f53d4073130 2024-03-26T06:40:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.329+0000 7f53ca7fc640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53c00a2a10 0x7f53c00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.329+0000 7f53d9662640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53c00a2a10 0x7f53c00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.330+0000 7f53ca7fc640 1 -- 172.21.15.37:0/166230686 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f53cc0cf420 con 0x7f53d4073130 2024-03-26T06:40:59.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.330+0000 7f53de585740 1 Processor -- start 2024-03-26T06:40:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.330+0000 7f53de585740 1 -- start start 2024-03-26T06:40:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.330+0000 7f53d9662640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53c00a2a10 0x7f53c00a4ed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53d0006010 tx=0x7f53d0005dd0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.330+0000 7f53de585740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651956dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.330+0000 7f53de585740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x55651956e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.330+0000 7f53de585740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x556519579d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.330+0000 7f53de585740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565194ce5f0 con 0x55651956f150 2024-03-26T06:40:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.330+0000 7f53de585740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556519477930 con 0x556519577960 2024-03-26T06:40:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.330+0000 7f53de585740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565194ca690 con 0x55651956e3e0 2024-03-26T06:40:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.331+0000 7f53d9662640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651956dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.331+0000 7f53d9e63640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x556519579d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.331+0000 7f53d9662640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651956dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46494/0 (socket says 172.21.15.37:46494) 2024-03-26T06:40:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.331+0000 7f53d9662640 1 -- 172.21.15.37:0/3968676397 learned_addr learned my addr 172.21.15.37:0/3968676397 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.331+0000 7f53d8e61640 1 --2- 172.21.15.37:0/3968676397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x55651956e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.331+0000 7f53d9662640 1 -- 172.21.15.37:0/3968676397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 msgr2=0x55651956e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.331+0000 7f53d9662640 1 --2- 172.21.15.37:0/3968676397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x55651956e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.331+0000 7f53d9662640 1 -- 172.21.15.37:0/3968676397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 msgr2=0x556519579d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.331+0000 7f53d9662640 1 --2- 172.21.15.37:0/3968676397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x556519579d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.331+0000 7f53d9662640 1 -- 172.21.15.37:0/3968676397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565191e3210 con 0x55651956f150 2024-03-26T06:40:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.331+0000 7f53d9662640 1 --2- 172.21.15.37:0/3968676397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651956dea0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009900 tx=0x7f53d003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be77e2d867c1fed server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.332+0000 7f53bf7fe640 1 -- 172.21.15.37:0/3968676397 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00041c0 con 0x55651956f150 2024-03-26T06:40:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.332+0000 7f53bf7fe640 1 -- 172.21.15.37:0/3968676397 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53d0004ba0 con 0x55651956f150 2024-03-26T06:40:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.332+0000 7f53bf7fe640 1 -- 172.21.15.37:0/3968676397 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00035f0 con 0x55651956f150 2024-03-26T06:40:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.333+0000 7f53de585740 1 -- 172.21.15.37:0/3968676397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 msgr2=0x55651956dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.333+0000 7f53de585740 1 --2- 172.21.15.37:0/3968676397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651956dea0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009900 tx=0x7f53d003f000 comp rx=0 tx=0).stop 2024-03-26T06:40:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.333+0000 7f53de585740 1 -- 172.21.15.37:0/3968676397 shutdown_connections 2024-03-26T06:40:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.333+0000 7f53de585740 1 --2- 172.21.15.37:0/3968676397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x556519579d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.333+0000 7f53de585740 1 --2- 172.21.15.37:0/3968676397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x55651956e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.333+0000 7f53de585740 1 --2- 172.21.15.37:0/3968676397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651956dea0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.333+0000 7f53de585740 1 -- 172.21.15.37:0/3968676397 >> 172.21.15.37:0/3968676397 conn(0x5565195315b0 msgr2=0x5565195690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.333+0000 7f53de585740 1 -- 172.21.15.37:0/3968676397 shutdown_connections 2024-03-26T06:40:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.333+0000 7f53de585740 1 -- 172.21.15.37:0/3968676397 wait complete. 2024-03-26T06:40:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.333+0000 7f53de585740 1 -- start start 2024-03-26T06:40:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53de585740 1 Processor -- start 2024-03-26T06:40:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53de585740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x556519574a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53de585740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55651956f150 0x556519574fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53de585740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556519577960 0x556519575500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53d9662640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x556519574a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53de585740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651957a630 con 0x556519577960 2024-03-26T06:40:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53d9662640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x556519574a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.37:48342/0 (socket says 172.21.15.37:48342) 2024-03-26T06:40:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53d9662640 1 -- 172.21.15.37:0/186735252 learned_addr learned my addr 172.21.15.37:0/186735252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53de585740 1 -- 172.21.15.37:0/186735252 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651948ad90 con 0x55651956f150 2024-03-26T06:40:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53de585740 1 -- 172.21.15.37:0/186735252 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651957a7b0 con 0x55651956e3e0 2024-03-26T06:40:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53d9e63640 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556519577960 0x556519575500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53d8e61640 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55651956f150 0x556519574fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.334+0000 7f53de585740 10 client.0 _ll_register_callbacks cb 0x5565191e2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.335+0000 7f53d9662640 1 -- 172.21.15.37:0/186735252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55651956f150 msgr2=0x556519574fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.335+0000 7f53d9662640 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55651956f150 0x556519574fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.335+0000 7f53d9662640 1 -- 172.21.15.37:0/186735252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556519577960 msgr2=0x556519575500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.335+0000 7f53d9662640 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556519577960 0x556519575500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.335+0000 7f53d9662640 1 -- 172.21.15.37:0/186735252 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565192af620 con 0x55651956e3e0 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.335+0000 7f53d8e61640 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55651956f150 0x556519574fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.335+0000 7f53d9662640 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x556519574a80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f53d0036040 tx=0x7f53d00043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=717de212b9148d81 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.335+0000 7f53bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.335+0000 7f53bcff9640 1 -- 172.21.15.37:0/186735252 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00048d0 con 0x55651956e3e0 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.335+0000 7f53bcff9640 1 -- 172.21.15.37:0/186735252 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f53d00035f0 con 0x55651956e3e0 2024-03-26T06:40:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.335+0000 7f53de585740 1 -- 172.21.15.37:0/186735252 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53d0004ba0 con 0x55651956e3e0 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53bcff9640 1 -- 172.21.15.37:0/186735252 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53d00037f0 con 0x55651956e3e0 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53de585740 10 client.26338 fetch_fsmap learned FSMap version 70 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53de585740 1 -- 172.21.15.37:0/186735252 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55651957a630 con 0x55651956e3e0 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53bcff9640 1 -- 172.21.15.37:0/186735252 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53d0004ba0 con 0x55651956e3e0 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53bcff9640 1 -- 172.21.15.37:0/186735252 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 70) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f53d005c4f0 con 0x55651956e3e0 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53de585740 10 client.26338 fetch_fsmap finished waiting for FSMap version 70 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53de585740 10 client.26338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53de585740 20 client.26338 populate_metadata read hostname 'smithi037' 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53de585740 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556519587b60 0x556519589f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53de585740 4 client.26338 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53de585740 1 -- 172.21.15.37:0/186735252 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565194dbb30 con 0x556519587b60 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.336+0000 7f53d8e61640 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556519587b60 0x556519589f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.337+0000 7f53bcff9640 10 client.26338 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.337+0000 7f53d8e61640 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556519587b60 0x556519589f50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa00d2748ff12ebf server_cookie=cc19b81528a556ea in_seq=0 out_seq=0 2024-03-26T06:40:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.338+0000 7f53bcff9640 1 -- 172.21.15.37:0/186735252 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565194dbb30 con 0x556519587b60 2024-03-26T06:40:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.338+0000 7f53bcff9640 10 client.26338 handle_command_reply: tid=0 2024-03-26T06:40:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.338+0000 7f53de585740 1 client.26338 shutdown 2024-03-26T06:40:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.339+0000 7f53de585740 1 -- 172.21.15.37:0/186735252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 msgr2=0x556519574a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.339+0000 7f53de585740 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x556519574a80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f53d0036040 tx=0x7f53d00043f0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.339+0000 7f53de585740 1 -- 172.21.15.37:0/186735252 shutdown_connections 2024-03-26T06:40:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.339+0000 7f53de585740 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556519587b60 0x556519589f50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.339+0000 7f53bcff9640 0 client.26338 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.339+0000 7f53de585740 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556519577960 0x556519575500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.339+0000 7f53de585740 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55651956f150 0x556519574fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.340+0000 7f53de585740 1 --2- 172.21.15.37:0/186735252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x556519574a80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.340+0000 7f53de585740 1 -- 172.21.15.37:0/186735252 >> 172.21.15.37:0/186735252 conn(0x5565195315b0 msgr2=0x55651956a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.340+0000 7f53de585740 1 -- 172.21.15.37:0/186735252 shutdown_connections 2024-03-26T06:40:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.340+0000 7f53de585740 1 -- 172.21.15.37:0/186735252 wait complete. 2024-03-26T06:40:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.340+0000 7f53de585740 20 client.26338 trim_cache size 0 max 16384 2024-03-26T06:40:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53de585740 1 Processor -- start 2024-03-26T06:40:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53de585740 1 -- start start 2024-03-26T06:40:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53de585740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956e3e0 0x55651956abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53de585740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956f150 0x55651957ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53de585740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x55651957ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53de585740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41b3280 con 0x55651956e3e0 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53d9662640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956e3e0 0x55651956abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53d9662640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956e3e0 0x55651956abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46504/0 (socket says 172.21.15.37:46504) 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53d9662640 1 -- 172.21.15.37:0/1067353921 learned_addr learned my addr 172.21.15.37:0/1067353921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53d8e61640 1 --2- 172.21.15.37:0/1067353921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956f150 0x55651957ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53de585740 1 -- 172.21.15.37:0/1067353921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41b4c60 con 0x556519577960 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53de585740 1 -- 172.21.15.37:0/1067353921 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41b38e0 con 0x55651956f150 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.361+0000 7f53d9e63640 1 --2- 172.21.15.37:0/1067353921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x55651957ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.362+0000 7f53d8e61640 1 -- 172.21.15.37:0/1067353921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 msgr2=0x55651957ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.362+0000 7f53d8e61640 1 --2- 172.21.15.37:0/1067353921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x55651957ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.362+0000 7f53d8e61640 1 -- 172.21.15.37:0/1067353921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956e3e0 msgr2=0x55651956abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.362+0000 7f53d8e61640 1 --2- 172.21.15.37:0/1067353921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956e3e0 0x55651956abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.362+0000 7f53d8e61640 1 -- 172.21.15.37:0/1067353921 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55651956d6b0 con 0x55651956f150 2024-03-26T06:40:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.362+0000 7f53d9662640 1 --2- 172.21.15.37:0/1067353921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956e3e0 0x55651956abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:40:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.362+0000 7f53d8e61640 1 --2- 172.21.15.37:0/1067353921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956f150 0x55651957ea30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5565195742e0 tx=0x7f53c40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45eb53c921e7616b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.362+0000 7f53beffd640 1 -- 172.21.15.37:0/1067353921 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4003de0 con 0x55651956f150 2024-03-26T06:40:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.362+0000 7f53beffd640 1 -- 172.21.15.37:0/1067353921 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53c4023030 con 0x55651956f150 2024-03-26T06:40:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.362+0000 7f53beffd640 1 -- 172.21.15.37:0/1067353921 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4012980 con 0x55651956f150 2024-03-26T06:40:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.363+0000 7f53de585740 1 -- 172.21.15.37:0/1067353921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956f150 msgr2=0x55651957ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:40:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.363+0000 7f53de585740 1 --2- 172.21.15.37:0/1067353921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956f150 0x55651957ea30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5565195742e0 tx=0x7f53c40046c0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.363+0000 7f53de585740 1 -- 172.21.15.37:0/1067353921 shutdown_connections 2024-03-26T06:40:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.363+0000 7f53de585740 1 --2- 172.21.15.37:0/1067353921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x55651957ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.363+0000 7f53de585740 1 --2- 172.21.15.37:0/1067353921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956f150 0x55651957ea30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.363+0000 7f53de585740 1 --2- 172.21.15.37:0/1067353921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956e3e0 0x55651956abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.363+0000 7f53de585740 1 -- 172.21.15.37:0/1067353921 >> 172.21.15.37:0/1067353921 conn(0x5565195315b0 msgr2=0x55651956a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:40:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.363+0000 7f53de585740 1 -- 172.21.15.37:0/1067353921 shutdown_connections 2024-03-26T06:40:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.363+0000 7f53de585740 1 -- 172.21.15.37:0/1067353921 wait complete. 2024-03-26T06:40:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53de585740 1 -- start start 2024-03-26T06:40:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53de585740 1 Processor -- start 2024-03-26T06:40:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53de585740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x55651957d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53de585740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651957daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53de585740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x55651957e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53de585740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41b4600 con 0x55651956f150 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53de585740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41b3f40 con 0x556519577960 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53de585740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d41b1540 con 0x55651956e3e0 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53de585740 10 client.0 _ll_register_callbacks cb 0x7f53d0042070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53d8e61640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651957daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53d8e61640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651957daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46516/0 (socket says 172.21.15.37:46516) 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53d8e61640 1 -- 172.21.15.37:0/1141231504 learned_addr learned my addr 172.21.15.37:0/1141231504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53d9662640 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x55651957d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.364+0000 7f53d9e63640 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x55651957e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53d9662640 1 -- 172.21.15.37:0/1141231504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 msgr2=0x55651957e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53d9662640 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x55651957e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53d9662640 1 -- 172.21.15.37:0/1141231504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 msgr2=0x55651957daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53d9662640 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651957daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53d9662640 1 -- 172.21.15.37:0/1141231504 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565195de060 con 0x55651956e3e0 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53d8e61640 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651957daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53d9e63640 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x55651957e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53d9662640 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x55651957d5b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f53d002c930 tx=0x7f53d005c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4eededb0946239 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53bcff9640 1 -- 172.21.15.37:0/1141231504 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0005850 con 0x55651956e3e0 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53de585740 1 -- 172.21.15.37:0/1141231504 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53c4023030 con 0x55651956e3e0 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.365+0000 7f53bcff9640 1 -- 172.21.15.37:0/1141231504 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f53d0058040 con 0x55651956e3e0 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53bcff9640 1 -- 172.21.15.37:0/1141231504 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53d0074050 con 0x55651956e3e0 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53de585740 10 client.26344 fetch_fsmap learned FSMap version 70 2024-03-26T06:40:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53de585740 1 -- 172.21.15.37:0/1141231504 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5565195df350 con 0x55651956e3e0 2024-03-26T06:40:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53bcff9640 1 -- 172.21.15.37:0/1141231504 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53c4023030 con 0x55651956e3e0 2024-03-26T06:40:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53bcff9640 1 -- 172.21.15.37:0/1141231504 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 70) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f53d00050d0 con 0x55651956e3e0 2024-03-26T06:40:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53de585740 10 client.26344 fetch_fsmap finished waiting for FSMap version 70 2024-03-26T06:40:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53de585740 10 client.26344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T06:40:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53de585740 20 client.26344 populate_metadata read hostname 'smithi037' 2024-03-26T06:40:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53de585740 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556519587b60 0x5565195e19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:40:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53de585740 4 client.26344 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T06:40:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53de585740 1 -- 172.21.15.37:0/1141231504 --> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565194d5e80 con 0x556519587b60 2024-03-26T06:40:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.366+0000 7f53d8e61640 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556519587b60 0x5565195e19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:40:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.367+0000 7f53bcff9640 10 client.26344 ms_handle_connect on v2:172.21.15.66:6838/1420321367 2024-03-26T06:40:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:40:59.367+0000 7f53d8e61640 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556519587b60 0x5565195e19d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f458d72e17e8709 server_cookie=7e4cb07cc07cc4ee in_seq=0 out_seq=0 2024-03-26T06:40:59.856 INFO:tasks.workunit.client.0.smithi037.stdout: 10 210261 35.71 MB/sec execute 154 sec latency 431.555 ms 2024-03-26T06:41:00.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:00 smithi149 ceph-mon[36681]: pgmap v569: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.1 MiB/s wr, 538 op/s 2024-03-26T06:41:00.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:00 smithi037 ceph-mon[31974]: pgmap v569: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.1 MiB/s wr, 538 op/s 2024-03-26T06:41:00.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:00 smithi066 ceph-mon[36504]: pgmap v569: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.1 MiB/s wr, 538 op/s 2024-03-26T06:41:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.674+0000 7f53bcff9640 1 -- 172.21.15.37:0/1141231504 <== mds.0 v2:172.21.15.66:6838/1420321367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565194d5e80 con 0x556519587b60 2024-03-26T06:41:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.674+0000 7f53bcff9640 10 client.26344 handle_command_reply: tid=0 2024-03-26T06:41:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.675+0000 7f53de585740 1 client.26344 shutdown 2024-03-26T06:41:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.675+0000 7f53de585740 1 -- 172.21.15.37:0/1141231504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 msgr2=0x55651957d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.675+0000 7f53de585740 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x55651957d5b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f53d002c930 tx=0x7f53d005c6f0 comp rx=0 tx=0).stop 2024-03-26T06:41:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.675+0000 7f53de585740 1 -- 172.21.15.37:0/1141231504 shutdown_connections 2024-03-26T06:41:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.675+0000 7f53de585740 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.66:6838/1420321367,v1:172.21.15.66:6839/1420321367] conn(0x556519587b60 0x5565195e19d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.675+0000 7f53de585740 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556519577960 0x55651957e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.675+0000 7f53bcff9640 0 client.26344 ms_handle_reset on v2:172.21.15.66:6838/1420321367 2024-03-26T06:41:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.675+0000 7f53de585740 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55651956f150 0x55651957daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.675+0000 7f53de585740 1 --2- 172.21.15.37:0/1141231504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55651956e3e0 0x55651957d5b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.675+0000 7f53de585740 1 -- 172.21.15.37:0/1141231504 >> 172.21.15.37:0/1141231504 conn(0x5565195315b0 msgr2=0x556519573350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.676+0000 7f53de585740 1 -- 172.21.15.37:0/1141231504 shutdown_connections 2024-03-26T06:41:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.676+0000 7f53de585740 1 -- 172.21.15.37:0/1141231504 wait complete. 2024-03-26T06:41:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.676+0000 7f53de585740 20 client.26344 trim_cache size 0 max 16384 2024-03-26T06:41:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-03-26T06:41:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-03-26T06:41:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-03-26T06:41:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-03-26T06:41:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.677+0000 7f53db8c6640 1 -- 172.21.15.37:0/166230686 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53c00a2a10 msgr2=0x7f53c00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.677+0000 7f53db8c6640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53c00a2a10 0x7f53c00a4ed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53d0006010 tx=0x7f53d0005dd0 comp rx=0 tx=0).stop 2024-03-26T06:41:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.677+0000 7f53db8c6640 1 -- 172.21.15.37:0/166230686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d4073130 msgr2=0x7f53d406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.677+0000 7f53db8c6640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d4073130 0x7f53d406c170 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00eb10 tx=0x7f53cc00c2d0 comp rx=0 tx=0).stop 2024-03-26T06:41:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.677+0000 7f53db8c6640 1 -- 172.21.15.37:0/166230686 shutdown_connections 2024-03-26T06:41:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.677+0000 7f53db8c6640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f53c00a2a10 0x7f53c00a4ed0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.677+0000 7f53db8c6640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53d4073130 0x7f53d406c170 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.677+0000 7f53db8c6640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53d41065e0 0x7f53d406bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.677+0000 7f53db8c6640 1 --2- 172.21.15.37:0/166230686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53d4069d00 0x7f53d4072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.677+0000 7f53db8c6640 1 -- 172.21.15.37:0/166230686 >> 172.21.15.37:0/166230686 conn(0x7f53d40771d0 msgr2=0x7f53d40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.678+0000 7f53db8c6640 1 -- 172.21.15.37:0/166230686 shutdown_connections 2024-03-26T06:41:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-03-26T06:41:00.678+0000 7f53db8c6640 1 -- 172.21.15.37:0/166230686 wait complete. 2024-03-26T06:41:00.856 INFO:tasks.workunit.client.0.smithi037.stdout: 10 210271 35.48 MB/sec execute 155 sec latency 845.109 ms 2024-03-26T06:41:01.421 INFO:tasks.workunit.client.0.smithi037.stdout:[214978] write failed on handle 10600 (Resource temporarily unavailable) 2024-03-26T06:41:01.421 INFO:tasks.workunit.client.0.smithi037.stdout:Child failed with status 1 2024-03-26T06:41:01.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:41:01.466 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-03-26T06:41:01.466 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-26T06:41:01.922 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_7e98f5f6b535defc7fa715b456904ffc0500aa9f/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_7e98f5f6b535defc7fa715b456904ffc0500aa9f/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_7e98f5f6b535defc7fa715b456904ffc0500aa9f/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi037 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7e98f5f6b535defc7fa715b456904ffc0500aa9f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-26T06:41:02.149 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5ecbdf16442b4ecf84e4ade9321961ee Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_7e98f5f6b535defc7fa715b456904ffc0500aa9f/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_7e98f5f6b535defc7fa715b456904ffc0500aa9f/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_7e98f5f6b535defc7fa715b456904ffc0500aa9f/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi037 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7e98f5f6b535defc7fa715b456904ffc0500aa9f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-26T06:41:02.152 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-26T06:41:02.162 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi037.front.sepia.ceph.com 2024-03-26T06:41:02.175 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-26T06:41:02.197 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-26T06:41:02.197 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T06:41:02.197 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-26T06:41:02.198 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-26T06:41:02.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:02 smithi037 ceph-mon[31974]: pgmap v570: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 17 MiB/s wr, 932 op/s 2024-03-26T06:41:02.228 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-26T06:41:02.296 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-26T06:41:02.402 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:02 smithi066 ceph-mon[36504]: pgmap v570: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 17 MiB/s wr, 932 op/s 2024-03-26T06:41:02.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:02 smithi149 ceph-mon[36681]: pgmap v570: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 17 MiB/s wr, 932 op/s 2024-03-26T06:41:02.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.440+0000 7f9478e85640 1 -- 172.21.15.37:0/3953170798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94741065d0 msgr2=0x7f947410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:02.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.440+0000 7f9478e85640 1 --2- 172.21.15.37:0/3953170798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94741065d0 0x7f947410e1a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f946c009ca0 tx=0x7f946c02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.440+0000 7f9478e85640 1 -- 172.21.15.37:0/3953170798 shutdown_connections 2024-03-26T06:41:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.440+0000 7f9478e85640 1 --2- 172.21.15.37:0/3953170798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94741065d0 0x7f947410e1a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.440+0000 7f9478e85640 1 --2- 172.21.15.37:0/3953170798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9474101eb0 0x7f9474105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.440+0000 7f9478e85640 1 --2- 172.21.15.37:0/3953170798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9474101500 0x7f94741018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.440+0000 7f9478e85640 1 -- 172.21.15.37:0/3953170798 >> 172.21.15.37:0/3953170798 conn(0x7f94740fcd80 msgr2=0x7f94740ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.440+0000 7f9478e85640 1 -- 172.21.15.37:0/3953170798 shutdown_connections 2024-03-26T06:41:02.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.440+0000 7f9478e85640 1 -- 172.21.15.37:0/3953170798 wait complete. 2024-03-26T06:41:02.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f9478e85640 1 Processor -- start 2024-03-26T06:41:02.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f9478e85640 1 -- start start 2024-03-26T06:41:02.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f9478e85640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9474101500 0x7f947419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:02.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f9478e85640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94741065d0 0x7f947419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:02.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f9478e85640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9474197890 0x7f9474197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:02.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f9478e85640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9474111670 con 0x7f9474197890 2024-03-26T06:41:02.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f9478e85640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94741114f0 con 0x7f94741065d0 2024-03-26T06:41:02.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f9478e85640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94741117f0 con 0x7f9474101500 2024-03-26T06:41:02.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f9472d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9474197890 0x7f9474197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:02.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f946bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94741065d0 0x7f947419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:02.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f9472d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9474197890 0x7f9474197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46528/0 (socket says 172.21.15.37:46528) 2024-03-26T06:41:02.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f9472d9d640 1 -- 172.21.15.37:0/4161504417 learned_addr learned my addr 172.21.15.37:0/4161504417 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:02.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.441+0000 7f947259c640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9474101500 0x7f947419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:02.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f946bfff640 1 -- 172.21.15.37:0/4161504417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9474101500 msgr2=0x7f947419d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:02.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f946bfff640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9474101500 0x7f947419d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f946bfff640 1 -- 172.21.15.37:0/4161504417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9474197890 msgr2=0x7f9474197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:02.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f946bfff640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9474197890 0x7f9474197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f946bfff640 1 -- 172.21.15.37:0/4161504417 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94741983b0 con 0x7f94741065d0 2024-03-26T06:41:02.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f9472d9d640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9474197890 0x7f9474197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:41:02.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f946bfff640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94741065d0 0x7f947419dc20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f945c009b90 tx=0x7f945c00b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:02.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f946b7fe640 1 -- 172.21.15.37:0/4161504417 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c01e070 con 0x7f94741065d0 2024-03-26T06:41:02.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f9478e85640 1 -- 172.21.15.37:0/4161504417 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94741986a0 con 0x7f94741065d0 2024-03-26T06:41:02.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f9478e85640 1 -- 172.21.15.37:0/4161504417 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94740738c0 con 0x7f94741065d0 2024-03-26T06:41:02.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f946b7fe640 1 -- 172.21.15.37:0/4161504417 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f945c0198f0 con 0x7f94741065d0 2024-03-26T06:41:02.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.442+0000 7f946b7fe640 1 -- 172.21.15.37:0/4161504417 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c022610 con 0x7f94741065d0 2024-03-26T06:41:02.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.443+0000 7f9478e85640 1 -- 172.21.15.37:0/4161504417 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947404f050 con 0x7f94741065d0 2024-03-26T06:41:02.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.444+0000 7f946b7fe640 1 -- 172.21.15.37:0/4161504417 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f945c021070 con 0x7f94741065d0 2024-03-26T06:41:02.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.445+0000 7f946b7fe640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f94600a2a10 0x7f94600a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:02.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.445+0000 7f947259c640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f94600a2a10 0x7f94600a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:02.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.445+0000 7f946b7fe640 1 -- 172.21.15.37:0/4161504417 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f945c0d2a20 con 0x7f94741065d0 2024-03-26T06:41:02.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.445+0000 7f947259c640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f94600a2a10 0x7f94600a4ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94741989d0 tx=0x7f946400b410 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:02.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.449+0000 7f946b7fe640 1 -- 172.21.15.37:0/4161504417 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f945c09f130 con 0x7f94741065d0 2024-03-26T06:41:02.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.657+0000 7f9478e85640 1 -- 172.21.15.37:0/4161504417 --> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f9474109870 con 0x7f94600a2a10 2024-03-26T06:41:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.668+0000 7f946b7fe640 1 -- 172.21.15.37:0/4161504417 <== mgr.15411 v2:172.21.15.37:6800/3894397572 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f946400bc40 con 0x7f94600a2a10 2024-03-26T06:41:02.671 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-26T00:00:00", "created": "2024-03-26T06:22:59", "first": "2024-03-26T06:24:00", "last": "2024-03-26T06:41:00", "last_pruned": "2024-03-26T06:41:01", "created_count": 18, "pruned_count": 12, "active": true} 2024-03-26T06:41:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 -- 172.21.15.37:0/4161504417 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f94600a2a10 msgr2=0x7f94600a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f94600a2a10 0x7f94600a4ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94741989d0 tx=0x7f946400b410 comp rx=0 tx=0).stop 2024-03-26T06:41:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 -- 172.21.15.37:0/4161504417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94741065d0 msgr2=0x7f947419dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94741065d0 0x7f947419dc20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f945c009b90 tx=0x7f945c00b690 comp rx=0 tx=0).stop 2024-03-26T06:41:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 -- 172.21.15.37:0/4161504417 shutdown_connections 2024-03-26T06:41:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f94600a2a10 0x7f94600a4ed0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9474197890 0x7f9474197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94741065d0 0x7f947419dc20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 --2- 172.21.15.37:0/4161504417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9474101500 0x7f947419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 -- 172.21.15.37:0/4161504417 >> 172.21.15.37:0/4161504417 conn(0x7f94740fcd80 msgr2=0x7f94740fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 -- 172.21.15.37:0/4161504417 shutdown_connections 2024-03-26T06:41:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.670+0000 7f9478e85640 1 -- 172.21.15.37:0/4161504417 wait complete. 2024-03-26T06:41:02.685 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-26T06:41:02.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.786+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3764091332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec0100300 msgr2=0x7f3ec010c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:02.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.786+0000 7f3ec4ccd640 1 --2- 172.21.15.37:0/3764091332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec0100300 0x7f3ec010c810 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009cd0 tx=0x7f3eb802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.786+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3764091332 shutdown_connections 2024-03-26T06:41:02.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.786+0000 7f3ec4ccd640 1 --2- 172.21.15.37:0/3764091332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec0100300 0x7f3ec010c810 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.786+0000 7f3ec4ccd640 1 --2- 172.21.15.37:0/3764091332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ec00ff940 0x7f3ec00ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.786+0000 7f3ec4ccd640 1 --2- 172.21.15.37:0/3764091332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ec0105950 0x7f3ec0105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.786+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3764091332 >> 172.21.15.37:0/3764091332 conn(0x7f3ec0076780 msgr2=0x7f3ec00fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.786+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3764091332 shutdown_connections 2024-03-26T06:41:02.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.786+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3764091332 wait complete. 2024-03-26T06:41:02.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3ec4ccd640 1 Processor -- start 2024-03-26T06:41:02.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3ec4ccd640 1 -- start start 2024-03-26T06:41:02.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3ec4ccd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ec00ff940 0x7f3ec00fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:02.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3ec4ccd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec0100300 0x7f3ec00fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:02.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3ec4ccd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ec0105950 0x7f3ec00fc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3ec4ccd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec010f4e0 con 0x7f3ec0100300 2024-03-26T06:41:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3ec4ccd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec010f360 con 0x7f3ec0105950 2024-03-26T06:41:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3ec4ccd640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec010f660 con 0x7f3ec00ff940 2024-03-26T06:41:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3eb5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec0100300 0x7f3ec00fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3eb5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec0100300 0x7f3ec00fc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46540/0 (socket says 172.21.15.37:46540) 2024-03-26T06:41:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3eb5d9b640 1 -- 172.21.15.37:0/3127364689 learned_addr learned my addr 172.21.15.37:0/3127364689 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3ebed9d640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ec0105950 0x7f3ec00fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.787+0000 7f3ebe59c640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ec00ff940 0x7f3ec00fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.788+0000 7f3eb5d9b640 1 -- 172.21.15.37:0/3127364689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ec00ff940 msgr2=0x7f3ec00fbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.788+0000 7f3eb5d9b640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ec00ff940 0x7f3ec00fbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.788+0000 7f3eb5d9b640 1 -- 172.21.15.37:0/3127364689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ec0105950 msgr2=0x7f3ec00fc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.788+0000 7f3eb5d9b640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ec0105950 0x7f3ec00fc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.788+0000 7f3eb5d9b640 1 -- 172.21.15.37:0/3127364689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec01a5fb0 con 0x7f3ec0100300 2024-03-26T06:41:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.788+0000 7f3eb5d9b640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec0100300 0x7f3ec00fc100 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800d950 tx=0x7f3ea800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.788+0000 7f3eb7fff640 1 -- 172.21.15.37:0/3127364689 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8017030 con 0x7f3ec0100300 2024-03-26T06:41:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.788+0000 7f3eb7fff640 1 -- 172.21.15.37:0/3127364689 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ea8004020 con 0x7f3ec0100300 2024-03-26T06:41:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.788+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3127364689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec01a62a0 con 0x7f3ec0100300 2024-03-26T06:41:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.788+0000 7f3eb7fff640 1 -- 172.21.15.37:0/3127364689 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80047a0 con 0x7f3ec0100300 2024-03-26T06:41:02.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.788+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3127364689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec01a67f0 con 0x7f3ec0100300 2024-03-26T06:41:02.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.789+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3127364689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec004f050 con 0x7f3ec0100300 2024-03-26T06:41:02.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.790+0000 7f3eb7fff640 1 -- 172.21.15.37:0/3127364689 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3ea80041d0 con 0x7f3ec0100300 2024-03-26T06:41:02.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.791+0000 7f3eb7fff640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3eac0a2a90 0x7f3eac0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:02.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.791+0000 7f3ebe59c640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3eac0a2a90 0x7f3eac0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:02.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.791+0000 7f3eb7fff640 1 -- 172.21.15.37:0/3127364689 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3ea80ce8b0 con 0x7f3ec0100300 2024-03-26T06:41:02.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.791+0000 7f3ebe59c640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3eac0a2a90 0x7f3eac0a4f50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000ab50 tx=0x7f3eb0005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:02.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:02.796+0000 7f3eb7fff640 1 -- 172.21.15.37:0/3127364689 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3ea809b5b0 con 0x7f3ec0100300 2024-03-26T06:41:03.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.001+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3127364689 --> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3ec0109f30 con 0x7f3eac0a2a90 2024-03-26T06:41:03.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.011+0000 7f3eb7fff640 1 -- 172.21.15.37:0/3127364689 <== mgr.15411 v2:172.21.15.37:6800/3894397572 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f3eb000a880 con 0x7f3eac0a2a90 2024-03-26T06:41:03.014 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-03-26T06:41:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.013+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3127364689 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3eac0a2a90 msgr2=0x7f3eac0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.013+0000 7f3ec4ccd640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3eac0a2a90 0x7f3eac0a4f50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000ab50 tx=0x7f3eb0005f90 comp rx=0 tx=0).stop 2024-03-26T06:41:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.013+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3127364689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec0100300 msgr2=0x7f3ec00fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.013+0000 7f3ec4ccd640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec0100300 0x7f3ec00fc100 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800d950 tx=0x7f3ea800de10 comp rx=0 tx=0).stop 2024-03-26T06:41:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.013+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3127364689 shutdown_connections 2024-03-26T06:41:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.013+0000 7f3ec4ccd640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3eac0a2a90 0x7f3eac0a4f50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.013+0000 7f3ec4ccd640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ec0105950 0x7f3ec00fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.013+0000 7f3ec4ccd640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec0100300 0x7f3ec00fc100 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.013+0000 7f3ec4ccd640 1 --2- 172.21.15.37:0/3127364689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ec00ff940 0x7f3ec00fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.013+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3127364689 >> 172.21.15.37:0/3127364689 conn(0x7f3ec0076780 msgr2=0x7f3ec00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:03.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.014+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3127364689 shutdown_connections 2024-03-26T06:41:03.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.014+0000 7f3ec4ccd640 1 -- 172.21.15.37:0/3127364689 wait complete. 2024-03-26T06:41:03.027 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-26T06:41:03.038 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:41:03.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.139+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1976053857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d941054c0 msgr2=0x7f1d9410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.139+0000 7f1d98bc4640 1 --2- 172.21.15.37:0/1976053857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d941054c0 0x7f1d9410d090 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009900 tx=0x7f1d8c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:41:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.139+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1976053857 shutdown_connections 2024-03-26T06:41:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.139+0000 7f1d98bc4640 1 --2- 172.21.15.37:0/1976053857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d941054c0 0x7f1d9410d090 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.139+0000 7f1d98bc4640 1 --2- 172.21.15.37:0/1976053857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d94100da0 0x7f1d94104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.139+0000 7f1d98bc4640 1 --2- 172.21.15.37:0/1976053857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d941003f0 0x7f1d941007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.139+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1976053857 >> 172.21.15.37:0/1976053857 conn(0x7f1d940772c0 msgr2=0x7f1d940776d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.139+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1976053857 shutdown_connections 2024-03-26T06:41:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.139+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1976053857 wait complete. 2024-03-26T06:41:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d98bc4640 1 Processor -- start 2024-03-26T06:41:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d98bc4640 1 -- start start 2024-03-26T06:41:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d98bc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d941003f0 0x7f1d940fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d98bc4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d941054c0 0x7f1d940fe540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d98bc4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d940fea80 0x7f1d941a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d98bc4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d94110560 con 0x7f1d941003f0 2024-03-26T06:41:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d98bc4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d941103e0 con 0x7f1d941054c0 2024-03-26T06:41:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d98bc4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d941106e0 con 0x7f1d940fea80 2024-03-26T06:41:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d9259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d941003f0 0x7f1d940fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d9259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d941003f0 0x7f1d940fe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46558/0 (socket says 172.21.15.37:46558) 2024-03-26T06:41:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d9259c640 1 -- 172.21.15.37:0/1631209654 learned_addr learned my addr 172.21.15.37:0/1631209654 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d91d9b640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d941054c0 0x7f1d940fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d92d9d640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d940fea80 0x7f1d941a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d9259c640 1 -- 172.21.15.37:0/1631209654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d940fea80 msgr2=0x7f1d941a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d9259c640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d940fea80 0x7f1d941a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d9259c640 1 -- 172.21.15.37:0/1631209654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d941054c0 msgr2=0x7f1d940fe540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d9259c640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d941054c0 0x7f1d940fe540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d9259c640 1 -- 172.21.15.37:0/1631209654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d941a6400 con 0x7f1d941003f0 2024-03-26T06:41:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.140+0000 7f1d92d9d640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d940fea80 0x7f1d941a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:41:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.141+0000 7f1d9259c640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d941003f0 0x7f1d940fe000 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400ebc0 tx=0x7f1d8400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.141+0000 7f1d8b7fe640 1 -- 172.21.15.37:0/1631209654 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84010630 con 0x7f1d941003f0 2024-03-26T06:41:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.141+0000 7f1d8b7fe640 1 -- 172.21.15.37:0/1631209654 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d84010cb0 con 0x7f1d941003f0 2024-03-26T06:41:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.141+0000 7f1d8b7fe640 1 -- 172.21.15.37:0/1631209654 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84018bd0 con 0x7f1d941003f0 2024-03-26T06:41:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.141+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1631209654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d941a66f0 con 0x7f1d941003f0 2024-03-26T06:41:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.141+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1631209654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d941a6c40 con 0x7f1d941003f0 2024-03-26T06:41:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.142+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1631209654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d94068530 con 0x7f1d941003f0 2024-03-26T06:41:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.143+0000 7f1d8b7fe640 1 -- 172.21.15.37:0/1631209654 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1d840107d0 con 0x7f1d941003f0 2024-03-26T06:41:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.143+0000 7f1d8b7fe640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1d800a29a0 0x7f1d800a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.144+0000 7f1d91d9b640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1d800a29a0 0x7f1d800a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:03.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.144+0000 7f1d8b7fe640 1 -- 172.21.15.37:0/1631209654 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1d840cf570 con 0x7f1d941003f0 2024-03-26T06:41:03.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.144+0000 7f1d91d9b640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1d800a29a0 0x7f1d800a4e60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005fd0 tx=0x7f1d7c005850 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:03.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.148+0000 7f1d8b7fe640 1 -- 172.21.15.37:0/1631209654 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1d8409c2d0 con 0x7f1d941003f0 2024-03-26T06:41:03.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.335+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1631209654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d940ff530 con 0x7f1d941003f0 2024-03-26T06:41:03.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.336+0000 7f1d8b7fe640 1 -- 172.21.15.37:0/1631209654 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+23048 (secure 0 0 0) 0x7f1d8409bbd0 con 0x7f1d941003f0 2024-03-26T06:41:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:41:03.340 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":114,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:40:19.822990+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/641545212":"2024-03-27T06:22:58.745093+0000","172.21.15.37:6800/641545212":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/2221744469":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/656518427":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/374271722":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/3212150316":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:41:03.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1631209654 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1d800a29a0 msgr2=0x7f1d800a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1d800a29a0 0x7f1d800a4e60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005fd0 tx=0x7f1d7c005850 comp rx=0 tx=0).stop 2024-03-26T06:41:03.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1631209654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d941003f0 msgr2=0x7f1d940fe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d941003f0 0x7f1d940fe000 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400ebc0 tx=0x7f1d8400c6a0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1631209654 shutdown_connections 2024-03-26T06:41:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f1d800a29a0 0x7f1d800a4e60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d940fea80 0x7f1d941a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d941054c0 0x7f1d940fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 --2- 172.21.15.37:0/1631209654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d941003f0 0x7f1d940fe000 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1631209654 >> 172.21.15.37:0/1631209654 conn(0x7f1d940772c0 msgr2=0x7f1d94069c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1631209654 shutdown_connections 2024-03-26T06:41:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.339+0000 7f1d98bc4640 1 -- 172.21.15.37:0/1631209654 wait complete. 2024-03-26T06:41:03.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:03 smithi037 ceph-mon[31974]: from='client.26456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:41:03.353 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T06:16:17.005303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T06:20:51.878592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T06:20:52.836733+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T06:41:03.354 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T06:41:03.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:03 smithi149 ceph-mon[36681]: from='client.26456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:41:03.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.457+0000 7f31924b2640 1 -- 172.21.15.37:0/239172885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c102e80 msgr2=0x7f318c10b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.457+0000 7f31924b2640 1 --2- 172.21.15.37:0/239172885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c102e80 0x7f318c10b360 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f317c009900 tx=0x7f317c02bcd0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.457+0000 7f31924b2640 1 -- 172.21.15.37:0/239172885 shutdown_connections 2024-03-26T06:41:03.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.457+0000 7f31924b2640 1 --2- 172.21.15.37:0/239172885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f318c10b930 0x7f318c10dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.457+0000 7f31924b2640 1 --2- 172.21.15.37:0/239172885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c102e80 0x7f318c10b360 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.457+0000 7f31924b2640 1 --2- 172.21.15.37:0/239172885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f318c102560 0x7f318c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.457+0000 7f31924b2640 1 -- 172.21.15.37:0/239172885 >> 172.21.15.37:0/239172885 conn(0x7f318c077350 msgr2=0x7f318c077760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:03.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.457+0000 7f31924b2640 1 -- 172.21.15.37:0/239172885 shutdown_connections 2024-03-26T06:41:03.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.457+0000 7f31924b2640 1 -- 172.21.15.37:0/239172885 wait complete. 2024-03-26T06:41:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f31924b2640 1 Processor -- start 2024-03-26T06:41:03.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f31924b2640 1 -- start start 2024-03-26T06:41:03.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f31924b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c102560 0x7f318c19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f31924b2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f318c102e80 0x7f318c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f31924b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f318c10b930 0x7f318c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f31924b2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c1116e0 con 0x7f318c102560 2024-03-26T06:41:03.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f31924b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c111560 con 0x7f318c10b930 2024-03-26T06:41:03.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f31924b2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c111860 con 0x7f318c102e80 2024-03-26T06:41:03.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f318bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c102560 0x7f318c19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:03.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f318bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c102560 0x7f318c19d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46572/0 (socket says 172.21.15.37:46572) 2024-03-26T06:41:03.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f318bfff640 1 -- 172.21.15.37:0/2785519725 learned_addr learned my addr 172.21.15.37:0/2785519725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:03.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f3190a4f640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f318c10b930 0x7f318c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:03.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.458+0000 7f318bfff640 1 -- 172.21.15.37:0/2785519725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f318c102e80 msgr2=0x7f318c19dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.459+0000 7f318bfff640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f318c102e80 0x7f318c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.459+0000 7f318bfff640 1 -- 172.21.15.37:0/2785519725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f318c10b930 msgr2=0x7f318c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.459+0000 7f318bfff640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f318c10b930 0x7f318c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.459+0000 7f318bfff640 1 -- 172.21.15.37:0/2785519725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c198180 con 0x7f318c102560 2024-03-26T06:41:03.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.459+0000 7f318bfff640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c102560 0x7f318c19d850 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f318000e9f0 tx=0x7f318000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:03.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.459+0000 7f31897fa640 1 -- 172.21.15.37:0/2785519725 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318000cd80 con 0x7f318c102560 2024-03-26T06:41:03.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.459+0000 7f31897fa640 1 -- 172.21.15.37:0/2785519725 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3180010910 con 0x7f318c102560 2024-03-26T06:41:03.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.459+0000 7f31924b2640 1 -- 172.21.15.37:0/2785519725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c198470 con 0x7f318c102560 2024-03-26T06:41:03.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.459+0000 7f31897fa640 1 -- 172.21.15.37:0/2785519725 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180018990 con 0x7f318c102560 2024-03-26T06:41:03.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.459+0000 7f31924b2640 1 -- 172.21.15.37:0/2785519725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c1a4710 con 0x7f318c102560 2024-03-26T06:41:03.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.460+0000 7f31924b2640 1 -- 172.21.15.37:0/2785519725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c068530 con 0x7f318c102560 2024-03-26T06:41:03.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.461+0000 7f31897fa640 1 -- 172.21.15.37:0/2785519725 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3180010ac0 con 0x7f318c102560 2024-03-26T06:41:03.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.462+0000 7f31897fa640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31740a2970 0x7f31740a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.462+0000 7f318b7fe640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31740a2970 0x7f31740a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:03.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.462+0000 7f31897fa640 1 -- 172.21.15.37:0/2785519725 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f31800cf250 con 0x7f318c102560 2024-03-26T06:41:03.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.462+0000 7f318b7fe640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31740a2970 0x7f31740a4e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f317c0096c0 tx=0x7f317c02cf90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:03.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.466+0000 7f31897fa640 1 -- 172.21.15.37:0/2785519725 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f318009bfb0 con 0x7f318c102560 2024-03-26T06:41:03.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:03 smithi066 ceph-mon[36504]: from='client.26456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:41:03.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.648+0000 7f31924b2640 1 -- 172.21.15.37:0/2785519725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f318c106a80 con 0x7f318c102560 2024-03-26T06:41:03.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.648+0000 7f31897fa640 1 -- 172.21.15.37:0/2785519725 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f318009b8b0 con 0x7f318c102560 2024-03-26T06:41:03.652 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-26T06:41:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.650+0000 7f31924b2640 1 -- 172.21.15.37:0/2785519725 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31740a2970 msgr2=0x7f31740a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.650+0000 7f31924b2640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31740a2970 0x7f31740a4e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f317c0096c0 tx=0x7f317c02cf90 comp rx=0 tx=0).stop 2024-03-26T06:41:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.651+0000 7f31924b2640 1 -- 172.21.15.37:0/2785519725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c102560 msgr2=0x7f318c19d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.651+0000 7f31924b2640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c102560 0x7f318c19d850 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f318000e9f0 tx=0x7f318000eeb0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.651+0000 7f31924b2640 1 -- 172.21.15.37:0/2785519725 shutdown_connections 2024-03-26T06:41:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.651+0000 7f31924b2640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f31740a2970 0x7f31740a4e30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.651+0000 7f31924b2640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f318c10b930 0x7f318c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.651+0000 7f31924b2640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f318c102e80 0x7f318c19dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.651+0000 7f31924b2640 1 --2- 172.21.15.37:0/2785519725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c102560 0x7f318c19d850 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.651+0000 7f31924b2640 1 -- 172.21.15.37:0/2785519725 >> 172.21.15.37:0/2785519725 conn(0x7f318c077350 msgr2=0x7f318c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:03.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.651+0000 7f31924b2640 1 -- 172.21.15.37:0/2785519725 shutdown_connections 2024-03-26T06:41:03.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.651+0000 7f31924b2640 1 -- 172.21.15.37:0/2785519725 wait complete. 2024-03-26T06:41:03.665 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T06:41:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.769+0000 7f2dbd159640 1 -- 172.21.15.37:0/374886447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db8100450 msgr2=0x7f2db8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.769+0000 7f2dbd159640 1 --2- 172.21.15.37:0/374886447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db8100450 0x7f2db8100830 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009900 tx=0x7f2da802bc30 comp rx=0 tx=0).stop 2024-03-26T06:41:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.769+0000 7f2dbd159640 1 -- 172.21.15.37:0/374886447 shutdown_connections 2024-03-26T06:41:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.769+0000 7f2dbd159640 1 --2- 172.21.15.37:0/374886447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db81054d0 0x7f2db810d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.769+0000 7f2dbd159640 1 --2- 172.21.15.37:0/374886447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db8100e00 0x7f2db8104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.769+0000 7f2dbd159640 1 --2- 172.21.15.37:0/374886447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db8100450 0x7f2db8100830 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.769+0000 7f2dbd159640 1 -- 172.21.15.37:0/374886447 >> 172.21.15.37:0/374886447 conn(0x7f2db8077350 msgr2=0x7f2db8077760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.769+0000 7f2dbd159640 1 -- 172.21.15.37:0/374886447 shutdown_connections 2024-03-26T06:41:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.769+0000 7f2dbd159640 1 -- 172.21.15.37:0/374886447 wait complete. 2024-03-26T06:41:03.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2dbd159640 1 Processor -- start 2024-03-26T06:41:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2dbd159640 1 -- start start 2024-03-26T06:41:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2dbd159640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db8100450 0x7f2db8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2dbd159640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db81054d0 0x7f2db8198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2dbd159640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db8199460 0x7f2db81a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2dbd159640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db81105b0 con 0x7f2db8199460 2024-03-26T06:41:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2dbd159640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db8110430 con 0x7f2db8100450 2024-03-26T06:41:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2dbd159640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db8110730 con 0x7f2db81054d0 2024-03-26T06:41:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2db759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db8199460 0x7f2db81a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2db759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db8199460 0x7f2db81a3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46588/0 (socket says 172.21.15.37:46588) 2024-03-26T06:41:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2db759e640 1 -- 172.21.15.37:0/510900141 learned_addr learned my addr 172.21.15.37:0/510900141 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2db6d9d640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db8100450 0x7f2db8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2daffff640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db81054d0 0x7f2db8198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2db759e640 1 -- 172.21.15.37:0/510900141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db81054d0 msgr2=0x7f2db8198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2db759e640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db81054d0 0x7f2db8198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2db759e640 1 -- 172.21.15.37:0/510900141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db8100450 msgr2=0x7f2db8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2db759e640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db8100450 0x7f2db8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2db759e640 1 -- 172.21.15.37:0/510900141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db81a41e0 con 0x7f2db8199460 2024-03-26T06:41:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.770+0000 7f2db759e640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db8199460 0x7f2db81a3a10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2db000eaa0 tx=0x7f2db000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.771+0000 7f2db4d99640 1 -- 172.21.15.37:0/510900141 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db000cdb0 con 0x7f2db8199460 2024-03-26T06:41:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.771+0000 7f2db4d99640 1 -- 172.21.15.37:0/510900141 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2db0005020 con 0x7f2db8199460 2024-03-26T06:41:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.771+0000 7f2dbd159640 1 -- 172.21.15.37:0/510900141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db81a44d0 con 0x7f2db8199460 2024-03-26T06:41:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.771+0000 7f2db4d99640 1 -- 172.21.15.37:0/510900141 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db00116b0 con 0x7f2db8199460 2024-03-26T06:41:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.771+0000 7f2dbd159640 1 -- 172.21.15.37:0/510900141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db81a49d0 con 0x7f2db8199460 2024-03-26T06:41:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.772+0000 7f2dbd159640 1 -- 172.21.15.37:0/510900141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db8068530 con 0x7f2db8199460 2024-03-26T06:41:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.773+0000 7f2db4d99640 1 -- 172.21.15.37:0/510900141 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2db0005560 con 0x7f2db8199460 2024-03-26T06:41:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.773+0000 7f2db4d99640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2da40a2970 0x7f2da40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.773+0000 7f2db6d9d640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2da40a2970 0x7f2da40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.773+0000 7f2db4d99640 1 -- 172.21.15.37:0/510900141 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2db00cefb0 con 0x7f2db8199460 2024-03-26T06:41:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.774+0000 7f2db6d9d640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2da40a2970 0x7f2da40a4e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2da8000c00 tx=0x7f2da802df90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.778+0000 7f2db4d99640 1 -- 172.21.15.37:0/510900141 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2db0016030 con 0x7f2db8199460 2024-03-26T06:41:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.959+0000 7f2dbd159640 1 -- 172.21.15.37:0/510900141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2db8100e00 con 0x7f2db8199460 2024-03-26T06:41:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.960+0000 7f2db4d99640 1 -- 172.21.15.37:0/510900141 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2db009b6c0 con 0x7f2db8199460 2024-03-26T06:41:03.963 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-26T06:41:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 -- 172.21.15.37:0/510900141 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2da40a2970 msgr2=0x7f2da40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2da40a2970 0x7f2da40a4e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2da8000c00 tx=0x7f2da802df90 comp rx=0 tx=0).stop 2024-03-26T06:41:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 -- 172.21.15.37:0/510900141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db8199460 msgr2=0x7f2db81a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db8199460 0x7f2db81a3a10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2db000eaa0 tx=0x7f2db000ef60 comp rx=0 tx=0).stop 2024-03-26T06:41:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 -- 172.21.15.37:0/510900141 shutdown_connections 2024-03-26T06:41:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2da40a2970 0x7f2da40a4e30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2db8199460 0x7f2db81a3a10 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db81054d0 0x7f2db8198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 --2- 172.21.15.37:0/510900141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db8100450 0x7f2db8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 -- 172.21.15.37:0/510900141 >> 172.21.15.37:0/510900141 conn(0x7f2db8077350 msgr2=0x7f2db80fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 -- 172.21.15.37:0/510900141 shutdown_connections 2024-03-26T06:41:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:03.962+0000 7f2dbd159640 1 -- 172.21.15.37:0/510900141 wait complete. 2024-03-26T06:41:03.976 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T06:41:04.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.079+0000 7fa124275640 1 -- 172.21.15.37:0/1925757995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa11c1054e0 msgr2=0x7fa11c10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:04.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.079+0000 7fa124275640 1 --2- 172.21.15.37:0/1925757995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa11c1054e0 0x7fa11c10d0b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa114009ca0 tx=0x7fa11402c0f0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.079+0000 7fa124275640 1 -- 172.21.15.37:0/1925757995 shutdown_connections 2024-03-26T06:41:04.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.079+0000 7fa124275640 1 --2- 172.21.15.37:0/1925757995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa11c1054e0 0x7fa11c10d0b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.079+0000 7fa124275640 1 --2- 172.21.15.37:0/1925757995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa11c100dc0 0x7fa11c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.079+0000 7fa124275640 1 --2- 172.21.15.37:0/1925757995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c100410 0x7fa11c1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.079+0000 7fa124275640 1 -- 172.21.15.37:0/1925757995 >> 172.21.15.37:0/1925757995 conn(0x7fa11c076570 msgr2=0x7fa11c076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:04.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.079+0000 7fa124275640 1 -- 172.21.15.37:0/1925757995 shutdown_connections 2024-03-26T06:41:04.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.079+0000 7fa124275640 1 -- 172.21.15.37:0/1925757995 wait complete. 2024-03-26T06:41:04.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa124275640 1 Processor -- start 2024-03-26T06:41:04.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa124275640 1 -- start start 2024-03-26T06:41:04.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa124275640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa11c100410 0x7fa11c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:04.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa124275640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa11c1054e0 0x7fa11c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:04.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa124275640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c19e610 0x7fa11c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:04.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa124275640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1105a0 con 0x7fa11c100410 2024-03-26T06:41:04.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa124275640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c110420 con 0x7fa11c1054e0 2024-03-26T06:41:04.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa124275640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c110720 con 0x7fa11c19e610 2024-03-26T06:41:04.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa122011640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa11c100410 0x7fa11c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:04.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa122011640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa11c100410 0x7fa11c199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46614/0 (socket says 172.21.15.37:46614) 2024-03-26T06:41:04.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa122011640 1 -- 172.21.15.37:0/1606326912 learned_addr learned my addr 172.21.15.37:0/1606326912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:04.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa121810640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa11c1054e0 0x7fa11c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:04.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.080+0000 7fa122812640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c19e610 0x7fa11c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:04.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa122011640 1 -- 172.21.15.37:0/1606326912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c19e610 msgr2=0x7fa11c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:04.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa122011640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c19e610 0x7fa11c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa122011640 1 -- 172.21.15.37:0/1606326912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa11c1054e0 msgr2=0x7fa11c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:04.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa122011640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa11c1054e0 0x7fa11c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa122011640 1 -- 172.21.15.37:0/1606326912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c19f100 con 0x7fa11c100410 2024-03-26T06:41:04.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa122812640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c19e610 0x7fa11c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:41:04.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa122011640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa11c100410 0x7fa11c199d50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa11800d770 tx=0x7fa11800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:04.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa1137fe640 1 -- 172.21.15.37:0/1606326912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118017070 con 0x7fa11c100410 2024-03-26T06:41:04.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa124275640 1 -- 172.21.15.37:0/1606326912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c19f3f0 con 0x7fa11c100410 2024-03-26T06:41:04.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa1137fe640 1 -- 172.21.15.37:0/1606326912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa118005500 con 0x7fa11c100410 2024-03-26T06:41:04.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa1137fe640 1 -- 172.21.15.37:0/1606326912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118004e00 con 0x7fa11c100410 2024-03-26T06:41:04.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.081+0000 7fa124275640 1 -- 172.21.15.37:0/1606326912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c1a6b90 con 0x7fa11c100410 2024-03-26T06:41:04.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.082+0000 7fa124275640 1 -- 172.21.15.37:0/1606326912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c1018e0 con 0x7fa11c100410 2024-03-26T06:41:04.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.083+0000 7fa1137fe640 1 -- 172.21.15.37:0/1606326912 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fa118004fa0 con 0x7fa11c100410 2024-03-26T06:41:04.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.083+0000 7fa1137fe640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa1080a2970 0x7fa1080a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:04.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.084+0000 7fa1137fe640 1 -- 172.21.15.37:0/1606326912 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa1180ce8b0 con 0x7fa11c100410 2024-03-26T06:41:04.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.084+0000 7fa121810640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa1080a2970 0x7fa1080a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:04.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.084+0000 7fa121810640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa1080a2970 0x7fa1080a4e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0ffd20 tx=0x7fa10c00b410 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.088+0000 7fa1137fe640 1 -- 172.21.15.37:0/1606326912 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa11809b610 con 0x7fa11c100410 2024-03-26T06:41:04.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.272+0000 7fa124275640 1 -- 172.21.15.37:0/1606326912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa11c100dc0 con 0x7fa11c100410 2024-03-26T06:41:04.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.272+0000 7fa1137fe640 1 -- 172.21.15.37:0/1606326912 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa11809af10 con 0x7fa11c100410 2024-03-26T06:41:04.276 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-26T06:41:04.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.274+0000 7fa124275640 1 -- 172.21.15.37:0/1606326912 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa1080a2970 msgr2=0x7fa1080a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:04.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.274+0000 7fa124275640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa1080a2970 0x7fa1080a4e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0ffd20 tx=0x7fa10c00b410 comp rx=0 tx=0).stop 2024-03-26T06:41:04.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.274+0000 7fa124275640 1 -- 172.21.15.37:0/1606326912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa11c100410 msgr2=0x7fa11c199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:04.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.274+0000 7fa124275640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa11c100410 0x7fa11c199d50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa11800d770 tx=0x7fa11800dc30 comp rx=0 tx=0).stop 2024-03-26T06:41:04.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.274+0000 7fa124275640 1 -- 172.21.15.37:0/1606326912 shutdown_connections 2024-03-26T06:41:04.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.274+0000 7fa124275640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fa1080a2970 0x7fa1080a4e30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.274+0000 7fa124275640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c19e610 0x7fa11c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.274+0000 7fa124275640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa11c1054e0 0x7fa11c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.274+0000 7fa124275640 1 --2- 172.21.15.37:0/1606326912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa11c100410 0x7fa11c199d50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.274+0000 7fa124275640 1 -- 172.21.15.37:0/1606326912 >> 172.21.15.37:0/1606326912 conn(0x7fa11c076570 msgr2=0x7fa11c10c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:04.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.275+0000 7fa124275640 1 -- 172.21.15.37:0/1606326912 shutdown_connections 2024-03-26T06:41:04.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.275+0000 7fa124275640 1 -- 172.21.15.37:0/1606326912 wait complete. 2024-03-26T06:41:04.287 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:04 smithi037 ceph-mon[31974]: from='client.16959 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:41:04.287 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:04 smithi037 ceph-mon[31974]: pgmap v571: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-03-26T06:41:04.288 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:04 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1631209654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:41:04.288 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:04 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2785519725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:41:04.288 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:04 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/510900141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:41:04.289 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-26T06:41:04.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.389+0000 7f2f04f97640 1 -- 172.21.15.37:0/1791066561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f00101ed0 msgr2=0x7f2f00105f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:04.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.389+0000 7f2f04f97640 1 --2- 172.21.15.37:0/1791066561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f00101ed0 0x7f2f00105f10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009900 tx=0x7f2ee802bc30 comp rx=0 tx=0).stop 2024-03-26T06:41:04.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.389+0000 7f2f04f97640 1 -- 172.21.15.37:0/1791066561 shutdown_connections 2024-03-26T06:41:04.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.389+0000 7f2f04f97640 1 --2- 172.21.15.37:0/1791066561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f001065f0 0x7f2f0010e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.389+0000 7f2f04f97640 1 --2- 172.21.15.37:0/1791066561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f00101ed0 0x7f2f00105f10 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.389+0000 7f2f04f97640 1 --2- 172.21.15.37:0/1791066561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f00101520 0x7f2f00101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.389+0000 7f2f04f97640 1 -- 172.21.15.37:0/1791066561 >> 172.21.15.37:0/1791066561 conn(0x7f2f000fcd60 msgr2=0x7f2f000ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:04.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.389+0000 7f2f04f97640 1 -- 172.21.15.37:0/1791066561 shutdown_connections 2024-03-26T06:41:04.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.389+0000 7f2f04f97640 1 -- 172.21.15.37:0/1791066561 wait complete. 2024-03-26T06:41:04.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2f04f97640 1 Processor -- start 2024-03-26T06:41:04.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2f04f97640 1 -- start start 2024-03-26T06:41:04.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2f04f97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f00101520 0x7f2f0019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:04.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2f04f97640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f001065f0 0x7f2f0019dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:04.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2f04f97640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f001979a0 0x7f2f00197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:04.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2f04f97640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f00111680 con 0x7f2f00101520 2024-03-26T06:41:04.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2f04f97640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f00111500 con 0x7f2f001065f0 2024-03-26T06:41:04.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2f04f97640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f00111800 con 0x7f2f001979a0 2024-03-26T06:41:04.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2efe59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f00101520 0x7f2f0019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:04.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2efe59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f00101520 0x7f2f0019d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46620/0 (socket says 172.21.15.37:46620) 2024-03-26T06:41:04.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2efe59c640 1 -- 172.21.15.37:0/1209195006 learned_addr learned my addr 172.21.15.37:0/1209195006 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:04.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2efed9d640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f001979a0 0x7f2f00197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:04.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.390+0000 7f2ef7fff640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f001065f0 0x7f2f0019dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:04.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.391+0000 7f2efe59c640 1 -- 172.21.15.37:0/1209195006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f001979a0 msgr2=0x7f2f00197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:04.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.391+0000 7f2efe59c640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f001979a0 0x7f2f00197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.391+0000 7f2efe59c640 1 -- 172.21.15.37:0/1209195006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f001065f0 msgr2=0x7f2f0019dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:04.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.391+0000 7f2efe59c640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f001065f0 0x7f2f0019dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:04.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.391+0000 7f2efe59c640 1 -- 172.21.15.37:0/1209195006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f001984c0 con 0x7f2f00101520 2024-03-26T06:41:04.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.391+0000 7f2efe59c640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f00101520 0x7f2f0019d7f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000d770 tx=0x7f2ef000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:04.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.391+0000 7f2ef77fe640 1 -- 172.21.15.37:0/1209195006 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0017070 con 0x7f2f00101520 2024-03-26T06:41:04.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.391+0000 7f2ef77fe640 1 -- 172.21.15.37:0/1209195006 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ef0005500 con 0x7f2f00101520 2024-03-26T06:41:04.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.391+0000 7f2f04f97640 1 -- 172.21.15.37:0/1209195006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f001987b0 con 0x7f2f00101520 2024-03-26T06:41:04.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.391+0000 7f2ef77fe640 1 -- 172.21.15.37:0/1209195006 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0004e00 con 0x7f2f00101520 2024-03-26T06:41:04.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.391+0000 7f2f04f97640 1 -- 172.21.15.37:0/1209195006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f001a49b0 con 0x7f2f00101520 2024-03-26T06:41:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.392+0000 7f2f04f97640 1 -- 172.21.15.37:0/1209195006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f00068530 con 0x7f2f00101520 2024-03-26T06:41:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.393+0000 7f2ef77fe640 1 -- 172.21.15.37:0/1209195006 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f2ef0004fa0 con 0x7f2f00101520 2024-03-26T06:41:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.394+0000 7f2ef77fe640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2eec0a2970 0x7f2eec0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.394+0000 7f2ef7fff640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2eec0a2970 0x7f2eec0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.394+0000 7f2ef77fe640 1 -- 172.21.15.37:0/1209195006 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2ef00ceda0 con 0x7f2f00101520 2024-03-26T06:41:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.394+0000 7f2ef7fff640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2eec0a2970 0x7f2eec0a4e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8006010 tx=0x7f2ee8000bc0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:04.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.397+0000 7f2ef77fe640 1 -- 172.21.15.37:0/1209195006 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2ef009bb00 con 0x7f2f00101520 2024-03-26T06:41:04.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:04 smithi149 ceph-mon[36681]: from='client.16959 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:41:04.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:04 smithi149 ceph-mon[36681]: pgmap v571: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-03-26T06:41:04.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:04 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1631209654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:41:04.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:04 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2785519725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:41:04.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:04 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/510900141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:41:04.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:04 smithi066 ceph-mon[36504]: from='client.16959 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T06:41:04.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:04 smithi066 ceph-mon[36504]: pgmap v571: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-03-26T06:41:04.600 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:04 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1631209654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:41:04.600 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:04 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2785519725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:41:04.600 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:04 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/510900141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:41:04.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.628+0000 7f2f04f97640 1 -- 172.21.15.37:0/1209195006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f2f00198ed0 con 0x7f2f00101520 2024-03-26T06:41:04.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.637+0000 7f2ef77fe640 1 -- 172.21.15.37:0/1209195006 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+740362 (secure 0 0 0) 0x7f2f00198ed0 con 0x7f2f00101520 2024-03-26T06:41:04.649 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:41:04.650 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:41:04.650 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-03-26T06:41:04.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.650 INFO:teuthology.orchestra.run.smithi037.stdout: "active_gid": 15411, 2024-03-26T06:41:04.650 INFO:teuthology.orchestra.run.smithi037.stdout: "active_name": "x", 2024-03-26T06:41:04.650 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addrs": { 2024-03-26T06:41:04.650 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-26T06:41:04.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-26T06:41:04.651 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6800", 2024-03-26T06:41:04.651 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3894397572 2024-03-26T06:41:04.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-03-26T06:41:04.651 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6801", 2024-03-26T06:41:04.651 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3894397572 2024-03-26T06:41:04.652 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.652 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:41:04.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.652 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addr": "172.21.15.37:6801/3894397572", 2024-03-26T06:41:04.652 INFO:teuthology.orchestra.run.smithi037.stdout: "active_change": "2024-03-26T06:22:58.745299+0000", 2024-03-26T06:41:04.652 INFO:teuthology.orchestra.run.smithi037.stdout: "active_mgr_features": 4540701547738038271, 2024-03-26T06:41:04.652 INFO:teuthology.orchestra.run.smithi037.stdout: "available": true, 2024-03-26T06:41:04.652 INFO:teuthology.orchestra.run.smithi037.stdout: "standbys": [ 2024-03-26T06:41:04.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.653 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24889, 2024-03-26T06:41:04.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "z", 2024-03-26T06:41:04.653 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-03-26T06:41:04.653 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-03-26T06:41:04.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-03-26T06:41:04.653 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:04.653 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:04.653 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:04.654 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:04.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:04.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.654 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:04.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T06:41:04.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.655 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.655 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:04.655 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:04.655 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.656 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.656 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.656 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.656 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.656 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.656 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.656 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.656 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.657 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.657 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.657 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.658 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:04.658 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:04.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.658 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.659 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.659 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.659 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.659 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.659 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:04.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:04.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.660 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.660 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.660 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:04.660 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.660 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.660 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.660 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.660 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.662 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:04.662 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:04.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.662 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.663 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.663 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-03-26T06:41:04.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-03-26T06:41:04.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.664 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-03-26T06:41:04.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.665 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-03-26T06:41:04.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-03-26T06:41:04.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.665 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-03-26T06:41:04.665 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.666 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-03-26T06:41:04.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.666 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-03-26T06:41:04.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-03-26T06:41:04.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.667 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.667 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-03-26T06:41:04.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.668 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-03-26T06:41:04.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-03-26T06:41:04.668 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.668 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.668 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.668 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.669 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.669 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-03-26T06:41:04.669 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.669 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.669 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.669 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-03-26T06:41:04.670 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-03-26T06:41:04.670 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.670 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.670 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-03-26T06:41:04.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-03-26T06:41:04.671 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.671 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-03-26T06:41:04.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-03-26T06:41:04.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-03-26T06:41:04.672 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.672 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.673 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-03-26T06:41:04.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-03-26T06:41:04.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T06:41:04.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.674 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.675 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-03-26T06:41:04.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-03-26T06:41:04.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.675 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.675 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.675 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.675 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-03-26T06:41:04.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.676 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.676 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-03-26T06:41:04.677 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:04.677 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:04.677 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:04.677 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-03-26T06:41:04.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-03-26T06:41:04.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.678 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.678 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.678 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T06:41:04.678 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.678 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.679 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-03-26T06:41:04.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-03-26T06:41:04.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.679 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.679 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.679 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-03-26T06:41:04.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T06:41:04.680 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T06:41:04.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.680 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-03-26T06:41:04.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-03-26T06:41:04.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:04.681 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.681 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:04.681 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-26T06:41:04.681 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-26T06:41:04.681 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T06:41:04.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T06:41:04.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.682 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-03-26T06:41:04.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-03-26T06:41:04.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:04.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-03-26T06:41:04.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:04.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-03-26T06:41:04.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T06:41:04.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.684 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-03-26T06:41:04.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-03-26T06:41:04.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T06:41:04.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T06:41:04.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T06:41:04.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.685 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-03-26T06:41:04.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-03-26T06:41:04.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:04.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-03-26T06:41:04.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-03-26T06:41:04.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-03-26T06:41:04.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-03-26T06:41:04.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T06:41:04.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.687 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-03-26T06:41:04.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-03-26T06:41:04.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-03-26T06:41:04.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T06:41:04.688 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T06:41:04.688 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.688 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.688 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-03-26T06:41:04.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-03-26T06:41:04.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:04.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.689 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:04.689 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-26T06:41:04.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-26T06:41:04.689 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.690 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T06:41:04.690 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T06:41:04.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.690 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:04.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:04.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.690 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.691 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.691 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.691 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.691 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.691 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.692 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.692 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.692 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.692 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.692 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.692 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:04.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:04.693 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.693 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.693 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.693 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.693 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.694 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.694 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.694 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.694 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.694 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:04.694 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:04.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.695 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.695 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.695 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:04.695 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.695 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.695 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.695 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.695 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.695 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.696 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.696 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.696 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.696 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.697 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:04.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:04.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.697 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.698 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-03-26T06:41:04.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-03-26T06:41:04.698 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:04.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:04.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.699 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T06:41:04.699 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.700 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-26T06:41:04.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-26T06:41:04.700 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.700 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-03-26T06:41:04.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.701 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.701 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-03-26T06:41:04.701 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-03-26T06:41:04.701 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-03-26T06:41:04.701 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-03-26T06:41:04.701 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-03-26T06:41:04.702 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-03-26T06:41:04.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.702 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-03-26T06:41:04.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-03-26T06:41:04.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.703 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.703 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T06:41:04.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.704 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.704 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:04.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:04.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:04.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.705 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T06:41:04.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.705 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.705 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.706 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-03-26T06:41:04.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-03-26T06:41:04.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.706 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:04.706 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:04.706 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.706 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.707 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T06:41:04.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T06:41:04.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.707 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.707 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-03-26T06:41:04.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-03-26T06:41:04.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:04.707 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.708 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:04.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.708 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T06:41:04.708 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.709 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.709 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.709 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-03-26T06:41:04.709 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:04.709 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:04.709 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:04.710 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-03-26T06:41:04.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-03-26T06:41:04.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:04.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.710 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-26T06:41:04.710 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.710 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.710 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T06:41:04.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.711 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-03-26T06:41:04.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-03-26T06:41:04.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.711 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.711 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-03-26T06:41:04.712 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.712 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.712 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.712 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T06:41:04.712 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.712 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.713 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-03-26T06:41:04.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-03-26T06:41:04.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.713 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-03-26T06:41:04.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.713 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.713 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.714 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T06:41:04.714 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.714 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-03-26T06:41:04.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-03-26T06:41:04.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.715 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T06:41:04.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T06:41:04.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.716 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.716 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-03-26T06:41:04.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-03-26T06:41:04.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.716 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.717 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:04.717 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.717 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.717 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.717 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T06:41:04.717 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.717 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.718 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-03-26T06:41:04.718 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T06:41:04.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:04.718 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.718 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-03-26T06:41:04.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.719 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T06:41:04.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.719 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.719 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-03-26T06:41:04.719 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-03-26T06:41:04.719 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.720 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.720 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.720 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-03-26T06:41:04.720 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-03-26T06:41:04.720 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-03-26T06:41:04.721 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T06:41:04.721 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.721 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-03-26T06:41:04.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-03-26T06:41:04.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.722 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.722 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.722 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T06:41:04.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.723 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.723 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-03-26T06:41:04.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-03-26T06:41:04.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.723 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.723 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.724 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T06:41:04.724 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.724 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-03-26T06:41:04.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-03-26T06:41:04.725 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.725 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.726 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T06:41:04.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.726 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-03-26T06:41:04.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-03-26T06:41:04.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.726 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T06:41:04.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.727 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:04.727 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.727 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.727 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.728 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-03-26T06:41:04.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-03-26T06:41:04.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.728 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.728 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.728 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T06:41:04.728 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.728 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.728 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-03-26T06:41:04.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.729 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.729 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-03-26T06:41:04.729 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-03-26T06:41:04.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.730 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T06:41:04.730 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.730 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.730 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.730 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-03-26T06:41:04.730 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.731 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-03-26T06:41:04.731 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-03-26T06:41:04.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T06:41:04.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.732 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.732 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:04.732 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.732 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.732 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-03-26T06:41:04.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-03-26T06:41:04.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.733 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.733 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.733 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T06:41:04.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-03-26T06:41:04.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.734 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-03-26T06:41:04.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-03-26T06:41:04.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T06:41:04.735 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.735 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-03-26T06:41:04.735 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.735 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.735 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-03-26T06:41:04.736 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-03-26T06:41:04.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.736 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T06:41:04.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.736 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-03-26T06:41:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-03-26T06:41:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-03-26T06:41:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.738 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T06:41:04.738 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.738 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.738 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.738 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-03-26T06:41:04.738 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.739 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-03-26T06:41:04.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-03-26T06:41:04.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.739 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T06:41:04.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.739 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-03-26T06:41:04.740 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.740 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.740 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-03-26T06:41:04.740 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-03-26T06:41:04.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.741 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T06:41:04.741 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.741 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.741 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.741 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-03-26T06:41:04.741 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.741 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.741 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.742 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-03-26T06:41:04.742 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-03-26T06:41:04.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.742 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.742 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T06:41:04.742 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.742 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.743 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:04.743 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.743 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-03-26T06:41:04.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-03-26T06:41:04.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T06:41:04.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.744 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-03-26T06:41:04.744 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.744 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.744 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-03-26T06:41:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-03-26T06:41:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T06:41:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.746 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.746 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.746 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:04.746 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.746 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.746 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.746 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-03-26T06:41:04.747 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-03-26T06:41:04.747 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.747 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.747 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T06:41:04.747 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.747 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-03-26T06:41:04.748 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.748 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.748 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-03-26T06:41:04.748 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-03-26T06:41:04.748 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.748 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.749 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T06:41:04.749 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.749 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.749 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-03-26T06:41:04.749 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.749 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.750 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-03-26T06:41:04.750 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-03-26T06:41:04.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.750 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.750 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.751 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.751 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T06:41:04.751 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.751 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-03-26T06:41:04.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-03-26T06:41:04.751 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:04.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.752 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T06:41:04.752 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-03-26T06:41:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T06:41:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-03-26T06:41:04.753 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.754 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.754 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.754 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T06:41:04.754 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.754 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.754 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-03-26T06:41:04.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-03-26T06:41:04.755 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.755 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.755 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.755 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-03-26T06:41:04.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.755 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.755 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T06:41:04.756 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.756 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.756 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.756 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-03-26T06:41:04.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-03-26T06:41:04.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.757 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-03-26T06:41:04.757 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-03-26T06:41:04.757 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.757 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.757 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-03-26T06:41:04.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-03-26T06:41:04.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.758 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.758 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.758 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.758 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.758 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.758 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T06:41:04.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.759 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.759 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.759 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-03-26T06:41:04.759 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-03-26T06:41:04.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.760 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:04.760 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.760 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.760 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.760 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-03-26T06:41:04.760 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.760 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.760 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.761 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-03-26T06:41:04.761 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-03-26T06:41:04.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.761 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.761 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:04.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.762 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-03-26T06:41:04.762 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.762 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-03-26T06:41:04.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-03-26T06:41:04.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.763 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.763 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T06:41:04.763 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.763 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.764 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-03-26T06:41:04.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-03-26T06:41:04.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.764 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.764 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.764 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.764 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.764 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.764 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T06:41:04.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.765 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.765 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.765 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:04.765 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:04.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.766 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.766 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.766 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.766 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.766 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.766 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.766 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.767 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.767 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.767 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.767 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.767 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.767 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.767 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.767 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.768 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-03-26T06:41:04.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-03-26T06:41:04.768 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.768 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.768 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.768 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.768 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.769 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T06:41:04.769 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.769 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.769 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.769 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:04.769 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:04.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.770 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.770 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.770 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.770 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.770 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.770 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.770 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T06:41:04.770 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.771 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.771 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:04.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:04.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.771 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.771 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.771 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:04.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.772 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.772 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.772 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.772 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.772 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.772 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.773 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.773 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.773 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:04.773 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:04.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.774 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.774 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.774 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.774 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.774 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.774 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.774 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.774 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.775 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T06:41:04.775 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T06:41:04.775 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.775 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.775 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.776 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T06:41:04.776 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.776 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.776 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.777 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T06:41:04.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T06:41:04.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-03-26T06:41:04.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.778 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.778 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T06:41:04.778 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.778 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.778 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-03-26T06:41:04.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-03-26T06:41:04.778 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.778 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:04.779 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.779 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.779 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-03-26T06:41:04.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.780 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-03-26T06:41:04.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-03-26T06:41:04.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.780 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.780 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:04.780 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.780 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T06:41:04.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.781 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-03-26T06:41:04.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-03-26T06:41:04.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.782 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.782 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.782 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.782 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-03-26T06:41:04.782 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.782 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.783 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-26T06:41:04.783 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-26T06:41:04.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.783 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.783 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-03-26T06:41:04.783 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.784 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.784 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-03-26T06:41:04.784 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-03-26T06:41:04.784 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T06:41:04.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.784 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.785 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-03-26T06:41:04.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-03-26T06:41:04.785 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.785 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.785 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-03-26T06:41:04.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.786 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T06:41:04.786 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.786 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.786 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-03-26T06:41:04.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-03-26T06:41:04.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.787 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T06:41:04.787 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.787 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.787 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T06:41:04.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.788 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-03-26T06:41:04.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-03-26T06:41:04.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.789 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T06:41:04.789 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.789 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.789 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-03-26T06:41:04.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-03-26T06:41:04.790 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.790 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.790 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.790 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.790 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.790 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.790 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.790 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T06:41:04.791 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.791 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.791 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.791 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-03-26T06:41:04.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-03-26T06:41:04.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.791 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.792 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.792 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.792 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.792 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T06:41:04.792 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.792 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.793 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-03-26T06:41:04.793 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-03-26T06:41:04.793 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.793 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.793 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.793 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.793 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.793 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.793 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.794 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T06:41:04.794 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.794 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.794 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.794 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-03-26T06:41:04.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-03-26T06:41:04.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.795 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.795 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.795 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.795 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.795 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T06:41:04.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.795 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.796 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-03-26T06:41:04.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-03-26T06:41:04.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-03-26T06:41:04.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.797 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.797 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-03-26T06:41:04.797 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.797 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-03-26T06:41:04.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-03-26T06:41:04.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.798 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.798 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.798 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.798 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.798 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.798 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T06:41:04.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.799 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.799 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-03-26T06:41:04.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-03-26T06:41:04.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.799 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.800 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.800 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.800 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T06:41:04.800 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.800 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.800 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-03-26T06:41:04.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-03-26T06:41:04.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.801 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.801 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.801 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.801 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T06:41:04.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.802 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.802 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-03-26T06:41:04.802 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-03-26T06:41:04.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.802 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.803 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.803 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.803 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.803 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.803 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T06:41:04.803 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.804 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.804 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-03-26T06:41:04.804 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-03-26T06:41:04.804 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.804 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.804 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.804 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.805 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T06:41:04.805 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.805 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.806 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-03-26T06:41:04.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-03-26T06:41:04.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.806 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.806 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.806 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.806 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T06:41:04.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.807 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.807 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-03-26T06:41:04.808 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:04.808 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:04.808 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:04.808 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:04.808 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:04.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.808 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.809 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.809 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.809 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.809 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.809 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.809 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.809 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.809 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.809 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.809 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.810 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:04.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:04.810 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.811 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.811 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.811 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.812 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:04.812 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:04.812 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.812 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.812 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.812 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:04.812 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.812 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.813 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.813 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.813 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.813 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.813 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.813 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.813 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.813 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.813 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.814 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.814 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.814 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:04.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:04.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.815 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.815 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.815 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.816 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-03-26T06:41:04.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-03-26T06:41:04.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.816 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-03-26T06:41:04.816 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.816 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T06:41:04.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.817 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-03-26T06:41:04.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-03-26T06:41:04.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-03-26T06:41:04.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.818 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.818 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T06:41:04.818 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.818 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.819 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.819 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-03-26T06:41:04.819 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:04.819 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:04.819 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:04.819 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T06:41:04.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T06:41:04.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.820 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:04.820 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.820 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.820 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.820 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.821 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.821 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T06:41:04.821 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T06:41:04.821 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.821 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.821 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.822 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.822 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.822 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.822 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.822 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.823 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T06:41:04.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T06:41:04.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.823 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.823 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.823 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.823 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.823 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.824 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.824 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.824 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.824 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-03-26T06:41:04.824 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T06:41:04.824 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.824 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.824 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.825 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.825 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.825 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.825 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.825 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.826 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T06:41:04.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T06:41:04.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.826 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.826 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.827 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.827 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T06:41:04.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T06:41:04.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.828 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.829 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.829 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T06:41:04.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T06:41:04.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.830 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.830 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.830 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.830 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T06:41:04.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T06:41:04.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.832 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.832 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T06:41:04.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T06:41:04.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.832 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.832 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.833 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.833 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.834 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T06:41:04.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T06:41:04.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.834 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.834 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.834 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.835 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T06:41:04.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T06:41:04.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.835 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.836 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.837 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T06:41:04.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T06:41:04.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.837 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.838 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.838 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.838 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.838 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.838 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T06:41:04.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T06:41:04.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.839 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.839 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.839 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.839 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.840 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.840 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T06:41:04.840 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T06:41:04.840 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.840 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.840 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.841 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.841 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.841 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.841 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T06:41:04.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T06:41:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-26T06:41:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.843 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.843 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.843 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.843 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T06:41:04.843 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T06:41:04.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.844 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.844 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.844 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.844 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.844 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.844 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.844 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-03-26T06:41:04.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-03-26T06:41:04.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.845 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.845 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.845 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.845 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.846 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.846 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.846 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.846 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T06:41:04.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T06:41:04.846 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.847 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-26T06:41:04.847 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.847 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.847 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.847 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.847 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.848 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T06:41:04.848 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T06:41:04.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.848 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.848 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.848 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.849 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.849 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.849 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.849 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.849 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.849 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.849 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T06:41:04.849 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T06:41:04.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.850 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.850 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.850 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.850 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.850 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.850 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T06:41:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T06:41:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T06:41:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T06:41:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.854 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.854 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.854 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T06:41:04.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T06:41:04.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.855 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-03-26T06:41:04.855 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.856 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T06:41:04.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T06:41:04.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.856 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.857 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.857 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.857 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.857 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.857 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T06:41:04.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T06:41:04.857 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.857 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.858 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.858 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.859 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.859 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T06:41:04.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T06:41:04.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.860 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.860 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.860 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T06:41:04.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T06:41:04.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.861 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.861 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.861 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.861 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.862 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T06:41:04.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T06:41:04.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.862 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.862 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.863 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.863 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T06:41:04.863 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T06:41:04.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.864 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.864 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.864 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.865 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T06:41:04.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T06:41:04.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.865 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.866 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.867 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T06:41:04.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T06:41:04.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.867 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.867 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.868 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T06:41:04.868 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T06:41:04.868 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.868 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.869 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.869 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.869 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.869 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.870 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T06:41:04.870 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T06:41:04.870 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.870 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.870 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.870 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.870 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.870 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.870 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.871 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.871 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.871 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.871 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T06:41:04.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T06:41:04.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.871 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T06:41:04.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.872 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.872 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.873 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T06:41:04.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T06:41:04.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.873 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:04.873 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.873 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.874 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.874 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.874 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.874 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.874 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T06:41:04.874 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T06:41:04.874 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.875 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.875 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.875 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-03-26T06:41:04.875 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.875 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.875 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.876 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.876 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.876 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T06:41:04.876 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T06:41:04.876 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-03-26T06:41:04.877 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.877 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.877 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.877 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.877 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.877 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.877 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.877 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T06:41:04.878 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T06:41:04.878 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.878 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.878 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.878 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.879 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.879 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.879 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.879 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T06:41:04.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T06:41:04.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.880 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-26T06:41:04.880 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.880 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.880 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.880 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.880 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.880 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T06:41:04.881 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T06:41:04.881 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.881 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.881 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.882 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.882 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.882 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.882 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T06:41:04.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T06:41:04.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.883 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.883 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.883 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.883 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.883 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.883 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.883 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.884 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T06:41:04.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T06:41:04.884 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.884 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.884 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.884 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.884 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.884 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.885 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.885 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.885 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T06:41:04.885 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T06:41:04.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.885 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.887 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.887 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T06:41:04.887 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T06:41:04.887 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.887 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.887 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.887 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:04.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.888 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.888 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T06:41:04.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T06:41:04.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:04.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.889 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.889 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.889 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.890 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.890 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-03-26T06:41:04.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-03-26T06:41:04.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.890 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.890 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.890 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.891 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.891 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-03-26T06:41:04.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-03-26T06:41:04.892 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.892 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.892 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.892 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.892 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.893 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.893 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-03-26T06:41:04.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-03-26T06:41:04.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.893 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.894 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.894 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.894 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-03-26T06:41:04.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.895 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-03-26T06:41:04.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-03-26T06:41:04.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.895 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-03-26T06:41:04.895 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.895 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.896 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.896 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.896 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.896 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-26T06:41:04.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-26T06:41:04.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.897 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.897 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.898 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.898 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:04.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:04.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.898 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.898 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.899 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.899 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.899 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.899 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.899 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.899 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.899 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.900 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.900 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.900 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.900 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:04.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:04.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.901 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.901 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.902 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.902 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:04.902 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:04.902 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.902 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.902 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.903 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:04.903 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.903 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.903 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.903 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.903 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.903 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.903 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.903 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.904 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.904 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.904 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.904 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.904 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:04.904 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:04.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.905 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.905 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.905 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-03-26T06:41:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-03-26T06:41:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.907 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-03-26T06:41:04.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.907 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.908 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-03-26T06:41:04.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T06:41:04.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.908 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.909 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-26T06:41:04.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-26T06:41:04.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.909 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-26T06:41:04.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.910 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.910 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.910 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.910 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.911 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-26T06:41:04.911 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-26T06:41:04.911 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.911 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.911 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-03-26T06:41:04.911 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.912 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.912 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.912 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.912 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.912 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.912 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-26T06:41:04.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-26T06:41:04.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.913 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.913 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.913 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.914 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-03-26T06:41:04.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-03-26T06:41:04.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.914 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.914 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-03-26T06:41:04.914 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.915 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.915 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.915 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.915 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-26T06:41:04.916 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-26T06:41:04.916 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.916 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.916 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.916 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-03-26T06:41:04.916 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.916 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.917 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.917 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.917 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-03-26T06:41:04.917 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.917 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.917 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.917 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.917 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.918 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-26T06:41:04.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-26T06:41:04.918 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.918 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.918 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.918 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-26T06:41:04.918 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-26T06:41:04.918 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-26T06:41:04.919 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.919 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.919 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.919 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.919 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-03-26T06:41:04.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-03-26T06:41:04.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.920 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.920 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.920 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.920 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.920 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.920 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.920 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.920 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-03-26T06:41:04.921 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:04.921 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:04.922 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:04.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-03-26T06:41:04.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-03-26T06:41:04.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.922 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.922 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.922 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.923 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.923 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.923 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-03-26T06:41:04.923 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.923 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.923 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.923 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:04.924 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:04.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.924 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.924 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.924 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.924 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.924 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.925 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.925 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.925 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.925 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.925 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.925 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.925 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.925 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.926 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.926 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:04.926 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:04.926 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.926 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.926 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.927 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.927 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.927 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.927 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.927 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.927 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.927 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.928 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:04.928 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:04.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.928 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:04.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.928 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.929 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.929 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.929 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.929 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.929 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.929 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.929 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.929 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.929 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.930 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.930 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.930 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.930 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:04.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:04.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.930 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.931 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.931 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.931 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.931 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.931 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.932 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-03-26T06:41:04.932 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-03-26T06:41:04.932 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.932 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.932 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.932 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-03-26T06:41:04.932 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.932 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.933 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.933 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T06:41:04.933 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.933 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.933 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.933 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-03-26T06:41:04.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-03-26T06:41:04.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.933 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.934 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.934 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-03-26T06:41:04.934 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T06:41:04.934 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.934 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.935 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-03-26T06:41:04.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-03-26T06:41:04.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.935 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.935 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-03-26T06:41:04.935 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-03-26T06:41:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-03-26T06:41:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-03-26T06:41:04.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.937 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.937 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.937 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-26T06:41:04.937 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.937 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.937 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T06:41:04.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.938 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.938 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.938 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-03-26T06:41:04.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-03-26T06:41:04.938 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.938 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:04.939 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.939 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T06:41:04.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.939 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.940 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:04.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:04.940 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.940 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:04.940 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.940 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.941 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T06:41:04.941 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.941 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.941 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.941 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-03-26T06:41:04.941 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-03-26T06:41:04.941 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.941 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.942 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.942 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-03-26T06:41:04.942 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.942 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.942 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.942 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T06:41:04.942 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.943 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.943 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-03-26T06:41:04.943 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:04.943 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:04.943 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:04.943 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:04.944 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:04.944 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.944 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.944 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.944 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.944 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.944 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.944 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.944 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.945 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.945 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.945 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.945 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.945 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.945 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.945 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.945 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.945 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.946 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:04.946 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:04.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.946 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.946 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.946 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.949 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.949 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.949 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.949 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.949 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.949 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.949 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.949 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.949 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.950 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:04.950 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:04.950 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.950 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.950 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.950 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.950 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.950 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.951 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.951 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.951 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.951 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.951 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.951 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-03-26T06:41:04.951 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-03-26T06:41:04.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.951 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.952 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-26T06:41:04.952 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.952 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.952 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.952 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.952 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.952 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.953 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-03-26T06:41:04.953 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-03-26T06:41:04.953 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.953 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.953 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.953 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-03-26T06:41:04.953 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.953 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.954 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.954 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.954 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.954 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.954 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.954 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:04.954 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:04.954 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.954 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.955 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.955 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:04.955 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.955 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.955 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.955 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.955 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.955 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.956 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.956 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.956 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-03-26T06:41:04.956 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-03-26T06:41:04.956 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-03-26T06:41:04.956 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:04.956 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-03-26T06:41:04.957 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-03-26T06:41:04.957 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-03-26T06:41:04.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.957 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T06:41:04.958 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.958 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.958 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-03-26T06:41:04.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-03-26T06:41:04.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.958 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.958 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-03-26T06:41:04.959 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.959 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.959 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.959 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T06:41:04.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.959 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.960 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-03-26T06:41:04.960 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-03-26T06:41:04.960 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.960 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.960 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.960 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.960 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.961 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-03-26T06:41:04.961 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.961 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.961 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.961 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:04.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:04.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:04.961 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.962 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-03-26T06:41:04.962 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-03-26T06:41:04.962 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.962 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.962 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T06:41:04.962 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.962 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.963 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:04.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:04.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.963 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.963 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.963 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.963 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.964 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.964 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.964 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.964 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.964 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.964 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.964 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.964 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.964 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.964 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.965 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:04.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:04.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.965 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.965 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.966 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.966 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.966 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.966 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.966 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.966 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:04.967 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:04.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.967 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:04.967 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.967 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.967 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.967 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.968 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.968 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.968 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.968 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.968 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.968 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.968 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.968 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.968 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.969 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:04.969 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:04.969 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.969 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.969 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.969 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.969 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.970 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.970 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.970 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-03-26T06:41:04.970 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-03-26T06:41:04.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.971 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.971 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.971 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-03-26T06:41:04.971 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.971 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.972 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.972 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-03-26T06:41:04.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-03-26T06:41:04.972 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.972 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.972 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-03-26T06:41:04.972 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.973 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.973 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-03-26T06:41:04.973 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.973 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.973 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.973 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-26T06:41:04.973 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-26T06:41:04.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-03-26T06:41:04.974 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.974 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.974 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.974 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T06:41:04.974 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.975 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-03-26T06:41:04.975 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-03-26T06:41:04.975 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.975 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.975 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.975 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:04.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:04.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-03-26T06:41:04.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T06:41:04.976 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.976 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.977 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-03-26T06:41:04.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-03-26T06:41:04.977 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.977 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.977 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.977 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.977 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.977 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.977 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.978 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-03-26T06:41:04.978 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.978 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.978 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.978 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-03-26T06:41:04.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-03-26T06:41:04.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.978 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.979 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.979 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-03-26T06:41:04.979 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.979 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.979 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T06:41:04.979 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.979 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.980 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.980 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.980 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-03-26T06:41:04.980 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:04.980 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:04.980 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:04.980 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:04.981 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:04.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.981 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.981 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.981 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.981 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.981 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.981 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.982 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.982 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.982 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.982 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.982 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.982 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.982 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.982 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.982 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.983 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.983 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:04.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:04.983 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.983 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.983 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.983 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.983 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.984 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.984 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.984 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.984 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.984 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:04.984 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:04.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.985 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:04.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.985 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.985 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.985 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.985 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.986 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.986 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.986 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.986 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.986 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.986 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.986 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.986 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.987 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:04.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:04.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.987 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.987 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.988 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.988 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.988 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.988 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-03-26T06:41:04.989 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:04.989 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:04.989 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:04.989 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:04.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:04.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.989 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.989 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.990 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:04.990 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.990 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.990 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.990 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.990 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.990 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.990 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.990 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.991 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.991 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.991 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.991 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.991 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.991 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.991 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:04.991 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:04.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.992 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.992 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.992 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.992 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.992 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.992 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.992 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.993 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:04.993 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:04.993 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:04.993 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.993 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.993 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:04.994 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.994 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.994 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:04.994 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:04.994 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:04.994 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:04.994 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:04.994 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:04.995 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:04.995 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:04.995 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.995 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.995 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.995 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:04.995 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:04.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:04.996 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:04.996 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:04.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.996 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.996 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.996 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:04.996 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.997 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.997 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.997 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:04.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:04.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-03-26T06:41:04.997 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:04.997 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:04.998 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:04.998 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-03-26T06:41:04.998 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-03-26T06:41:04.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.998 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:04.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:04.998 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-03-26T06:41:04.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:04.998 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:04.999 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:04.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T06:41:04.999 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:04.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:04.999 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:04.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:04.999 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-03-26T06:41:04.999 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-03-26T06:41:04.999 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:04.999 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.000 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.000 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:05.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-03-26T06:41:05.000 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.000 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.000 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T06:41:05.000 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.002 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.002 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.002 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.002 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.002 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.002 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.003 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.003 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.003 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.003 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.003 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.003 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.003 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.004 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.004 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.004 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.004 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.004 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.004 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.004 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.005 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.005 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.005 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.005 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.005 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.005 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.005 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.006 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.006 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.006 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.006 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.006 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.006 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.006 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.006 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.007 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.007 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.007 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.007 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.007 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.007 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.007 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.007 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.007 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.007 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.008 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.008 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.008 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.008 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.008 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.008 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.008 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.009 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.009 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.009 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.009 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.009 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.010 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-03-26T06:41:05.010 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.010 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.010 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.010 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-03-26T06:41:05.010 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-03-26T06:41:05.010 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.010 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.010 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-03-26T06:41:05.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.011 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-03-26T06:41:05.011 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T06:41:05.011 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.011 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.012 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.012 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.012 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.012 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.012 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.012 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.012 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.012 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.012 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.012 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.013 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.013 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.013 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.013 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.013 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.013 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.013 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.013 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.014 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.014 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.014 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.014 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.015 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.015 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.015 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.015 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.015 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.016 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.016 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.016 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.016 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.016 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.016 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.016 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.016 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.017 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.017 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.017 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.017 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.017 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.017 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.017 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.018 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.018 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.018 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.018 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.018 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.018 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.019 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-03-26T06:41:05.019 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-03-26T06:41:05.019 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.020 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.020 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.020 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.020 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.020 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-03-26T06:41:05.020 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T06:41:05.020 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.020 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.021 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-03-26T06:41:05.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-03-26T06:41:05.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.021 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-03-26T06:41:05.021 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.021 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.022 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.022 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-03-26T06:41:05.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.022 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.022 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-03-26T06:41:05.022 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-03-26T06:41:05.023 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.023 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.023 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.023 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-03-26T06:41:05.023 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.023 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.023 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.023 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T06:41:05.023 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.024 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.024 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.024 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-03-26T06:41:05.024 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-03-26T06:41:05.024 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.025 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.025 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.025 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.025 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-03-26T06:41:05.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.025 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.025 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-03-26T06:41:05.026 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-03-26T06:41:05.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.026 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-03-26T06:41:05.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.026 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T06:41:05.027 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T06:41:05.027 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.027 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.027 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.027 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-03-26T06:41:05.028 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.028 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.028 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.028 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.028 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.028 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.028 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.028 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.028 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.029 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.029 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.029 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.029 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.029 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.029 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.029 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.029 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.029 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.030 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.030 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.030 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.030 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.031 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.031 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.031 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.031 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.031 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.032 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.032 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.032 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.032 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.032 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.032 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.033 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.033 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.033 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.033 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.033 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.033 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.033 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.033 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.034 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.034 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.034 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.034 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.034 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.034 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.034 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.034 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.035 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.035 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.035 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.035 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.035 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.035 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.036 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.036 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.036 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.036 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.036 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.037 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-03-26T06:41:05.037 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.037 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.037 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.037 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.037 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.038 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.038 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.038 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.038 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.039 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.039 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.039 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.039 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.039 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.039 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.039 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.039 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.040 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.040 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.040 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.040 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.040 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.040 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.040 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.041 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.041 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.041 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.041 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.041 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.041 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.041 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.041 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.042 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.042 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.042 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.042 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.042 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.042 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.042 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.043 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.043 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.043 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.043 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.043 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.043 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.043 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.043 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.044 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.044 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.044 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.044 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.044 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.044 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.044 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.045 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.045 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.045 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.045 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.045 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.045 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.045 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.046 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.046 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.046 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.046 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.046 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.046 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-03-26T06:41:05.047 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.047 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.047 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.047 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.048 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.048 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.048 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.048 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.048 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.048 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.048 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.048 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.049 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.049 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.049 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.049 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.049 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.050 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.050 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.050 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.050 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.050 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.050 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.051 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.051 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.051 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.051 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.051 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.051 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.052 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.052 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.052 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.052 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.052 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.052 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.052 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.052 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.052 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.053 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.053 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.053 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.053 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.053 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.053 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.053 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.054 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.054 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.054 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.054 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.054 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.054 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.055 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.055 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.055 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-26T06:41:05.055 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.055 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.055 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.056 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-03-26T06:41:05.056 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-03-26T06:41:05.056 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.056 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.056 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.056 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.056 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.056 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.056 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.057 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T06:41:05.057 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.057 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.057 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.057 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.058 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.058 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.058 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.058 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.058 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.058 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.058 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.058 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.059 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.059 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.059 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.059 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.059 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.059 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.059 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.059 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.059 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.060 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.060 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.060 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.060 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.060 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.060 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.061 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.061 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.061 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.061 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.062 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.062 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.062 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.062 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.062 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.062 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.062 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.062 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.062 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.063 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.063 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.063 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.063 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.063 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.063 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.063 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.063 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.064 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.064 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.064 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.064 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.064 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.064 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.064 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.065 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.065 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-03-26T06:41:05.065 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-26T06:41:05.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.065 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.065 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.065 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.066 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.066 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.066 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-03-26T06:41:05.066 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-03-26T06:41:05.066 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-03-26T06:41:05.066 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.066 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-03-26T06:41:05.066 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.067 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.067 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.067 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.067 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-03-26T06:41:05.067 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.067 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.067 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.068 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.068 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.068 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.068 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.068 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.068 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.069 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.069 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.069 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.069 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.069 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.069 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.069 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.069 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.069 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.070 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.070 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.070 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.071 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.071 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.071 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.071 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.071 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.071 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.071 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.072 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.072 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.072 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.072 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.072 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.072 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.073 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.073 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.073 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.073 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.073 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.073 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.073 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.073 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.073 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.073 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.074 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.074 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.074 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.075 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.075 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.075 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-03-26T06:41:05.076 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.076 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.076 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.076 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.077 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.077 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.077 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.077 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.077 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.077 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.077 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.077 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.077 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.078 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.078 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.078 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.078 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.078 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.079 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.079 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.079 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.079 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.079 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.079 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.079 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.080 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.080 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.080 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.080 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.080 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.081 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.081 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.081 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.081 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.081 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.081 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.081 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.081 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.081 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.082 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.082 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.082 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.082 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.082 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.082 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.083 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.083 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.083 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.083 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.083 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.083 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.084 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-03-26T06:41:05.084 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.085 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.085 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.085 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.085 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.085 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.085 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.086 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.086 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.086 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.086 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.086 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.086 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.086 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.086 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.086 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.087 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.087 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.087 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.087 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.087 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.087 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.088 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.088 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.088 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.088 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.088 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.089 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.089 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.089 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.089 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.089 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.089 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.090 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.090 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.090 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.090 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.090 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.090 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.090 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.090 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.091 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.091 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.091 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.091 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.091 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.091 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.091 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.091 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.092 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.092 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.092 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.092 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.092 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.092 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.092 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:05.093 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:05.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.093 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.093 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:05.093 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.093 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.093 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.093 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.094 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.094 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.094 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-03-26T06:41:05.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-03-26T06:41:05.094 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:05.094 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-26T06:41:05.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-03-26T06:41:05.095 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.095 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.095 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-03-26T06:41:05.095 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T06:41:05.096 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.096 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.096 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.096 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-03-26T06:41:05.096 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.097 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.097 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.097 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-03-26T06:41:05.097 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-03-26T06:41:05.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.097 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.097 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.097 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.097 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.097 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.098 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T06:41:05.098 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.098 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.098 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-26T06:41:05.098 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-26T06:41:05.099 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.099 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.099 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.099 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.099 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.099 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.099 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.099 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.099 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.100 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.100 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.100 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.100 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.100 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.102 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.102 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.102 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.102 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.102 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.102 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.102 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.103 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.103 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.103 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.103 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.103 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.103 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.103 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.103 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.104 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.104 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.104 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.104 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.104 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.104 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.105 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.105 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.105 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.105 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.105 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.105 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.105 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.105 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.106 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.106 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.106 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.106 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.106 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.106 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.106 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.106 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.107 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.107 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.107 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.107 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.107 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.108 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.108 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.108 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.108 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.108 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.108 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.108 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.108 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-03-26T06:41:05.109 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-03-26T06:41:05.109 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.109 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.109 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.109 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-03-26T06:41:05.109 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.109 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.109 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.109 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-03-26T06:41:05.109 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.110 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.110 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:05.110 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:05.110 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.110 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.110 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.110 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:05.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.111 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.111 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.111 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-03-26T06:41:05.111 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.111 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.111 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.111 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.111 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.112 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-03-26T06:41:05.112 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.112 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.112 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.112 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-03-26T06:41:05.112 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-03-26T06:41:05.112 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.113 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.113 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.113 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.113 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.113 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.113 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.113 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.113 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.114 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-03-26T06:41:05.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-03-26T06:41:05.114 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.114 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.114 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.114 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.114 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.114 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.115 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.115 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T06:41:05.115 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T06:41:05.115 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.115 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.115 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.115 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.116 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.116 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.116 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.116 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.116 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.116 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.116 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.116 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.117 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.117 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.117 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.117 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.117 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.117 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.117 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.118 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.118 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.118 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.118 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.119 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.119 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.119 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.119 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.119 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.120 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.120 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.120 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.120 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.120 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.120 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.120 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.121 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.121 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.121 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.121 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.121 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.121 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.121 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.121 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.122 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.122 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.122 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.122 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.122 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.122 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.122 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.123 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-03-26T06:41:05.123 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-03-26T06:41:05.123 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.123 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.123 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.124 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.124 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.124 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.124 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.124 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.124 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.124 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.124 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.124 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T06:41:05.125 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T06:41:05.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.125 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.125 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-03-26T06:41:05.125 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.125 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.126 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.126 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.126 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.126 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.126 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-03-26T06:41:05.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-03-26T06:41:05.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:05.127 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.127 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.127 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-03-26T06:41:05.127 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.127 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.127 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.128 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-26T06:41:05.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-26T06:41:05.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-26T06:41:05.128 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.128 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T06:41:05.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.129 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-26T06:41:05.129 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-26T06:41:05.129 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.129 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-03-26T06:41:05.130 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.130 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.130 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.130 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T06:41:05.130 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.130 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.130 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.131 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-03-26T06:41:05.131 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-03-26T06:41:05.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.131 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.131 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-03-26T06:41:05.131 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.132 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.132 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.132 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-26T06:41:05.132 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-26T06:41:05.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-03-26T06:41:05.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.133 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-03-26T06:41:05.133 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-03-26T06:41:05.134 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.134 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.134 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-26T06:41:05.134 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-26T06:41:05.134 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.134 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-26T06:41:05.135 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-26T06:41:05.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-26T06:41:05.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.135 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.135 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.135 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-03-26T06:41:05.136 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.136 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.137 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.137 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.137 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.137 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.137 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.137 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.137 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.137 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.137 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.138 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.138 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.138 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.138 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.138 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.138 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.138 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.138 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.138 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.139 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.139 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.139 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.140 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.140 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.140 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.140 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.141 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.141 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.141 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.141 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.141 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.141 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.141 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.141 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.142 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.142 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.142 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.142 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.142 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.142 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.142 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.142 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.143 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.143 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.143 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.143 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.143 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.143 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.143 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.144 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.144 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.144 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.144 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.144 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.144 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.144 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.144 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-03-26T06:41:05.145 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-03-26T06:41:05.145 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.145 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.145 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.145 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:05.145 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.145 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.145 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.145 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.146 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.146 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.146 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.146 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-03-26T06:41:05.146 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T06:41:05.146 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.146 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.146 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.147 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.147 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.147 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.147 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.148 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-03-26T06:41:05.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-03-26T06:41:05.148 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.148 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.148 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.148 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.148 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.149 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.149 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.149 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.150 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-03-26T06:41:05.150 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.150 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.150 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.150 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-03-26T06:41:05.150 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-03-26T06:41:05.150 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.150 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.151 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.151 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.151 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.151 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.151 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.151 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.151 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.151 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.152 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.152 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-26T06:41:05.152 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-26T06:41:05.152 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.152 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.152 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.152 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.153 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.153 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.153 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.153 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.153 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.153 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.153 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.154 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.154 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.154 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.154 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.154 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.154 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.155 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.155 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.155 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.155 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.155 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.155 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.155 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.155 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.155 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.155 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.156 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.156 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.156 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.157 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.157 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.157 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.157 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.157 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.158 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.158 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.158 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.158 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.158 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.158 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.159 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.159 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.159 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.159 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.159 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.159 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.159 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.159 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.159 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.160 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.160 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.160 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.160 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.160 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.160 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.161 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.161 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.161 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.161 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.161 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.162 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-26T06:41:05.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-26T06:41:05.162 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.162 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.162 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.162 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.162 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.162 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.163 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.163 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.163 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.163 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.163 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-26T06:41:05.163 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-26T06:41:05.163 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.164 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.164 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.164 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.164 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.164 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.164 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.164 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.164 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.165 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.165 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.165 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.165 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-03-26T06:41:05.165 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.165 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.166 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.166 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.166 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.166 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.166 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.166 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.166 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.167 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.167 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.167 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.167 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.167 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.167 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.167 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.167 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.167 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.168 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.168 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.168 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.168 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.168 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.168 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.169 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.169 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.169 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.169 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.169 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.169 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.170 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.170 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.170 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.170 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.170 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.171 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.171 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.171 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.171 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.171 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.171 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.171 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.171 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.171 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.172 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.172 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.172 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.172 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.172 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.172 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.172 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.172 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.173 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.173 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.173 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.173 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.173 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.173 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.173 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.173 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.173 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.174 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.174 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-03-26T06:41:05.174 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.174 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.174 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.174 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.175 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.175 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.175 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.175 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.175 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.176 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.176 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.176 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.176 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.176 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.176 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.176 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.176 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.177 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.177 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.177 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.177 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.178 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.178 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.178 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.178 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.178 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.178 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.178 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.178 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.179 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.179 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.179 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.179 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.179 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.179 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.179 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.179 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.180 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.180 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.180 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.180 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.180 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.180 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.180 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.180 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.181 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.181 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.181 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.181 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.181 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.181 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.181 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.182 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.182 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.182 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.182 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.182 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.182 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.182 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.182 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.183 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-03-26T06:41:05.183 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-03-26T06:41:05.183 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.183 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.183 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.183 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-03-26T06:41:05.183 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.183 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.183 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.184 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T06:41:05.184 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.184 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.184 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.184 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.184 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-03-26T06:41:05.185 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.185 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.185 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.185 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.185 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.185 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.185 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.185 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.186 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.186 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.186 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.186 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.186 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.186 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.186 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.186 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.187 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.187 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.187 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.187 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.187 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.187 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.187 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.187 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.188 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.188 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.188 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.188 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.188 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.188 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.188 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.188 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.189 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.189 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.189 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.189 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.189 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.190 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.190 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.190 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.190 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.190 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.190 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.190 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.190 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.190 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.190 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.191 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.191 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.191 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.191 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.191 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.191 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.191 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.191 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.192 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.192 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.192 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.192 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.192 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.192 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.193 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-03-26T06:41:05.193 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-03-26T06:41:05.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.193 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.193 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.193 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.194 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.194 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.194 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.194 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.194 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.194 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-03-26T06:41:05.194 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-03-26T06:41:05.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.195 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.195 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.195 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-03-26T06:41:05.195 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.195 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.195 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.195 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.196 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.196 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.196 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-03-26T06:41:05.196 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-03-26T06:41:05.196 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.196 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.197 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.197 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.197 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.197 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.197 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.197 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.197 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.197 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-03-26T06:41:05.198 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-03-26T06:41:05.198 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.198 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.198 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.198 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.198 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.198 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.198 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.199 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.199 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.199 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.199 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.199 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-03-26T06:41:05.199 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-03-26T06:41:05.199 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:05.200 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.200 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.200 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.200 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.200 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.200 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.200 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.200 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.201 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.201 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-03-26T06:41:05.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-03-26T06:41:05.201 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.201 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.201 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.201 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.201 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.202 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.202 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.202 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.202 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.202 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.202 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.202 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-03-26T06:41:05.203 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-03-26T06:41:05.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.203 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.203 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.203 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.203 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.203 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.203 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.204 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.204 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.204 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.204 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.204 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-03-26T06:41:05.204 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-03-26T06:41:05.204 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.205 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.205 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.205 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.205 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.205 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.205 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.205 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.205 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.206 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.206 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-03-26T06:41:05.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-03-26T06:41:05.206 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.206 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.206 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.206 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.207 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:05.207 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-03-26T06:41:05.207 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.207 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.207 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-03-26T06:41:05.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-03-26T06:41:05.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.208 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.208 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.208 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.208 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.208 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.209 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.209 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.209 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.209 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-03-26T06:41:05.209 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-03-26T06:41:05.209 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.209 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.209 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.210 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.210 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.210 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.210 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.210 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.210 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.210 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.210 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.211 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-03-26T06:41:05.211 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-03-26T06:41:05.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.211 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.211 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.211 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.211 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.211 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-03-26T06:41:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.213 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.213 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.213 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-03-26T06:41:05.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-03-26T06:41:05.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.213 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.213 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.213 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.214 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T06:41:05.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.214 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-03-26T06:41:05.214 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-03-26T06:41:05.214 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.215 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.215 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.215 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.215 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.215 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-03-26T06:41:05.215 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.215 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.216 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.216 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.216 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.216 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.216 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.216 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.216 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.217 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.217 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.217 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.217 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.217 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.217 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.217 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.217 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.217 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.218 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.218 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.218 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.218 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.218 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.218 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.219 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.219 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.219 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.219 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.219 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.219 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.219 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.219 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.220 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.220 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.220 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.220 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.221 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.221 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.221 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.221 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.221 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.221 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.221 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.221 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.221 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.222 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.222 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.222 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.222 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.222 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.222 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.223 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.223 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.223 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.223 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.223 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.223 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.223 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.224 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.224 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.224 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-03-26T06:41:05.224 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.224 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.224 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.224 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.225 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.225 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.225 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.225 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.225 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.225 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.225 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.225 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.226 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.226 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.226 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.226 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.226 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.226 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.226 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.226 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.227 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.227 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.227 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.227 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.227 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.227 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.227 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.227 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.228 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.228 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.228 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.228 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.228 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.228 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.229 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.229 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.229 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.229 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.229 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.229 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.229 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.230 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.230 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.230 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.230 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.230 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.230 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.230 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.230 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.231 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.231 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.231 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.231 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.231 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.231 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.232 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.232 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.232 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.232 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.232 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.232 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.232 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.233 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-03-26T06:41:05.233 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.233 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.233 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.233 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.233 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.234 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.234 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.234 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.234 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.234 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.234 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.235 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.235 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.235 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.235 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.235 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.235 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.235 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.236 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.236 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.236 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.236 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.237 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.237 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.237 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.237 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.238 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.238 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.238 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.238 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.238 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.238 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.238 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.238 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.239 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.239 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.239 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.239 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.239 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.239 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.239 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.240 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.240 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.240 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.240 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.240 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.240 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.240 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.240 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.240 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.240 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.241 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.241 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.241 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.241 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.241 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.241 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-03-26T06:41:05.241 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.242 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.242 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.242 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-03-26T06:41:05.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-03-26T06:41:05.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.242 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.242 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T06:41:05.242 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.243 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.243 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.243 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.243 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.243 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.243 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:05.243 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:05.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.244 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.244 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-03-26T06:41:05.244 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.244 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.244 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.244 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.244 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.244 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.245 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.245 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.245 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.245 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.245 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.245 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.245 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.246 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.246 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.246 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.246 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.246 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.246 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.246 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.246 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.247 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.247 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.247 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.247 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.247 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.247 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.247 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.248 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.248 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.248 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.248 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.248 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.248 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.248 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.249 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.249 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.249 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.249 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.250 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.250 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.250 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.250 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.250 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.250 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.250 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.251 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.251 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.251 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.251 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.251 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.251 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.251 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.252 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.252 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.252 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.252 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.252 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.252 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.252 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.252 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.253 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.253 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.253 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.253 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-03-26T06:41:05.253 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.253 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.253 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.253 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-03-26T06:41:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-03-26T06:41:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.255 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T06:41:05.255 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.255 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.255 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.255 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-03-26T06:41:05.255 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-03-26T06:41:05.255 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.256 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.256 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.256 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.256 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.256 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.256 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.256 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T06:41:05.256 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-03-26T06:41:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-03-26T06:41:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.258 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.258 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.258 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T06:41:05.258 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.258 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.258 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-03-26T06:41:05.259 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-03-26T06:41:05.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.259 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.259 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.259 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.259 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.259 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T06:41:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-03-26T06:41:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-03-26T06:41:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.261 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.261 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.261 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.261 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.261 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.261 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T06:41:05.261 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.261 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.262 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.262 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-03-26T06:41:05.262 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-03-26T06:41:05.262 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.262 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.262 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.262 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.263 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.263 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.263 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.263 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.263 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.263 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.263 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.263 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-03-26T06:41:05.263 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-03-26T06:41:05.264 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.264 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.264 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.264 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.264 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.264 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.264 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.264 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.265 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.265 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.265 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-03-26T06:41:05.265 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-03-26T06:41:05.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.265 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.266 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T06:41:05.266 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.266 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.266 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.266 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.266 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.266 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.267 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-26T06:41:05.267 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-26T06:41:05.267 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.267 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.267 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.267 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.267 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.268 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.268 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.268 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.268 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.268 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.268 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:05.268 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:05.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.269 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.269 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.269 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-03-26T06:41:05.269 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-03-26T06:41:05.269 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.269 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.269 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.269 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.270 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.270 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.270 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-03-26T06:41:05.270 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-03-26T06:41:05.270 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.270 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.270 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.271 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-03-26T06:41:05.271 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.271 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.271 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.271 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.271 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.271 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.272 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-03-26T06:41:05.272 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-03-26T06:41:05.272 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.272 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.272 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.272 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.272 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.272 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.272 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.273 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.273 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.273 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.273 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.273 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-03-26T06:41:05.273 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-03-26T06:41:05.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.273 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.274 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.274 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.274 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.274 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.274 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.274 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.274 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.275 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.275 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.275 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.275 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.275 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.275 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.276 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.276 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.276 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.276 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.276 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.276 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.276 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.276 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.276 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.277 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.277 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.277 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.277 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.278 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.278 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.278 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.278 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.278 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.278 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.279 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.279 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.279 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.279 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.279 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.279 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.280 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.280 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.280 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.280 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.280 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.280 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.280 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.280 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.281 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.281 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.281 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.281 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.281 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.281 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.281 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.281 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.282 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.282 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.282 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.282 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.282 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.282 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.283 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.283 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.283 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.283 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-03-26T06:41:05.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-03-26T06:41:05.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.283 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.284 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.284 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.284 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.284 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.284 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.285 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.285 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-03-26T06:41:05.285 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-03-26T06:41:05.285 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.285 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.285 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.285 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.286 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.286 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.286 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.286 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.286 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.286 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.286 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.287 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-03-26T06:41:05.287 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-03-26T06:41:05.287 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.287 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.287 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.287 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T06:41:05.287 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.287 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.288 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.288 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.288 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.288 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.288 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.289 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-03-26T06:41:05.289 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.289 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.289 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.289 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.289 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.289 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.290 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.290 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.290 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.290 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.290 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.290 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.290 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.290 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.291 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.291 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.291 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.291 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.291 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.291 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.291 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.291 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.292 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.292 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.292 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.292 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.293 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.293 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.293 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.293 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.294 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.294 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.294 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.294 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.295 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.295 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.295 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.295 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.295 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.295 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.296 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.296 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.296 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.296 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.296 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.296 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.297 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.297 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.297 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.297 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.297 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.297 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.298 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-03-26T06:41:05.298 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.298 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.298 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.298 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.299 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.299 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.299 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.299 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.299 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.299 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.299 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.300 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.300 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.300 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.300 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.300 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.300 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.300 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.301 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.301 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.301 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.301 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.301 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.301 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.302 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.302 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.302 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.302 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.302 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.302 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.302 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.302 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.303 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.303 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.303 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.303 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.303 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.303 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.304 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.304 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.304 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.304 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.304 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.304 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.305 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.305 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.305 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.305 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.305 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.306 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.306 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.306 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.306 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.306 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.306 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.306 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.306 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.306 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.307 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.307 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.307 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-03-26T06:41:05.307 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-03-26T06:41:05.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.307 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.308 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-03-26T06:41:05.308 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.308 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.308 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.308 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T06:41:05.308 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.308 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.308 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.309 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-03-26T06:41:05.309 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-03-26T06:41:05.309 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.309 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.309 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.309 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.309 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.310 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.310 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-03-26T06:41:05.310 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.310 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.310 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.311 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-03-26T06:41:05.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-03-26T06:41:05.311 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.311 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.311 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.311 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:05.311 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.312 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.312 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.312 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T06:41:05.312 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.312 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.312 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.312 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-03-26T06:41:05.313 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T06:41:05.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.313 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.313 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.313 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.313 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.313 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.313 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.313 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T06:41:05.314 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.314 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-03-26T06:41:05.315 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.315 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.315 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.315 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-03-26T06:41:05.315 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-03-26T06:41:05.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:05.315 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.316 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-03-26T06:41:05.316 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.316 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.316 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.316 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.316 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.316 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.316 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-03-26T06:41:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-03-26T06:41:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.329 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.329 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.329 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-03-26T06:41:05.329 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-03-26T06:41:05.329 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.329 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.329 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.330 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.330 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.330 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.330 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.330 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.330 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.330 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.331 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-03-26T06:41:05.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-03-26T06:41:05.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-03-26T06:41:05.332 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.332 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.332 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.332 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.332 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.332 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.332 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.333 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-03-26T06:41:05.333 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-03-26T06:41:05.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.333 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.333 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T06:41:05.333 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.333 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.334 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.334 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.334 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.334 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.335 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.335 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:41:05.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.335 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 25019, 2024-03-26T06:41:05.335 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "y", 2024-03-26T06:41:05.335 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-03-26T06:41:05.336 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-03-26T06:41:05.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-03-26T06:41:05.336 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.336 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.336 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.336 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:05.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:05.336 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.337 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:05.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.337 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T06:41:05.337 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.338 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.338 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.339 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.339 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.339 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.339 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.340 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.340 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.340 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.340 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.340 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.340 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.340 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.341 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.341 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.341 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.341 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.341 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.341 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.342 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.342 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.342 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.342 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.343 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.343 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.343 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.343 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.343 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.343 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.343 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.343 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.344 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.344 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.344 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.344 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.344 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.345 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.345 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.345 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.345 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.345 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.345 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.345 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.346 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.346 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.346 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.346 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.346 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-03-26T06:41:05.347 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-03-26T06:41:05.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.347 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.347 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.347 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.348 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-03-26T06:41:05.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.348 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-03-26T06:41:05.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-03-26T06:41:05.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-03-26T06:41:05.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.349 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.349 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.349 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-03-26T06:41:05.349 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.349 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.350 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-03-26T06:41:05.350 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-03-26T06:41:05.350 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.350 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.350 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.350 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.351 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.351 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-03-26T06:41:05.351 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.351 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.352 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-03-26T06:41:05.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-03-26T06:41:05.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.352 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.352 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.352 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.352 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.352 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.353 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.353 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-03-26T06:41:05.353 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.353 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.353 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-03-26T06:41:05.353 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-03-26T06:41:05.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.354 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.354 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.354 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-03-26T06:41:05.354 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.354 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.354 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.354 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-03-26T06:41:05.354 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.355 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-03-26T06:41:05.355 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-03-26T06:41:05.355 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.355 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.355 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.355 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.356 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.356 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.356 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-03-26T06:41:05.356 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.356 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.356 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.357 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-03-26T06:41:05.357 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-03-26T06:41:05.357 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.357 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.357 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.357 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.357 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.358 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T06:41:05.358 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.358 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.358 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.358 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-03-26T06:41:05.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-03-26T06:41:05.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.359 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.359 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.359 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.359 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.359 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.359 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-03-26T06:41:05.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.360 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.360 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-03-26T06:41:05.361 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.361 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.361 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.361 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-03-26T06:41:05.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-03-26T06:41:05.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.361 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.362 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.362 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.362 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.362 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.362 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T06:41:05.362 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.363 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.363 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-03-26T06:41:05.363 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-03-26T06:41:05.363 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.363 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.363 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.363 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-03-26T06:41:05.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.364 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.364 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T06:41:05.364 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T06:41:05.364 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.364 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.364 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-03-26T06:41:05.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-03-26T06:41:05.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:05.365 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.365 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.365 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:05.365 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-26T06:41:05.365 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-26T06:41:05.365 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.366 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T06:41:05.366 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T06:41:05.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.366 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.366 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-03-26T06:41:05.366 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-03-26T06:41:05.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:05.367 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.367 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-03-26T06:41:05.367 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:05.367 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-03-26T06:41:05.367 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.367 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T06:41:05.368 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.368 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-03-26T06:41:05.368 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-03-26T06:41:05.368 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.368 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.368 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.369 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T06:41:05.369 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.369 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.369 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.369 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T06:41:05.369 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T06:41:05.369 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.369 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.370 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-03-26T06:41:05.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-03-26T06:41:05.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:05.370 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.370 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-03-26T06:41:05.370 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-03-26T06:41:05.370 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-03-26T06:41:05.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.371 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-03-26T06:41:05.371 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T06:41:05.371 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.371 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.371 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-03-26T06:41:05.371 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-03-26T06:41:05.372 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.372 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.372 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.372 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-03-26T06:41:05.372 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.372 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.372 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.372 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T06:41:05.372 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T06:41:05.373 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.373 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.373 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-03-26T06:41:05.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-03-26T06:41:05.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:05.373 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.374 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:05.374 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-26T06:41:05.374 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-26T06:41:05.374 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.374 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T06:41:05.374 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T06:41:05.374 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.374 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.375 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.375 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.375 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.375 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.375 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.376 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.376 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.376 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.376 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.376 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.376 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.376 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.377 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.377 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.377 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.378 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.378 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.378 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.379 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.379 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.379 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.379 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.380 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.380 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.380 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.380 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.380 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.381 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.381 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.381 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.381 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.381 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.381 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.381 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.382 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.382 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.382 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.382 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.382 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.382 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.383 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.383 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-03-26T06:41:05.383 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-03-26T06:41:05.383 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:05.383 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.383 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.384 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:05.384 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.384 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.384 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.384 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T06:41:05.384 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.384 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.384 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.385 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-26T06:41:05.385 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-26T06:41:05.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.385 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.385 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-03-26T06:41:05.385 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.385 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.386 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.386 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-03-26T06:41:05.386 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-03-26T06:41:05.386 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-03-26T06:41:05.386 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-03-26T06:41:05.386 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-03-26T06:41:05.386 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.386 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-03-26T06:41:05.387 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.387 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.387 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-03-26T06:41:05.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-03-26T06:41:05.387 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.387 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.387 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.388 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.388 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.388 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T06:41:05.388 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.388 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.388 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.389 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:05.389 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:05.389 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.389 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.389 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:05.389 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.390 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T06:41:05.390 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.390 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.390 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-03-26T06:41:05.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-03-26T06:41:05.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.390 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:05.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:05.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T06:41:05.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T06:41:05.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.392 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-03-26T06:41:05.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-03-26T06:41:05.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:05.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:05.392 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.393 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T06:41:05.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.393 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.393 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.393 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.394 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-03-26T06:41:05.394 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.394 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.394 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.394 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-03-26T06:41:05.394 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-03-26T06:41:05.394 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:05.394 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-26T06:41:05.395 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.395 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.395 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.395 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T06:41:05.395 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.395 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.396 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-03-26T06:41:05.396 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-03-26T06:41:05.396 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.396 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.396 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.396 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-03-26T06:41:05.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.397 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.397 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T06:41:05.397 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.397 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.397 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.398 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-03-26T06:41:05.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-03-26T06:41:05.398 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.398 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.398 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.398 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-03-26T06:41:05.398 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.398 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.399 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T06:41:05.399 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.399 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.399 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-03-26T06:41:05.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-03-26T06:41:05.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.400 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.400 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T06:41:05.400 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T06:41:05.400 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.401 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.401 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-03-26T06:41:05.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-03-26T06:41:05.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:05.401 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.402 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T06:41:05.402 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.402 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.402 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.403 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-03-26T06:41:05.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T06:41:05.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:05.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.403 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-03-26T06:41:05.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.404 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T06:41:05.404 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.404 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.405 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-03-26T06:41:05.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-03-26T06:41:05.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.405 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.405 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.406 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-03-26T06:41:05.406 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-03-26T06:41:05.406 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-03-26T06:41:05.406 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.406 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T06:41:05.406 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.406 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.406 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-03-26T06:41:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-03-26T06:41:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.407 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.408 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T06:41:05.408 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.408 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.408 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.408 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-03-26T06:41:05.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-03-26T06:41:05.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.409 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.409 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.409 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.409 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.409 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.409 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T06:41:05.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.410 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.410 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.410 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-03-26T06:41:05.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-03-26T06:41:05.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.410 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.411 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.411 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.411 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.411 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T06:41:05.411 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.412 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-03-26T06:41:05.412 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-03-26T06:41:05.412 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.412 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T06:41:05.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.413 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:05.413 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.413 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-03-26T06:41:05.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-03-26T06:41:05.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.414 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.414 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T06:41:05.414 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.414 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.414 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-03-26T06:41:05.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.415 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.415 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.415 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-03-26T06:41:05.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-03-26T06:41:05.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.415 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.415 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.415 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T06:41:05.416 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-03-26T06:41:05.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.416 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.416 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-03-26T06:41:05.417 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-03-26T06:41:05.417 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.417 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.417 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T06:41:05.417 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.417 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.417 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:05.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.418 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-03-26T06:41:05.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-03-26T06:41:05.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.418 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T06:41:05.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.419 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.419 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-03-26T06:41:05.419 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.419 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.419 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.420 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-03-26T06:41:05.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-03-26T06:41:05.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.420 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.420 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.420 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T06:41:05.420 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-03-26T06:41:05.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.421 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.421 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-03-26T06:41:05.421 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-03-26T06:41:05.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.422 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.422 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T06:41:05.422 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.422 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.422 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.422 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-03-26T06:41:05.423 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.423 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-03-26T06:41:05.423 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-03-26T06:41:05.423 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.423 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.423 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.424 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T06:41:05.424 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.424 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.424 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.424 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-03-26T06:41:05.424 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.424 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.424 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.425 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-03-26T06:41:05.425 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-03-26T06:41:05.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.425 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T06:41:05.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.426 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.426 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-03-26T06:41:05.426 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.426 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-03-26T06:41:05.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-03-26T06:41:05.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.427 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.427 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.427 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T06:41:05.427 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.427 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-03-26T06:41:05.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.428 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.428 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-03-26T06:41:05.428 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-03-26T06:41:05.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T06:41:05.429 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.429 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.429 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.429 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:05.429 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.429 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.429 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-03-26T06:41:05.430 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-03-26T06:41:05.430 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.430 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.430 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.430 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T06:41:05.430 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.430 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.430 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-03-26T06:41:05.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.431 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-03-26T06:41:05.431 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-03-26T06:41:05.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T06:41:05.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.432 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:05.432 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.432 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.432 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.433 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-03-26T06:41:05.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-03-26T06:41:05.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.433 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.433 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T06:41:05.433 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.433 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.434 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.434 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-03-26T06:41:05.434 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.434 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.434 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.434 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-03-26T06:41:05.434 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-03-26T06:41:05.434 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.435 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T06:41:05.435 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.435 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.435 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.435 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-03-26T06:41:05.435 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.436 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-03-26T06:41:05.436 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-03-26T06:41:05.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.436 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.437 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.437 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.437 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T06:41:05.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.438 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-03-26T06:41:05.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-03-26T06:41:05.438 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.438 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.438 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.438 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:05.438 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.438 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.439 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.439 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T06:41:05.439 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.439 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.439 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-03-26T06:41:05.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T06:41:05.439 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.440 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.440 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-03-26T06:41:05.440 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.440 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.440 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.440 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T06:41:05.440 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.441 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.441 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-03-26T06:41:05.441 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-03-26T06:41:05.441 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.441 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.441 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.441 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-03-26T06:41:05.442 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.442 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.442 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.442 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T06:41:05.442 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.442 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.442 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.442 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-03-26T06:41:05.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-03-26T06:41:05.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.443 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.443 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-03-26T06:41:05.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.443 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-03-26T06:41:05.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.444 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.444 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.444 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-03-26T06:41:05.444 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-03-26T06:41:05.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.444 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.445 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.445 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.445 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.445 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.445 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T06:41:05.445 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.445 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.446 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-03-26T06:41:05.446 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-03-26T06:41:05.446 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.446 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.446 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.446 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:05.446 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.447 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.447 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.447 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-03-26T06:41:05.447 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.447 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.447 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-03-26T06:41:05.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-03-26T06:41:05.448 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.448 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.448 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:05.448 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.448 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.448 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-03-26T06:41:05.448 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.449 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.449 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-03-26T06:41:05.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-03-26T06:41:05.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.450 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.450 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T06:41:05.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.450 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-03-26T06:41:05.450 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-03-26T06:41:05.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.451 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.451 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T06:41:05.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.452 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.453 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.453 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.453 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.453 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.453 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.453 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.453 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.453 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.454 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.454 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.454 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-03-26T06:41:05.454 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-03-26T06:41:05.454 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.455 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.455 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.455 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.455 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.455 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T06:41:05.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.456 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.456 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.456 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.456 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.457 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.457 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T06:41:05.457 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.457 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.457 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.457 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.458 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.458 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.458 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.458 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.459 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.459 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.459 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.459 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.459 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.459 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.460 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.460 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.460 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.460 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.461 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.461 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T06:41:05.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T06:41:05.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.462 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.462 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T06:41:05.462 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.463 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.463 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T06:41:05.463 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T06:41:05.463 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.463 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.463 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.463 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-03-26T06:41:05.463 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.464 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.464 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T06:41:05.464 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.464 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-03-26T06:41:05.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-03-26T06:41:05.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.465 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.465 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.465 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:05.465 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-03-26T06:41:05.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.466 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.466 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.466 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-03-26T06:41:05.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-03-26T06:41:05.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.466 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:05.467 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.467 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.467 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T06:41:05.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.468 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-03-26T06:41:05.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-03-26T06:41:05.468 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.468 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.468 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.468 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.468 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.468 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.469 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-03-26T06:41:05.469 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.469 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.469 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.469 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-26T06:41:05.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-26T06:41:05.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.470 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-03-26T06:41:05.470 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.470 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.470 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.470 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-03-26T06:41:05.470 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-03-26T06:41:05.470 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T06:41:05.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.471 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.471 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.471 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-03-26T06:41:05.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-03-26T06:41:05.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.472 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-03-26T06:41:05.472 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.472 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T06:41:05.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.472 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-03-26T06:41:05.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-03-26T06:41:05.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T06:41:05.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.473 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T06:41:05.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.474 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-03-26T06:41:05.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-03-26T06:41:05.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.475 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.475 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.475 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T06:41:05.475 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.476 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-03-26T06:41:05.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-03-26T06:41:05.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T06:41:05.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.477 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.477 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.477 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-03-26T06:41:05.477 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-03-26T06:41:05.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.478 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.478 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.478 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.478 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.478 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.478 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T06:41:05.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.479 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.479 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-03-26T06:41:05.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-03-26T06:41:05.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.480 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.480 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.480 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T06:41:05.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.480 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-03-26T06:41:05.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-03-26T06:41:05.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T06:41:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-03-26T06:41:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-03-26T06:41:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.482 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-03-26T06:41:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-03-26T06:41:05.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.484 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.484 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.484 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-03-26T06:41:05.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-03-26T06:41:05.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.485 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.485 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.485 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T06:41:05.485 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.485 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.486 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-03-26T06:41:05.486 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-03-26T06:41:05.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.486 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T06:41:05.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.487 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-03-26T06:41:05.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-03-26T06:41:05.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.488 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.488 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.488 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T06:41:05.488 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.488 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.488 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.489 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-03-26T06:41:05.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-03-26T06:41:05.489 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.489 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.489 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.489 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.489 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.490 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T06:41:05.490 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.490 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.490 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.490 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-03-26T06:41:05.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-03-26T06:41:05.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.491 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.491 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.491 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.491 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T06:41:05.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.492 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.492 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-03-26T06:41:05.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-03-26T06:41:05.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.493 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.493 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T06:41:05.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.493 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.493 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-03-26T06:41:05.494 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.494 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.494 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.494 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.494 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.494 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.495 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.495 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.495 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.495 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.495 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.495 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.495 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.495 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.496 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.496 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.496 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.497 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.497 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.497 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.497 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.497 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.497 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.498 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.498 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.498 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.499 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.499 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.499 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.499 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.499 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.499 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.499 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.500 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.500 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.500 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.500 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.500 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.501 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.501 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.501 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.501 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.502 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.502 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.502 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-03-26T06:41:05.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-03-26T06:41:05.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.503 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-03-26T06:41:05.503 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.503 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.503 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.503 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T06:41:05.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.504 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.504 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.504 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-03-26T06:41:05.504 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-03-26T06:41:05.504 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.504 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.504 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.504 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-03-26T06:41:05.505 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.505 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.505 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T06:41:05.505 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.505 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.505 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.506 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.506 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-03-26T06:41:05.506 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.506 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.506 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.506 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T06:41:05.506 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T06:41:05.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.507 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:05.507 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.507 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.507 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.507 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.507 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.508 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T06:41:05.508 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T06:41:05.508 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.508 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.508 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.509 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.509 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.509 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.509 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.509 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.510 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T06:41:05.510 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T06:41:05.510 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.510 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.510 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.510 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.510 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.510 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.510 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.511 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.511 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.511 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.511 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-03-26T06:41:05.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T06:41:05.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.512 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.513 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T06:41:05.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T06:41:05.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.513 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.513 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.513 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.513 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.514 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.514 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.514 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.514 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.514 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T06:41:05.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T06:41:05.514 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.515 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.515 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.515 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.515 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.515 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.516 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T06:41:05.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T06:41:05.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.516 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.517 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.517 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T06:41:05.517 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T06:41:05.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.518 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.518 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.518 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.519 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T06:41:05.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T06:41:05.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.519 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.520 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.520 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.520 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.520 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.520 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.520 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.520 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T06:41:05.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T06:41:05.521 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.521 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.521 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.521 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.521 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.522 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T06:41:05.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T06:41:05.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.523 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.523 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T06:41:05.523 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T06:41:05.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.524 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.524 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.525 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T06:41:05.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T06:41:05.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.525 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.526 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.526 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.526 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.526 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.527 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T06:41:05.527 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T06:41:05.527 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.527 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.527 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.528 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.528 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.528 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.528 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T06:41:05.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T06:41:05.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.529 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.529 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-26T06:41:05.529 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.530 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T06:41:05.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T06:41:05.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.531 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.531 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.531 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-03-26T06:41:05.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-03-26T06:41:05.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.532 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.532 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.532 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.532 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.532 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.533 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T06:41:05.533 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T06:41:05.533 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.533 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.533 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.533 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-26T06:41:05.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.534 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.534 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.534 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.534 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.534 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T06:41:05.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T06:41:05.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.535 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.536 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T06:41:05.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T06:41:05.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.536 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.536 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.536 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.537 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.537 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T06:41:05.538 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T06:41:05.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.538 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.538 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.538 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.538 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.538 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T06:41:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T06:41:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T06:41:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T06:41:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-03-26T06:41:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T06:41:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T06:41:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.543 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.543 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.543 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.543 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.544 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T06:41:05.544 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T06:41:05.544 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.544 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.544 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T06:41:05.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T06:41:05.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.546 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.546 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.546 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.546 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.547 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.547 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.547 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T06:41:05.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T06:41:05.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.548 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.548 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.548 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.548 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T06:41:05.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T06:41:05.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.550 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T06:41:05.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T06:41:05.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.551 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.551 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.551 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.551 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.552 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T06:41:05.552 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T06:41:05.552 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.552 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.552 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.552 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.552 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.552 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.552 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.553 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.553 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.553 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.553 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.553 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T06:41:05.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T06:41:05.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.554 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.554 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.554 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.554 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.555 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T06:41:05.555 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T06:41:05.555 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.555 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.556 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.556 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.556 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.556 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.556 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.556 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T06:41:05.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T06:41:05.557 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.557 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.557 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.557 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.557 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.557 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.557 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.558 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.558 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T06:41:05.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T06:41:05.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.559 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T06:41:05.559 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.559 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.560 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T06:41:05.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T06:41:05.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:05.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.561 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T06:41:05.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T06:41:05.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.562 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-03-26T06:41:05.562 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.562 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.563 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T06:41:05.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T06:41:05.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.564 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.564 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-03-26T06:41:05.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.565 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T06:41:05.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T06:41:05.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.566 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.566 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.566 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.566 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.567 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T06:41:05.567 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T06:41:05.567 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.567 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.567 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-26T06:41:05.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.568 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T06:41:05.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T06:41:05.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.569 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.569 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.569 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.569 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.569 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.569 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.569 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.569 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.570 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T06:41:05.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T06:41:05.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.570 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.571 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.571 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T06:41:05.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T06:41:05.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.573 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T06:41:05.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T06:41:05.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:05.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.574 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.574 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.574 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.574 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.574 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.574 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T06:41:05.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T06:41:05.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.575 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.575 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:05.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.576 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T06:41:05.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T06:41:05.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:05.577 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.577 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.577 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.577 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-03-26T06:41:05.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-03-26T06:41:05.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.578 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.578 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.578 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.578 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.579 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.579 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.579 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.579 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-03-26T06:41:05.579 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-03-26T06:41:05.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.580 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.580 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.580 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.580 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.581 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-03-26T06:41:05.581 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-03-26T06:41:05.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.582 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-03-26T06:41:05.582 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.582 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-03-26T06:41:05.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-03-26T06:41:05.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.583 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-03-26T06:41:05.583 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.584 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-26T06:41:05.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-26T06:41:05.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.584 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.584 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.584 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.585 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.585 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.585 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.585 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.586 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.586 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.586 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.586 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.586 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.586 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.587 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.587 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.587 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.587 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.587 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.587 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.587 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.588 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.588 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.588 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.588 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.588 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.588 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.588 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.589 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.589 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.589 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.589 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.590 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.590 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.590 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.590 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.590 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.590 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.590 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.591 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.591 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.591 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.591 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.591 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.592 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.592 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.592 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.592 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.592 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.592 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.592 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.593 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.593 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.593 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.593 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-03-26T06:41:05.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-03-26T06:41:05.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.593 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-03-26T06:41:05.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.594 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.595 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-03-26T06:41:05.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T06:41:05.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.595 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.595 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.596 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.596 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-26T06:41:05.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-26T06:41:05.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-26T06:41:05.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.597 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.598 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-26T06:41:05.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-26T06:41:05.598 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.598 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.598 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-03-26T06:41:05.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.599 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.600 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-26T06:41:05.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-26T06:41:05.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.601 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-03-26T06:41:05.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-03-26T06:41:05.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-03-26T06:41:05.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.602 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.603 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-26T06:41:05.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-26T06:41:05.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.603 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-03-26T06:41:05.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.604 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.604 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-03-26T06:41:05.604 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.605 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-26T06:41:05.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-26T06:41:05.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-26T06:41:05.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-26T06:41:05.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-26T06:41:05.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.606 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-03-26T06:41:05.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-03-26T06:41:05.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-03-26T06:41:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.608 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-03-26T06:41:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-03-26T06:41:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.610 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.610 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-03-26T06:41:05.610 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.610 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.610 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.611 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.611 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.611 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.611 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.611 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.611 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.611 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.611 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.611 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.612 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.612 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.612 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.612 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.613 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.613 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.613 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.613 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.614 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.614 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.615 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.615 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.615 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.615 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.615 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.615 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.615 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.616 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.616 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.616 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.616 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.616 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.616 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.617 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.617 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.617 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.617 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.617 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.617 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.618 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.618 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.618 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.618 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-03-26T06:41:05.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-03-26T06:41:05.618 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.618 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-03-26T06:41:05.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.619 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.619 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.619 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T06:41:05.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.619 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-03-26T06:41:05.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-03-26T06:41:05.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.620 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-03-26T06:41:05.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T06:41:05.621 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.621 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-03-26T06:41:05.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-03-26T06:41:05.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.622 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-03-26T06:41:05.622 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.622 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.622 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.622 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-03-26T06:41:05.622 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.622 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.622 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.622 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-03-26T06:41:05.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-03-26T06:41:05.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.623 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.623 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-26T06:41:05.623 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.623 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.623 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T06:41:05.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.624 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.624 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-03-26T06:41:05.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-03-26T06:41:05.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.625 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.625 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.625 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.625 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.625 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.625 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T06:41:05.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.626 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:05.626 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:05.626 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.626 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.626 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:05.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.627 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.627 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.627 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T06:41:05.627 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.627 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.627 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.627 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-03-26T06:41:05.627 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-03-26T06:41:05.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.628 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-03-26T06:41:05.628 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.628 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.628 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T06:41:05.628 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.629 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.629 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-03-26T06:41:05.629 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.629 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.630 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.630 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.631 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.631 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.631 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.631 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.631 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.631 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.631 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.631 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.631 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.632 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.632 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.632 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.632 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.632 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.633 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.633 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.633 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.634 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.634 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.634 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.635 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.635 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.635 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.635 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.635 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.635 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.635 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.635 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.636 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.636 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.636 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.637 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.637 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.637 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.637 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.638 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-03-26T06:41:05.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-03-26T06:41:05.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-26T06:41:05.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.639 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-03-26T06:41:05.639 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-03-26T06:41:05.639 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-03-26T06:41:05.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.640 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.640 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.640 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.641 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:05.641 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:05.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.641 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:05.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.642 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.642 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.642 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-03-26T06:41:05.643 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-03-26T06:41:05.643 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-03-26T06:41:05.643 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.643 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-03-26T06:41:05.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-03-26T06:41:05.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-03-26T06:41:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T06:41:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-03-26T06:41:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-03-26T06:41:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-03-26T06:41:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.646 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T06:41:05.646 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.646 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.646 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-03-26T06:41:05.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-03-26T06:41:05.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.647 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.647 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.647 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-03-26T06:41:05.647 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.648 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:05.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:05.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.648 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.648 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-03-26T06:41:05.648 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-03-26T06:41:05.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.649 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T06:41:05.649 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.649 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.649 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.649 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.650 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.650 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.650 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.650 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.651 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.651 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.651 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.651 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.651 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.651 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.651 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.652 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.652 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.652 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.653 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.654 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.654 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.654 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.654 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.655 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.655 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.655 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.655 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.655 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.656 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.656 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.656 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.656 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.656 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.657 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-03-26T06:41:05.657 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-03-26T06:41:05.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.657 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.657 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-03-26T06:41:05.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.658 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-03-26T06:41:05.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-03-26T06:41:05.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-03-26T06:41:05.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.659 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-03-26T06:41:05.660 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.660 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.660 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.660 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-26T06:41:05.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-26T06:41:05.660 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.660 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.660 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.661 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-03-26T06:41:05.661 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.661 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.661 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T06:41:05.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.661 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-03-26T06:41:05.662 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-03-26T06:41:05.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.662 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:05.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:05.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-03-26T06:41:05.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T06:41:05.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.663 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-03-26T06:41:05.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-03-26T06:41:05.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.664 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-03-26T06:41:05.665 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.666 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-03-26T06:41:05.666 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-03-26T06:41:05.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-03-26T06:41:05.667 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.667 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T06:41:05.667 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.668 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.668 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-03-26T06:41:05.668 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.668 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.668 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.669 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.669 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.669 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.670 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.670 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.670 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.670 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.670 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.670 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.670 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.671 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.672 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.672 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.673 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.673 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.673 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.673 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.674 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.674 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.674 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.674 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.674 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.674 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.674 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.675 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.675 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.675 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.675 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.676 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-03-26T06:41:05.677 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.677 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.677 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.677 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.678 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.678 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.678 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.679 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.679 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.679 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.679 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.679 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.679 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.680 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.681 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.681 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.681 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.681 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.682 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.682 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.682 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.682 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.683 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.683 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.683 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.683 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.683 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.683 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.684 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.685 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-03-26T06:41:05.686 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.686 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.686 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.686 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-03-26T06:41:05.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-03-26T06:41:05.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.687 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-03-26T06:41:05.687 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.687 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.687 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.687 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T06:41:05.688 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.688 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.688 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.688 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-03-26T06:41:05.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-03-26T06:41:05.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.689 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:05.689 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-03-26T06:41:05.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.689 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.689 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T06:41:05.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.689 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.689 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.690 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.690 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.691 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.691 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.691 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.691 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.691 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.691 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.691 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.691 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.692 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.692 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.692 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.692 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.693 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.694 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.694 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.695 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.695 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.695 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.695 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.695 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.695 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.695 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.696 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.696 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.696 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.697 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.697 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.697 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.697 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.697 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.697 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-03-26T06:41:05.698 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.698 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.698 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.698 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-03-26T06:41:05.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-03-26T06:41:05.698 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.698 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-03-26T06:41:05.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.699 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-03-26T06:41:05.699 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T06:41:05.699 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.699 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.701 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.701 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.701 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.701 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.701 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.701 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.701 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.701 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.701 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.702 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.702 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.703 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.703 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.703 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.703 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.703 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.703 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.704 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.704 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.705 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.705 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.705 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.705 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.705 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.705 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.705 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.706 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.706 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.706 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.707 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.707 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.707 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.707 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-03-26T06:41:05.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-03-26T06:41:05.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.708 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.708 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.709 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-03-26T06:41:05.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T06:41:05.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.709 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.709 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-03-26T06:41:05.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-03-26T06:41:05.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.710 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-03-26T06:41:05.710 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.710 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.710 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-03-26T06:41:05.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.711 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-03-26T06:41:05.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-03-26T06:41:05.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.711 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-03-26T06:41:05.712 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.712 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.712 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.712 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T06:41:05.712 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.713 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.713 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-03-26T06:41:05.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-03-26T06:41:05.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.713 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.714 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.714 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.714 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-03-26T06:41:05.714 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.715 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-03-26T06:41:05.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-03-26T06:41:05.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.715 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-03-26T06:41:05.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.716 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T06:41:05.716 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T06:41:05.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.716 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.716 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.716 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-03-26T06:41:05.717 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.717 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.717 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.717 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.717 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.717 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.717 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.718 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.718 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.718 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.718 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.718 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.719 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.719 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.719 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.719 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.719 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.720 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.720 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.720 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.720 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.721 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.721 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.721 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.721 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.721 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.722 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.722 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.722 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.722 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.722 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.722 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.722 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.723 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.723 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.723 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.723 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.723 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.724 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.724 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.724 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.724 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-03-26T06:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.726 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.726 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.726 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.726 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.726 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.726 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.727 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.727 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.727 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.727 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.727 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.727 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.727 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.727 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.728 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.728 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.728 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.728 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.729 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.729 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.729 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.730 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.730 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.730 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.730 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.730 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.730 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.731 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.731 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.731 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.731 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.731 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.731 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.731 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.731 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.732 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.732 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.732 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.733 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.733 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-03-26T06:41:05.734 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.734 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.734 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.734 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.735 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.735 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.735 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.735 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.736 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.736 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.736 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.736 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.736 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.736 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.736 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.736 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.736 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.737 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.737 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.737 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.737 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.737 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.738 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.738 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.738 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.738 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.738 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.739 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.739 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.739 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.740 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.740 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.740 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.740 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.740 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.740 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.740 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.741 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.741 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.741 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.741 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.741 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.741 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.742 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.742 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.742 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.742 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-26T06:41:05.743 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.743 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.743 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.743 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-03-26T06:41:05.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-03-26T06:41:05.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.744 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T06:41:05.744 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.744 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.745 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.745 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.745 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.745 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.745 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.746 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.746 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.746 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.746 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.746 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.746 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.746 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.746 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.747 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.747 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.747 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.747 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.748 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.748 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.748 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.748 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.749 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.749 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.749 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.749 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.750 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.750 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.750 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.750 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.750 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.750 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.750 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.750 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.750 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.751 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-03-26T06:41:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-26T06:41:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.753 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.754 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.754 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.754 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-03-26T06:41:05.754 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-03-26T06:41:05.754 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-03-26T06:41:05.754 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.754 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-03-26T06:41:05.754 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.755 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.755 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.755 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-03-26T06:41:05.755 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.755 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.756 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.756 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.756 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.757 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.757 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.757 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.757 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.757 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.757 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.757 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.758 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.758 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.758 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.758 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.759 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.759 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.759 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.759 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.760 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.760 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.760 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.760 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.760 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.760 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.760 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.761 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.761 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.761 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.761 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.761 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.761 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.761 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.761 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.761 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.762 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.762 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.762 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.763 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.763 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.763 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.763 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.763 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-03-26T06:41:05.764 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.764 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.764 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.764 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.765 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.765 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.765 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.766 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.766 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.766 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.766 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.766 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.766 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.767 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.767 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.767 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.767 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.767 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.767 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.770 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.770 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.770 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.770 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.770 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.770 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.770 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.770 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.771 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.771 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.771 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.771 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.771 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.771 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.771 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.772 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.772 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.772 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.772 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.772 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-03-26T06:41:05.772 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.773 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.773 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.773 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.773 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.773 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.773 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.773 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.773 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.773 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.773 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.774 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.774 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.774 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.774 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.774 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.774 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.774 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.774 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.775 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.775 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.775 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.775 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.776 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.776 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.776 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.776 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.777 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.777 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.778 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.778 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.778 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.778 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.778 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.778 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.778 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.778 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.779 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.779 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.780 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.780 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.780 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.780 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.780 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.781 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:05.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:05.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.781 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.781 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:05.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.782 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.782 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.782 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.782 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.782 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-03-26T06:41:05.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-03-26T06:41:05.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:05.782 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.783 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-26T06:41:05.783 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-03-26T06:41:05.783 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.783 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.783 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-03-26T06:41:05.783 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T06:41:05.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.783 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.784 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-03-26T06:41:05.784 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.784 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.784 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.784 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-03-26T06:41:05.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-03-26T06:41:05.785 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.785 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.785 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.785 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T06:41:05.785 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.786 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.786 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-26T06:41:05.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-26T06:41:05.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.786 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.787 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.787 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.787 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.787 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.787 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.788 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.788 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.789 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.789 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.789 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.789 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.789 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.789 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.789 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.789 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.790 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.790 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.790 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.790 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.790 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.790 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.790 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.791 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.791 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.791 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.791 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.791 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.791 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.791 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.791 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.791 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.792 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.792 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.792 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.792 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.792 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.792 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.793 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.793 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.793 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.793 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.793 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.793 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.794 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.794 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.795 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.795 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.795 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.795 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.795 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-03-26T06:41:05.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-03-26T06:41:05.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-03-26T06:41:05.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-03-26T06:41:05.797 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.797 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:05.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:05.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.797 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.798 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.798 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:05.798 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.798 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.798 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-03-26T06:41:05.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.799 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.799 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.799 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-03-26T06:41:05.799 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.799 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.800 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.800 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-03-26T06:41:05.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-03-26T06:41:05.800 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.800 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.800 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.800 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.800 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.800 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.801 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.801 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.801 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.801 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.801 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.801 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-03-26T06:41:05.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-03-26T06:41:05.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.802 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.802 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.802 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.802 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.802 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T06:41:05.803 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T06:41:05.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.803 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.804 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.804 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.804 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.804 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.804 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.804 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.805 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.805 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.805 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.805 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.805 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.805 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.806 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.806 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.806 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.806 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.807 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.807 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.808 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.808 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.808 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.808 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.809 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.809 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.809 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.809 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.809 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.809 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.809 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.810 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.810 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.810 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.810 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.811 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.811 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-03-26T06:41:05.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-03-26T06:41:05.811 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.811 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.812 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.812 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.812 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.812 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.812 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.812 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.812 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T06:41:05.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T06:41:05.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.813 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-03-26T06:41:05.813 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.813 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.813 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.813 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.814 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.814 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.814 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-03-26T06:41:05.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-03-26T06:41:05.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:05.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.815 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-03-26T06:41:05.815 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.815 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.816 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-26T06:41:05.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-26T06:41:05.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.816 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-26T06:41:05.816 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.816 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T06:41:05.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.817 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-26T06:41:05.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-26T06:41:05.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.817 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-03-26T06:41:05.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.818 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.818 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T06:41:05.818 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.818 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.819 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-03-26T06:41:05.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-03-26T06:41:05.819 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.819 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.819 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-03-26T06:41:05.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.819 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.820 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.820 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.820 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.820 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-26T06:41:05.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-26T06:41:05.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.821 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-03-26T06:41:05.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-03-26T06:41:05.821 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-03-26T06:41:05.821 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.822 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.822 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.822 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.822 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-26T06:41:05.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-26T06:41:05.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.823 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.823 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-26T06:41:05.823 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-26T06:41:05.823 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-26T06:41:05.823 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.823 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.824 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.824 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.824 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.824 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-03-26T06:41:05.824 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.824 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.824 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.825 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.825 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.825 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.825 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.826 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.826 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.826 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.826 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.826 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.826 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.826 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.826 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.827 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.827 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.829 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.830 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.830 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.830 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.830 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.830 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.830 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.830 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.830 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.833 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-03-26T06:41:05.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-03-26T06:41:05.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.833 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:05.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.834 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.834 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.834 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.834 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.834 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.834 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-03-26T06:41:05.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T06:41:05.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.835 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.835 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.835 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.835 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.835 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.835 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.836 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.836 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-03-26T06:41:05.836 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-03-26T06:41:05.836 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.837 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.837 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.837 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.837 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.837 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.838 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.838 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.838 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-03-26T06:41:05.838 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.838 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.839 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.839 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-03-26T06:41:05.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-03-26T06:41:05.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.839 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.840 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.840 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.840 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.840 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.840 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-26T06:41:05.840 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-26T06:41:05.840 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.841 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.841 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.841 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.841 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.841 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.841 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.842 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.842 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.842 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.842 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.843 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.843 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.843 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.843 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.843 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.843 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.844 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.844 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.844 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.844 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.844 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.845 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.845 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.845 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.845 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.846 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.846 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.846 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.846 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.846 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.846 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.847 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.847 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.847 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.847 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.847 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.847 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.847 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.847 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.848 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.848 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.848 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.848 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.849 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.849 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.849 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.849 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.849 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.849 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.849 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.850 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-26T06:41:05.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-26T06:41:05.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.850 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.850 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.850 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.851 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.851 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.851 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.851 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.851 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-26T06:41:05.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-26T06:41:05.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.852 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.852 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.852 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.852 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.852 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.853 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.853 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.853 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.853 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.853 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-03-26T06:41:05.853 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.854 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.854 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.854 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.854 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.855 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.855 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.855 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.855 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.855 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.855 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.855 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.856 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.856 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.856 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.856 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.857 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.857 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.857 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.857 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.857 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.857 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.857 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.858 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.858 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.858 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.859 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.859 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.859 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.859 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.859 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.859 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.859 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.859 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.860 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.860 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.860 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.860 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.861 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.861 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.861 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.861 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.861 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-03-26T06:41:05.862 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.862 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.862 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.862 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.863 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.863 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.863 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.863 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.863 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.863 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.863 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.863 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.864 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.864 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.864 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.864 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.864 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.864 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.864 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.864 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.864 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.865 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.865 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.865 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.865 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.866 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.866 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.866 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.867 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.867 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.867 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.867 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.867 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.868 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.868 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.868 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.868 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.868 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.869 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.870 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.870 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.870 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-03-26T06:41:05.870 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-03-26T06:41:05.870 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.870 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.870 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.871 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-03-26T06:41:05.874 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.874 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T06:41:05.874 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.874 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.874 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.874 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.875 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-03-26T06:41:05.875 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.875 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.875 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.875 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.876 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.876 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.876 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.877 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.877 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.877 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.877 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.877 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.877 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.877 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.877 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.877 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.878 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.878 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.878 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.878 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.878 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.878 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.879 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.879 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.879 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.879 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.879 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.880 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.880 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.880 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.880 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.880 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.880 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.880 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.880 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.880 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.881 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.881 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.881 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.882 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.882 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.883 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.883 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.883 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.883 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-03-26T06:41:05.883 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-03-26T06:41:05.883 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.883 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.883 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.884 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.884 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.884 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.884 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.884 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.884 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.885 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-03-26T06:41:05.885 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-03-26T06:41:05.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.885 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.885 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-03-26T06:41:05.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.885 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.886 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.886 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.886 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-03-26T06:41:05.886 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-03-26T06:41:05.887 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.887 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.887 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.887 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.887 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.887 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.888 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-03-26T06:41:05.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-03-26T06:41:05.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.888 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.888 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.889 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.889 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.889 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.889 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.889 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-03-26T06:41:05.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-03-26T06:41:05.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:05.890 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.890 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.890 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.890 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.890 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.891 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.891 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-03-26T06:41:05.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-03-26T06:41:05.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.891 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.892 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.892 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.892 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.892 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.893 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-03-26T06:41:05.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-03-26T06:41:05.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.893 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.893 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.894 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.894 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-03-26T06:41:05.894 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-03-26T06:41:05.894 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.895 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.895 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.895 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.895 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.895 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.895 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.895 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.896 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-03-26T06:41:05.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-03-26T06:41:05.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.896 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.896 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.896 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.896 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:05.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-03-26T06:41:05.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.897 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.897 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-03-26T06:41:05.897 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-03-26T06:41:05.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.898 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.898 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.898 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.898 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.899 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.899 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.899 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-03-26T06:41:05.899 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-03-26T06:41:05.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.899 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.900 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.900 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.900 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.900 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.900 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.900 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.901 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-03-26T06:41:05.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-03-26T06:41:05.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.901 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.901 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.902 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.902 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.902 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.902 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.902 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-03-26T06:41:05.903 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.903 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.903 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.903 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-03-26T06:41:05.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-03-26T06:41:05.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T06:41:05.904 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.904 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.905 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-03-26T06:41:05.905 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-03-26T06:41:05.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.905 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.905 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.906 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-03-26T06:41:05.906 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.906 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.907 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.907 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.907 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.907 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.908 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.908 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.908 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.908 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.908 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.908 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.908 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.908 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.908 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.909 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.909 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.909 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.909 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.909 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.910 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.910 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.910 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.910 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.910 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.910 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.912 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.912 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.912 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.912 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.912 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.912 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.912 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.913 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.913 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.913 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.913 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.913 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.914 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.914 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.914 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.914 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-03-26T06:41:05.915 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.915 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.915 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.915 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.915 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.915 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.916 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.916 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.916 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.916 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.916 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.916 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.916 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.916 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.916 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.916 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.917 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.917 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.917 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.917 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.917 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.918 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.918 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.918 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.918 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.918 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.918 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.918 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.918 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.919 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.919 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.919 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.919 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.920 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.920 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.920 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.920 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.920 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.920 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.920 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.920 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.920 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.921 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.921 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.921 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.921 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.921 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.922 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.922 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.922 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.922 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.922 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.922 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-03-26T06:41:05.923 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.923 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.923 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.923 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.924 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.924 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.924 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.924 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.924 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.925 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.925 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.925 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.925 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.925 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.925 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.925 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.925 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.925 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.926 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.926 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.926 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.926 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.926 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.926 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.926 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.927 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.927 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.927 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.927 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.927 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.927 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.928 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.928 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.928 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.928 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.928 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.929 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.929 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.929 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.929 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.929 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.929 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.929 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.929 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.929 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.930 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.930 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.930 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.930 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.930 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.930 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.931 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.931 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.931 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.931 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.931 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.931 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-03-26T06:41:05.932 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.932 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.932 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.932 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-03-26T06:41:05.932 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-03-26T06:41:05.932 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.932 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.932 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.932 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T06:41:05.933 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.933 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.933 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.933 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.933 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.933 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.933 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.933 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:05.934 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:05.934 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:05.934 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.934 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.934 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-03-26T06:41:05.934 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.935 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.935 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.935 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.935 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.935 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.936 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.936 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.936 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.936 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.936 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.936 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.936 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.937 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.937 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.937 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.937 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.938 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.938 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.938 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.938 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.938 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.939 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.939 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.939 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.940 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.940 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.940 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.940 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.940 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.940 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.941 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.941 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.941 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.941 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.941 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.941 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.941 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.941 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-03-26T06:41:05.944 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.944 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.944 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.944 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-03-26T06:41:05.944 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-03-26T06:41:05.944 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.944 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.944 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.944 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.945 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.945 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.945 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.945 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T06:41:05.945 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.945 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.945 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.945 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-03-26T06:41:05.946 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-03-26T06:41:05.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.946 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.946 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.946 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.946 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.946 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.947 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T06:41:05.947 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.947 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.947 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.947 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-03-26T06:41:05.947 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-03-26T06:41:05.947 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.948 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:05.948 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.948 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.948 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.948 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T06:41:05.948 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.948 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.949 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-03-26T06:41:05.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-03-26T06:41:05.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.949 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.949 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.949 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.949 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.950 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.950 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T06:41:05.950 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.950 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.950 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-03-26T06:41:05.950 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-03-26T06:41:05.950 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.951 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.951 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.951 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.951 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.951 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.951 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.951 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T06:41:05.951 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.951 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.952 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.952 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-03-26T06:41:05.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-03-26T06:41:05.952 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.952 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.952 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.952 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.952 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.953 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.953 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.953 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.953 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.953 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-03-26T06:41:05.953 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-03-26T06:41:05.953 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.954 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.954 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.954 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.954 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.954 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.954 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.954 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.954 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.955 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.955 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-03-26T06:41:05.955 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-03-26T06:41:05.955 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.955 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.955 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.955 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T06:41:05.956 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.956 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.956 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.956 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.956 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.956 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-26T06:41:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-26T06:41:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.959 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.959 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-03-26T06:41:05.959 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-03-26T06:41:05.959 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.959 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.959 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.960 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.960 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-03-26T06:41:05.960 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-03-26T06:41:05.960 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.960 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.960 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-03-26T06:41:05.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.961 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.961 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.961 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.961 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.961 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.961 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.961 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-03-26T06:41:05.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-03-26T06:41:05.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.962 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.962 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.962 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.962 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.962 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.962 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.962 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.963 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.963 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-03-26T06:41:05.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-03-26T06:41:05.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.963 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.963 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.964 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.964 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.964 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.964 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.964 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.964 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.964 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.964 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.965 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.965 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.965 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.965 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.965 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.965 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.966 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.966 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.966 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.966 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.966 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.966 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.966 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.966 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.967 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.967 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.967 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.967 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.968 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.968 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.968 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.968 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.968 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.968 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.968 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.969 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.969 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.969 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.969 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.969 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.969 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.969 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.969 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.969 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.970 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.970 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.970 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.970 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.970 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.970 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.971 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.971 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.972 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.972 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.972 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.972 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.972 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.972 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-03-26T06:41:05.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-03-26T06:41:05.973 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.973 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.973 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.973 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.973 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.973 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.973 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.973 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.973 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.974 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.974 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-03-26T06:41:05.974 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-03-26T06:41:05.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.975 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.975 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.975 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.975 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.975 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.976 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-03-26T06:41:05.976 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-03-26T06:41:05.976 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.976 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.976 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T06:41:05.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.977 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.977 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.977 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.977 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.977 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-03-26T06:41:05.978 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.978 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.978 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.978 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.978 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.978 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.979 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.979 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.979 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.979 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.979 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.979 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.979 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.979 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.980 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.980 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.980 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.980 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.980 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.980 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.980 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.980 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.981 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.981 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.981 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.981 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.981 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.981 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.981 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.982 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.982 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.982 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.982 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.982 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.983 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.983 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.983 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.983 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.983 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.983 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.983 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.983 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.983 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.984 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.984 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.984 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.984 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.984 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.984 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.985 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.985 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.985 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.985 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.985 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.986 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.986 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.986 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:05.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:05.986 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-03-26T06:41:05.986 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:05.986 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:05.987 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:05.987 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:05.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:05.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:05.987 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.988 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.988 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.988 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.988 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.988 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.988 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.988 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.989 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.989 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.989 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.989 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.989 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:05.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:05.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.989 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.990 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.990 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.990 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.990 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.990 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.990 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.990 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.991 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:05.991 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:05.991 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:05.991 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:05.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.992 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:05.992 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:05.992 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:05.992 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:05.992 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:05.992 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:05.992 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:05.992 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:05.992 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.992 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.993 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:05.993 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:05.993 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.993 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.993 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:05.993 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.994 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.994 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.994 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.994 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:05.994 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.994 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.994 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.994 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-03-26T06:41:05.995 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-03-26T06:41:05.995 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.995 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.995 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.995 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-03-26T06:41:05.995 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.995 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.995 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.995 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T06:41:05.996 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.996 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.996 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.996 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-03-26T06:41:05.996 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-03-26T06:41:05.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.996 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.997 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:05.997 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.997 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.997 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.997 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-03-26T06:41:05.997 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.997 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.998 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-03-26T06:41:05.998 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-03-26T06:41:05.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:05.998 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:05.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:05.998 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:05.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:05.998 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:05.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:05.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T06:41:05.999 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:05.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:05.999 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:05.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:05.999 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-03-26T06:41:05.999 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T06:41:05.999 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:05.999 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.000 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.000 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.000 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.000 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.000 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T06:41:06.000 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.000 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.001 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.001 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.001 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.001 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-03-26T06:41:06.001 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.001 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.001 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.002 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-03-26T06:41:06.002 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-03-26T06:41:06.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:06.002 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.002 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-03-26T06:41:06.002 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.002 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.003 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.003 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.003 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.003 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-03-26T06:41:06.003 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-03-26T06:41:06.003 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.003 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.004 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.004 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.004 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.004 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.004 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.005 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:06.005 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:06.005 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.005 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.005 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.005 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:06.005 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.006 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.006 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.006 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.006 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.006 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.006 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.006 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.007 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.007 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.007 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.007 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.007 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.007 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.008 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.008 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.008 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.008 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.008 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.008 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.008 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.008 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.008 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.009 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.009 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.009 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.009 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.010 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.010 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.010 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.010 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.010 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.010 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.011 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.011 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.011 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.011 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.012 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.012 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.012 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.012 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.012 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.012 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.012 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.013 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.013 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.013 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.013 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.013 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.014 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.014 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.014 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.014 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-03-26T06:41:06.015 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-03-26T06:41:06.015 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.015 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.015 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.016 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.016 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.016 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.016 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-03-26T06:41:06.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-03-26T06:41:06.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.016 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.016 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.017 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-03-26T06:41:06.017 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.017 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.017 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.017 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.017 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.018 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-03-26T06:41:06.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-03-26T06:41:06.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.018 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.018 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.018 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T06:41:06.018 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.018 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.018 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.019 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.019 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.019 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.019 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:41:06.019 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.020 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.020 INFO:teuthology.orchestra.run.smithi037.stdout: "modules": [ 2024-03-26T06:41:06.020 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-03-26T06:41:06.020 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard", 2024-03-26T06:41:06.020 INFO:teuthology.orchestra.run.smithi037.stdout: "iostat", 2024-03-26T06:41:06.020 INFO:teuthology.orchestra.run.smithi037.stdout: "nfs", 2024-03-26T06:41:06.020 INFO:teuthology.orchestra.run.smithi037.stdout: "restful", 2024-03-26T06:41:06.020 INFO:teuthology.orchestra.run.smithi037.stdout: "snap_schedule" 2024-03-26T06:41:06.020 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.021 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-03-26T06:41:06.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-03-26T06:41:06.021 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.021 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.021 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.021 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:06.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:06.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:06.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.022 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.022 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.022 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T06:41:06.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.022 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.023 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.023 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.023 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.023 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.023 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.023 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.023 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.024 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.024 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.024 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.024 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.024 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.024 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.024 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.024 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.024 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.025 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.025 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.025 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.026 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.026 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.026 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.026 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.027 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.027 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.027 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.028 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.029 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.029 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.029 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.029 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.029 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.029 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.029 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.030 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.030 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.030 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.030 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.030 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.031 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.031 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-03-26T06:41:06.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-03-26T06:41:06.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.031 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.031 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.032 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.032 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.032 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-03-26T06:41:06.032 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.032 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.032 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-03-26T06:41:06.032 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-03-26T06:41:06.033 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.033 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.033 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.033 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-03-26T06:41:06.033 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.033 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.033 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.033 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-03-26T06:41:06.034 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.034 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.034 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.034 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-03-26T06:41:06.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-03-26T06:41:06.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.034 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.035 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.035 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.035 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.035 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-03-26T06:41:06.035 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.035 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.036 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-03-26T06:41:06.036 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-03-26T06:41:06.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.036 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.036 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.036 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.037 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.037 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.037 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-03-26T06:41:06.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.037 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.038 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-03-26T06:41:06.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-03-26T06:41:06.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-03-26T06:41:06.038 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.038 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.039 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-03-26T06:41:06.039 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.039 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.039 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.039 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-03-26T06:41:06.039 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-03-26T06:41:06.040 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.040 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.040 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.040 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.040 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.040 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.040 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.040 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-03-26T06:41:06.041 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.041 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.041 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.041 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-03-26T06:41:06.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-03-26T06:41:06.041 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.041 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.042 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.042 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.042 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.042 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.042 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T06:41:06.042 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.042 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.043 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.043 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-03-26T06:41:06.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-03-26T06:41:06.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.043 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.043 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.043 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.044 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.044 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.044 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-03-26T06:41:06.044 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.044 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.044 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.044 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.044 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-03-26T06:41:06.045 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.045 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.045 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.045 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-03-26T06:41:06.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-03-26T06:41:06.046 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.046 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.046 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.046 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.046 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.046 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.046 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.046 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T06:41:06.047 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.047 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.047 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.047 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-03-26T06:41:06.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-03-26T06:41:06.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.048 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.048 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-03-26T06:41:06.048 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.048 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.048 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T06:41:06.048 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T06:41:06.048 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.048 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.049 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-03-26T06:41:06.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-03-26T06:41:06.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:06.049 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.049 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.049 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:06.049 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-26T06:41:06.049 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-26T06:41:06.049 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T06:41:06.050 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T06:41:06.050 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.050 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.050 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-03-26T06:41:06.050 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-03-26T06:41:06.050 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:06.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.051 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-03-26T06:41:06.051 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:06.051 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-03-26T06:41:06.051 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.051 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T06:41:06.051 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.051 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.051 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.052 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-03-26T06:41:06.052 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-03-26T06:41:06.052 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.052 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.052 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.052 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T06:41:06.052 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.052 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.053 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.053 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T06:41:06.053 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T06:41:06.053 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.053 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.053 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-03-26T06:41:06.053 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-03-26T06:41:06.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:06.054 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.054 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.054 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-03-26T06:41:06.054 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-03-26T06:41:06.054 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-03-26T06:41:06.054 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-03-26T06:41:06.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T06:41:06.055 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.055 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.055 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-03-26T06:41:06.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-03-26T06:41:06.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.055 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.055 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-03-26T06:41:06.056 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.056 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.056 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.056 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T06:41:06.056 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T06:41:06.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.056 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-03-26T06:41:06.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-03-26T06:41:06.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:06.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:06.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-03-26T06:41:06.057 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-03-26T06:41:06.057 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.057 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T06:41:06.058 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T06:41:06.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.058 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.058 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.058 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.058 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.058 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.059 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.059 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.059 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.059 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.059 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.059 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.059 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.060 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.060 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.060 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.060 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.060 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.061 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.061 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.061 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.061 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.061 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.062 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.062 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.062 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.062 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.062 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.063 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.063 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.063 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.063 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.063 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.063 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.063 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.063 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.063 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.063 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.064 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.064 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.064 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.064 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.064 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.064 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.064 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.065 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.065 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.065 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.065 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.065 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.065 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.065 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.066 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.066 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.066 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-03-26T06:41:06.066 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-03-26T06:41:06.066 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:06.066 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.066 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.067 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:06.067 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.067 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.067 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.067 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T06:41:06.067 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.067 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.068 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-26T06:41:06.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-26T06:41:06.068 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.068 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.068 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-03-26T06:41:06.068 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.068 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.069 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-03-26T06:41:06.069 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-03-26T06:41:06.069 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-03-26T06:41:06.069 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-03-26T06:41:06.069 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-03-26T06:41:06.069 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.069 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-03-26T06:41:06.069 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.069 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.070 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-03-26T06:41:06.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-03-26T06:41:06.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.070 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.070 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.070 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.071 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.071 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T06:41:06.071 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.071 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.071 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.071 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:06.072 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:06.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.072 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:06.072 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.072 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.072 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.072 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T06:41:06.072 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.073 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.073 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.073 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-03-26T06:41:06.073 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-03-26T06:41:06.073 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.073 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.073 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.073 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:06.074 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:06.074 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.074 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.074 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T06:41:06.074 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T06:41:06.074 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.074 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.075 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-03-26T06:41:06.075 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-03-26T06:41:06.075 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:06.075 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.075 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.075 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:06.075 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.076 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.076 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T06:41:06.076 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.076 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.076 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.076 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.076 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-03-26T06:41:06.077 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.077 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.077 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.077 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-03-26T06:41:06.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-03-26T06:41:06.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:06.077 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.077 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.078 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-26T06:41:06.078 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.078 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.078 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T06:41:06.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.078 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.079 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-03-26T06:41:06.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-03-26T06:41:06.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.079 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.079 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.079 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-03-26T06:41:06.079 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.079 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T06:41:06.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.080 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.080 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-03-26T06:41:06.080 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-03-26T06:41:06.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.081 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.081 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.081 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-03-26T06:41:06.081 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.081 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.081 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.081 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T06:41:06.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.082 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.082 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-03-26T06:41:06.082 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-03-26T06:41:06.082 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.083 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.083 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.083 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.083 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.083 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T06:41:06.083 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T06:41:06.083 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.084 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-03-26T06:41:06.084 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-03-26T06:41:06.084 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.084 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.084 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.084 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:06.084 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.085 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.085 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.085 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T06:41:06.085 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.085 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.085 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.085 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-03-26T06:41:06.086 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T06:41:06.086 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:06.086 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.086 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.086 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-03-26T06:41:06.086 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.086 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.086 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.087 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T06:41:06.087 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.087 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.087 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-03-26T06:41:06.087 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-03-26T06:41:06.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.088 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.088 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.088 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.088 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.088 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.088 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-03-26T06:41:06.088 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-03-26T06:41:06.089 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-03-26T06:41:06.089 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.089 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T06:41:06.089 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.089 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.089 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.089 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-03-26T06:41:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-03-26T06:41:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T06:41:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-03-26T06:41:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-03-26T06:41:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T06:41:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-03-26T06:41:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-03-26T06:41:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T06:41:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-03-26T06:41:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-03-26T06:41:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T06:41:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-03-26T06:41:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-03-26T06:41:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T06:41:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-03-26T06:41:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-03-26T06:41:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-03-26T06:41:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T06:41:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-03-26T06:41:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-03-26T06:41:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-03-26T06:41:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.100 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.100 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T06:41:06.100 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.100 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.100 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:06.100 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-03-26T06:41:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-03-26T06:41:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T06:41:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.102 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.102 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.102 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-03-26T06:41:06.102 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.102 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.102 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.102 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-03-26T06:41:06.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-03-26T06:41:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T06:41:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-03-26T06:41:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-03-26T06:41:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-03-26T06:41:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T06:41:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-03-26T06:41:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-03-26T06:41:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-03-26T06:41:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T06:41:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-03-26T06:41:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-03-26T06:41:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-03-26T06:41:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T06:41:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-03-26T06:41:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-03-26T06:41:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-03-26T06:41:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T06:41:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-03-26T06:41:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-03-26T06:41:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-03-26T06:41:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T06:41:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-03-26T06:41:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-03-26T06:41:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T06:41:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-03-26T06:41:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-03-26T06:41:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-03-26T06:41:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T06:41:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-03-26T06:41:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-03-26T06:41:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-03-26T06:41:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T06:41:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-03-26T06:41:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-03-26T06:41:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-03-26T06:41:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T06:41:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-03-26T06:41:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-03-26T06:41:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-03-26T06:41:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T06:41:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-03-26T06:41:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-03-26T06:41:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T06:41:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-03-26T06:41:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T06:41:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-03-26T06:41:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T06:41:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-03-26T06:41:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-03-26T06:41:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-03-26T06:41:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T06:41:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-03-26T06:41:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-03-26T06:41:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-03-26T06:41:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-03-26T06:41:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-03-26T06:41:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-03-26T06:41:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T06:41:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-03-26T06:41:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-03-26T06:41:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-03-26T06:41:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-03-26T06:41:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-03-26T06:41:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-03-26T06:41:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-03-26T06:41:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-03-26T06:41:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T06:41:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-03-26T06:41:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-03-26T06:41:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T06:41:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-03-26T06:41:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-03-26T06:41:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T06:41:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T06:41:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T06:41:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T06:41:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T06:41:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T06:41:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T06:41:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-03-26T06:41:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T06:41:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-03-26T06:41:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-03-26T06:41:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-03-26T06:41:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-03-26T06:41:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-03-26T06:41:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T06:41:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-03-26T06:41:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-03-26T06:41:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-03-26T06:41:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-03-26T06:41:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-03-26T06:41:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-03-26T06:41:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-03-26T06:41:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-03-26T06:41:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T06:41:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-03-26T06:41:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-03-26T06:41:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-03-26T06:41:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T06:41:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-03-26T06:41:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-03-26T06:41:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T06:41:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T06:41:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-03-26T06:41:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-03-26T06:41:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T06:41:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-03-26T06:41:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-03-26T06:41:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T06:41:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-03-26T06:41:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-03-26T06:41:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T06:41:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-03-26T06:41:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-03-26T06:41:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T06:41:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-03-26T06:41:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-03-26T06:41:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T06:41:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-03-26T06:41:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-03-26T06:41:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-03-26T06:41:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-03-26T06:41:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-03-26T06:41:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-03-26T06:41:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T06:41:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-03-26T06:41:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-03-26T06:41:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T06:41:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-03-26T06:41:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-03-26T06:41:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T06:41:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-03-26T06:41:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-03-26T06:41:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T06:41:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-03-26T06:41:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-03-26T06:41:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T06:41:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-03-26T06:41:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-03-26T06:41:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T06:41:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-03-26T06:41:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-03-26T06:41:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-03-26T06:41:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-03-26T06:41:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T06:41:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-03-26T06:41:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-03-26T06:41:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-03-26T06:41:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T06:41:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-03-26T06:41:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T06:41:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T06:41:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T06:41:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T06:41:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T06:41:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T06:41:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-03-26T06:41:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T06:41:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T06:41:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T06:41:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T06:41:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T06:41:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T06:41:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T06:41:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T06:41:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T06:41:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T06:41:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T06:41:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T06:41:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T06:41:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T06:41:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T06:41:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T06:41:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T06:41:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T06:41:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T06:41:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T06:41:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T06:41:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T06:41:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T06:41:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-26T06:41:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T06:41:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T06:41:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.214 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-03-26T06:41:06.214 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-03-26T06:41:06.214 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.214 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.214 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.215 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.215 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.215 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.215 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.215 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.215 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.215 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.216 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T06:41:06.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T06:41:06.216 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.216 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.216 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.216 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-26T06:41:06.216 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.216 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.217 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.217 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.217 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.217 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.217 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.217 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T06:41:06.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T06:41:06.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.218 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.218 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.218 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.218 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.218 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.218 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.218 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.218 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.219 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.219 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T06:41:06.219 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T06:41:06.219 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.219 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.219 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.219 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.220 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.220 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.220 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.220 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.220 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.220 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.220 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T06:41:06.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T06:41:06.221 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.221 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.221 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.221 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.221 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.221 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.221 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.221 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.221 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.222 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.222 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T06:41:06.222 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T06:41:06.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.222 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.222 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.223 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.223 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.223 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.223 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.223 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.223 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.223 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T06:41:06.224 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T06:41:06.224 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.224 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.224 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.224 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-03-26T06:41:06.224 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.224 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.224 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.224 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.225 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.225 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.225 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.225 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T06:41:06.225 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T06:41:06.225 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.225 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.226 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.226 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.226 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.226 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.226 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.226 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.226 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.227 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T06:41:06.227 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T06:41:06.227 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.227 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.227 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.227 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.227 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.228 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.228 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.228 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.228 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.228 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T06:41:06.229 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T06:41:06.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.229 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.229 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.229 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.229 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.229 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.230 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.230 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.230 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.230 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.230 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T06:41:06.230 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T06:41:06.230 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.230 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.231 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.231 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.231 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.231 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.231 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.231 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.231 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.231 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.232 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T06:41:06.232 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T06:41:06.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.232 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.232 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.232 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.233 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.233 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.233 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.233 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.233 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T06:41:06.233 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T06:41:06.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.234 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.234 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.234 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.234 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.234 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.234 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.235 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T06:41:06.235 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T06:41:06.235 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.235 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.235 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.236 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.236 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.236 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.236 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.237 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T06:41:06.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T06:41:06.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.237 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.237 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.237 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.238 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.238 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.238 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.238 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.238 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T06:41:06.238 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T06:41:06.238 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.239 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.239 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.239 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.239 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.239 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.239 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.239 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.240 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.240 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T06:41:06.240 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T06:41:06.240 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.240 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.240 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.240 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.241 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.241 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.241 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.241 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.241 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.241 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.242 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T06:41:06.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T06:41:06.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.242 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.242 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T06:41:06.242 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.242 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.242 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.243 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.243 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.243 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.243 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T06:41:06.243 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T06:41:06.243 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.244 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.244 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:06.244 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.244 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.244 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.244 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.244 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.244 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.245 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.245 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T06:41:06.245 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T06:41:06.245 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.245 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.245 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.246 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-03-26T06:41:06.246 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.246 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.246 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.246 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.246 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.246 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.246 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.247 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T06:41:06.247 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T06:41:06.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.247 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.247 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-03-26T06:41:06.247 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.247 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.248 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.248 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.248 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.248 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T06:41:06.248 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T06:41:06.248 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.249 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.249 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.249 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.249 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.249 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.249 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.249 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.250 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.250 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T06:41:06.250 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T06:41:06.250 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.250 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.250 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.250 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-03-26T06:41:06.250 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.251 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.251 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.251 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.251 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.251 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.251 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.251 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T06:41:06.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T06:41:06.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.252 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.252 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.252 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.252 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.252 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.252 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.252 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.253 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.253 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T06:41:06.253 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T06:41:06.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.253 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.253 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.254 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.254 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.254 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.254 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.254 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.254 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.255 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T06:41:06.255 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T06:41:06.255 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.255 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.255 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.255 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.255 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.255 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.256 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.256 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.256 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.256 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.256 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T06:41:06.256 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T06:41:06.256 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.257 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.257 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:06.257 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.257 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.257 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.257 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.257 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.258 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.258 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T06:41:06.258 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T06:41:06.258 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.258 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.258 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.259 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:06.259 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.259 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.259 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.259 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.259 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.259 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.259 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.259 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T06:41:06.260 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T06:41:06.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.260 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.260 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:06.260 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.260 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.260 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.261 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.261 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.261 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.261 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.261 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-03-26T06:41:06.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-03-26T06:41:06.261 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.261 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.261 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.262 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.262 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.262 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.262 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.262 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.262 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.262 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.262 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.263 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-03-26T06:41:06.263 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-03-26T06:41:06.263 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.263 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.263 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.263 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.263 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.264 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.264 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.264 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.264 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.264 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.264 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-03-26T06:41:06.264 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-03-26T06:41:06.264 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.265 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.265 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.265 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.265 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.265 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.265 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-03-26T06:41:06.265 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.266 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.266 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-03-26T06:41:06.266 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-03-26T06:41:06.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.266 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.266 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-03-26T06:41:06.266 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.267 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.267 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.267 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.267 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.267 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.267 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.267 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-26T06:41:06.268 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-26T06:41:06.268 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.268 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.268 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.268 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.268 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.268 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.268 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.268 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.269 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.269 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.269 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.269 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.269 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.269 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.269 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.270 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.270 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.270 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.270 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.270 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.270 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.270 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.270 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.271 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.271 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.271 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.271 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.271 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.271 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.271 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.272 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.272 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.272 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.272 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.272 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.272 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.272 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.272 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.272 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.273 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.273 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.273 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.273 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.273 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.273 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.274 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.274 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.274 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.274 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.274 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.274 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.274 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.274 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.274 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.275 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.275 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.275 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.275 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.275 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.275 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.276 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.276 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.276 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.276 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.276 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.276 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.276 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.276 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.276 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.277 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-03-26T06:41:06.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-03-26T06:41:06.277 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.277 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.277 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.277 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.277 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.278 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.278 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.278 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-03-26T06:41:06.278 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.278 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.278 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.278 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-03-26T06:41:06.279 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T06:41:06.279 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.279 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.279 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.279 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.279 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.279 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.280 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.280 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.280 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.280 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.280 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-26T06:41:06.280 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-26T06:41:06.280 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.280 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.281 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.281 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-26T06:41:06.281 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.281 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.281 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.281 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.281 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.281 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.281 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.282 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-26T06:41:06.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-26T06:41:06.282 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.282 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.282 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.282 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-03-26T06:41:06.282 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.283 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.283 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.283 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.283 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.283 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-26T06:41:06.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-26T06:41:06.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.284 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.284 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.284 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.284 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.284 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.284 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.285 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.285 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-03-26T06:41:06.285 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-03-26T06:41:06.285 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.285 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.285 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.285 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-03-26T06:41:06.285 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.286 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.286 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.286 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.286 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.286 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.286 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.286 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-26T06:41:06.286 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-26T06:41:06.287 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.287 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.287 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.287 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-03-26T06:41:06.287 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.287 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.287 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.288 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-03-26T06:41:06.288 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.288 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.288 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.288 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.288 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.288 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-26T06:41:06.288 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-26T06:41:06.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.289 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.289 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-26T06:41:06.289 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-26T06:41:06.289 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-26T06:41:06.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.289 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.289 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.290 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.290 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.290 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-03-26T06:41:06.290 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-03-26T06:41:06.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.290 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.290 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.290 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-03-26T06:41:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-03-26T06:41:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-03-26T06:41:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-03-26T06:41:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-03-26T06:41:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-03-26T06:41:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-03-26T06:41:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T06:41:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-03-26T06:41:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-03-26T06:41:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-03-26T06:41:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T06:41:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-03-26T06:41:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-03-26T06:41:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-03-26T06:41:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-03-26T06:41:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-03-26T06:41:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-03-26T06:41:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-26T06:41:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T06:41:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-03-26T06:41:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-03-26T06:41:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T06:41:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T06:41:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-03-26T06:41:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-03-26T06:41:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-03-26T06:41:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T06:41:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-03-26T06:41:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.317 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.317 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.317 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.317 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.317 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.317 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.317 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.317 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.318 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.318 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.318 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.318 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.318 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.319 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.319 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.319 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.319 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.319 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.319 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.320 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.320 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.320 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.320 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.320 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.321 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.321 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.321 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.321 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.321 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.321 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.321 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.321 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.322 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.322 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-03-26T06:41:06.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-03-26T06:41:06.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.322 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.322 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.322 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-03-26T06:41:06.323 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.323 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.323 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.323 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.323 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.323 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.323 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.324 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-03-26T06:41:06.324 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-03-26T06:41:06.324 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.324 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.324 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.324 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-03-26T06:41:06.324 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.324 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.325 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.325 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.325 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.325 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.325 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.325 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:06.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:06.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.326 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.326 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.326 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-03-26T06:41:06.326 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.326 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.326 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.326 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.326 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.326 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.327 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.327 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.327 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.327 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-03-26T06:41:06.327 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-03-26T06:41:06.327 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-03-26T06:41:06.328 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.328 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-03-26T06:41:06.328 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-03-26T06:41:06.328 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.328 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.328 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.328 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-03-26T06:41:06.328 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.328 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T06:41:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-03-26T06:41:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-03-26T06:41:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-03-26T06:41:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T06:41:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-03-26T06:41:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-03-26T06:41:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.332 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.332 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.332 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-03-26T06:41:06.332 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.332 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.332 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.332 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:06.332 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:06.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.333 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.333 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-03-26T06:41:06.333 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-03-26T06:41:06.333 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.333 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.333 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T06:41:06.333 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.334 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.334 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.334 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.334 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.334 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.335 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.335 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.335 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.335 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.335 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.336 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.336 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.336 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.336 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.336 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.336 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.336 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.337 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.337 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.337 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.337 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.338 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.338 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.338 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.339 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.339 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.339 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.339 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.340 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.340 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.340 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.340 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.340 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.340 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.340 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.341 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.341 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.341 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.341 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.341 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.341 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.342 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.342 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.342 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-03-26T06:41:06.342 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-03-26T06:41:06.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.342 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.343 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.343 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.343 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.343 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.343 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-03-26T06:41:06.343 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.343 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.343 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.344 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-03-26T06:41:06.344 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-03-26T06:41:06.344 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.344 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.344 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-03-26T06:41:06.344 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.345 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.345 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-03-26T06:41:06.345 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.345 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.345 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.345 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-03-26T06:41:06.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-03-26T06:41:06.346 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.346 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.346 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.346 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-03-26T06:41:06.346 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.346 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T06:41:06.347 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.347 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.347 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.347 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-03-26T06:41:06.347 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-03-26T06:41:06.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.348 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.348 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:06.348 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:06.348 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-03-26T06:41:06.348 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.348 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T06:41:06.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.349 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.349 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-03-26T06:41:06.349 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-03-26T06:41:06.349 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-03-26T06:41:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-03-26T06:41:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-03-26T06:41:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-03-26T06:41:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T06:41:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-03-26T06:41:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.353 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.353 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.353 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.353 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.353 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.353 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.353 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.353 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.354 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.354 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.354 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.354 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.354 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.354 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.354 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.354 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.354 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.355 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.355 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.355 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.355 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.356 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.356 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.356 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.356 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.356 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.356 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.357 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.357 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.357 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.357 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.357 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.357 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.357 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.360 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.360 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.360 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.360 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.360 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.360 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-03-26T06:41:06.361 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.361 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.361 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.361 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.362 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.362 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.362 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.362 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.362 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.362 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.363 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.363 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.363 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.363 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.363 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.363 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.363 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.363 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.363 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.364 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.364 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.364 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.364 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.365 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.365 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.365 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.365 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.365 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.365 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.365 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.366 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.366 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.366 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.366 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.366 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.366 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.367 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.367 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.367 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.367 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.367 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.367 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.367 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.367 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.368 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.368 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.368 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.368 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.368 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.368 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.368 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.369 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.369 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.369 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.369 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.369 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.369 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.369 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.369 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-03-26T06:41:06.370 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.370 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.370 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.370 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-03-26T06:41:06.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-03-26T06:41:06.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.370 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.371 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-03-26T06:41:06.371 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.371 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.371 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T06:41:06.371 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.371 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.371 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.372 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-03-26T06:41:06.372 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-03-26T06:41:06.372 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.372 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.372 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.372 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:06.372 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-03-26T06:41:06.372 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.372 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.372 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T06:41:06.373 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.373 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.373 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.373 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.373 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.374 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.374 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.374 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.374 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.374 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.374 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.374 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.374 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.374 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.375 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.375 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.375 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.375 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.375 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.375 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.376 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.376 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.377 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.377 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.377 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.377 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.378 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.378 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.378 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.378 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.379 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.379 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.379 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.379 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.379 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.379 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.380 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.380 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.380 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.380 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.381 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.381 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.381 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.381 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.381 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-03-26T06:41:06.382 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.382 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.382 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.382 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-03-26T06:41:06.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-03-26T06:41:06.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.382 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-03-26T06:41:06.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.383 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-03-26T06:41:06.383 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T06:41:06.383 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.383 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.384 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.384 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.384 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.384 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.384 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.384 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.385 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.385 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.385 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.385 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.385 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.385 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.385 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.386 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.386 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.386 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.386 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.386 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.386 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.387 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.387 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.387 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.387 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.387 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.387 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.387 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.387 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.388 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.388 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.389 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.389 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.389 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.389 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.389 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.389 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.389 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.389 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.390 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.390 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.390 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.390 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.390 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.392 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-03-26T06:41:06.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-03-26T06:41:06.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.392 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.392 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.392 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.393 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-03-26T06:41:06.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T06:41:06.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.393 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.393 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-03-26T06:41:06.393 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-03-26T06:41:06.393 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.393 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.394 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.394 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-03-26T06:41:06.394 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.394 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.394 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-03-26T06:41:06.394 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.394 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.395 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.395 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-03-26T06:41:06.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-03-26T06:41:06.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-03-26T06:41:06.396 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.396 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T06:41:06.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.397 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-03-26T06:41:06.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-03-26T06:41:06.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-03-26T06:41:06.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.398 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.398 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-03-26T06:41:06.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-03-26T06:41:06.398 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-03-26T06:41:06.399 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.399 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.399 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.399 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T06:41:06.399 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T06:41:06.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.400 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.400 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.400 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.400 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-03-26T06:41:06.400 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.400 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.400 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.401 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.401 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.401 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.402 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.402 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.402 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.402 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.402 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.402 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.402 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.403 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.403 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.403 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.403 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.404 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.404 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.404 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.404 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.405 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.405 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.405 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.406 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.406 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.406 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.406 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.406 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.406 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.407 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.407 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.407 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.408 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.409 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.409 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-03-26T06:41:06.410 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.410 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.410 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.410 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.410 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.411 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.411 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.411 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.411 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.411 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.411 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.411 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.411 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.413 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.413 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.413 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.413 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.413 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.413 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.414 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.414 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.414 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.414 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.414 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.414 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.415 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.415 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.415 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.415 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.415 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.415 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.415 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.415 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.415 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.416 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.416 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.416 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.416 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.416 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.417 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.417 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.417 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.417 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.417 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.417 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.417 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.417 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.418 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.418 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-03-26T06:41:06.418 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.419 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.419 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.419 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.419 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.419 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.420 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.420 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.420 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.420 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.420 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.420 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.420 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.420 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.420 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.420 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.421 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.421 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.421 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.421 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.421 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.422 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.422 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.422 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.422 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.422 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.422 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.422 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.423 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.423 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.423 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.423 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.423 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.423 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.423 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.423 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.424 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.424 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.424 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.424 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.424 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.424 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.424 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.424 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.424 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.424 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.425 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.425 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.425 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.425 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.426 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.426 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.426 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.426 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.426 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.426 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-26T06:41:06.427 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.427 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.427 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.427 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-03-26T06:41:06.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-03-26T06:41:06.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T06:41:06.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.429 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.429 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.429 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.429 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.429 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.429 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.430 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.430 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.430 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.430 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.430 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.430 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.430 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.430 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.431 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.431 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.432 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.432 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.432 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.432 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.433 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.433 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.433 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.433 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.433 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.433 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.434 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.434 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.434 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.434 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.434 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.434 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.434 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.434 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.435 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.435 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.435 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.436 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.437 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-03-26T06:41:06.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-03-26T06:41:06.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.437 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.437 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.438 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.438 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-03-26T06:41:06.438 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-03-26T06:41:06.438 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-03-26T06:41:06.438 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.438 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-03-26T06:41:06.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.439 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.439 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.439 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.439 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-03-26T06:41:06.439 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.440 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.440 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.440 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.440 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.440 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.440 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.440 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.441 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.441 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.441 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.441 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.441 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.441 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.441 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.442 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.442 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.442 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.442 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.442 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.442 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.442 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.443 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.443 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.443 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.443 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.444 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.444 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.444 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.444 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.444 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.444 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.445 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.445 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.445 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.445 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.445 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.445 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.445 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.445 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.446 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.446 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.446 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.446 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.446 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.446 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.446 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.447 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.447 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.447 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.447 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.447 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.447 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.448 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.448 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.448 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-03-26T06:41:06.448 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.448 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.448 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.448 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.449 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.449 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.450 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.450 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.450 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.450 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.450 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.450 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.451 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.451 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.452 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.452 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.452 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.452 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.452 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.452 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.453 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.453 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.453 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.453 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.453 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.454 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.454 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.454 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.454 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.454 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.454 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.454 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.455 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.456 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.456 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-03-26T06:41:06.457 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.457 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.457 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.457 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.458 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.458 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.458 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.458 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.459 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.459 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.459 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.459 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.459 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.459 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.459 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.460 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.460 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.460 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.460 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.460 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.461 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.461 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.462 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.462 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.462 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.463 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.463 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.463 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.463 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.463 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.463 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.463 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.464 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.465 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:06.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:06.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.466 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.466 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:06.466 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.466 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.466 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.466 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.467 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-03-26T06:41:06.467 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-03-26T06:41:06.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:06.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-03-26T06:41:06.468 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-03-26T06:41:06.468 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-03-26T06:41:06.468 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T06:41:06.468 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.468 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.468 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.468 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-03-26T06:41:06.469 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.469 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.469 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.469 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-03-26T06:41:06.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-03-26T06:41:06.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.470 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.470 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.470 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.470 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.470 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.470 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T06:41:06.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.471 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.471 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-26T06:41:06.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-26T06:41:06.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.472 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.472 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.472 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.473 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.473 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.473 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.474 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.474 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.474 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.474 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.475 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.475 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.475 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.476 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.477 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.477 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.477 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.478 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.478 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.478 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.478 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.479 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.479 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.480 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-03-26T06:41:06.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-03-26T06:41:06.480 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.480 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.480 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-03-26T06:41:06.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-03-26T06:41:06.481 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.481 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.481 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.482 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-03-26T06:41:06.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-03-26T06:41:06.482 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.482 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.482 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.482 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-03-26T06:41:06.482 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.482 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.483 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-03-26T06:41:06.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.483 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.483 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.483 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.483 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-03-26T06:41:06.484 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.484 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.484 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.484 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-03-26T06:41:06.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-03-26T06:41:06.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.485 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.485 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.485 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.485 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.485 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.485 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.486 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-03-26T06:41:06.486 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-03-26T06:41:06.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.486 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.487 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.487 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T06:41:06.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T06:41:06.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.487 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.488 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.488 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.488 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.488 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.488 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.489 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.489 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.489 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.489 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.489 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.489 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.489 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.489 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.489 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.490 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.490 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.490 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.490 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.491 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.491 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.491 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.492 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.492 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.492 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.492 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.492 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.493 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.493 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.493 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.493 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.493 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.494 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.494 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.494 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.494 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.494 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.494 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.494 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.494 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.495 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.495 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.495 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-03-26T06:41:06.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-03-26T06:41:06.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.495 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.496 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.496 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.496 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.497 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T06:41:06.497 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T06:41:06.497 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.497 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.497 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.497 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-03-26T06:41:06.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.498 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.498 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.498 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.498 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-03-26T06:41:06.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-03-26T06:41:06.499 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:06.499 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.499 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.499 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-03-26T06:41:06.499 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.499 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.499 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.500 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.500 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-26T06:41:06.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-26T06:41:06.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-03-26T06:41:06.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.501 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.501 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T06:41:06.501 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.502 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-26T06:41:06.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-26T06:41:06.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-03-26T06:41:06.502 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.502 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.502 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.503 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T06:41:06.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.503 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-03-26T06:41:06.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-03-26T06:41:06.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.503 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.504 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.504 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-03-26T06:41:06.504 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.504 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.504 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.504 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.504 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.504 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.504 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.505 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-03-26T06:41:06.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-03-26T06:41:06.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.505 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.505 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-03-26T06:41:06.505 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-03-26T06:41:06.506 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-03-26T06:41:06.506 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.506 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.506 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.506 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.506 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.507 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-03-26T06:41:06.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-03-26T06:41:06.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.507 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-03-26T06:41:06.507 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-03-26T06:41:06.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-03-26T06:41:06.508 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.508 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.508 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.509 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-03-26T06:41:06.509 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.509 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.509 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.509 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.509 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.510 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.510 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.510 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.510 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.510 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.510 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.510 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.510 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.510 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.511 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.511 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.511 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.511 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.511 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.511 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.511 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.512 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.513 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.513 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.513 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.513 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.513 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.514 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.514 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.514 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.514 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.514 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.514 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.514 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.514 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.514 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.515 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.515 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.515 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.515 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.516 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.516 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.516 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.516 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.516 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.516 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.517 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-03-26T06:41:06.517 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-03-26T06:41:06.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.517 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-03-26T06:41:06.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.518 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.518 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.518 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.518 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-03-26T06:41:06.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T06:41:06.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.519 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.519 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.519 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.520 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.520 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.520 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.520 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-03-26T06:41:06.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-03-26T06:41:06.520 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.521 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.521 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.521 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.521 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.522 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-03-26T06:41:06.522 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.522 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.522 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.522 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-03-26T06:41:06.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-03-26T06:41:06.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.524 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-03-26T06:41:06.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-03-26T06:41:06.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.525 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.525 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.526 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.526 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.526 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.526 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.526 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.527 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.527 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.527 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.527 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.527 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.527 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.527 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.528 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.528 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.529 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.529 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.530 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.530 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.531 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.531 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.531 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.531 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.531 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.531 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.531 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.532 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.532 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.532 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.532 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.532 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.533 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.533 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.533 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-03-26T06:41:06.533 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-03-26T06:41:06.533 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.534 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.534 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.534 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.534 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.534 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.534 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.534 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.534 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.535 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.535 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-03-26T06:41:06.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-03-26T06:41:06.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.536 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.536 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.536 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.536 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.537 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-03-26T06:41:06.537 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.537 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.537 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.537 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.538 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.538 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.538 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.538 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.538 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.538 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.539 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.539 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.539 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.539 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.539 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.539 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.539 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.539 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.540 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.540 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.540 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.540 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.541 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.541 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.541 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.541 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.541 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.541 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.542 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.542 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.542 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.542 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.542 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.542 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.542 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.543 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.543 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.543 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.543 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.543 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.543 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.543 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.544 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.544 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.544 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.544 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.544 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.545 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.545 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.545 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-03-26T06:41:06.546 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.546 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.546 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.546 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.547 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.547 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.547 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.548 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.548 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.548 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.548 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.548 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.549 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.550 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.550 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.550 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.551 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.551 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.552 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.552 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.552 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.552 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.552 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.553 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.553 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.553 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.553 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.553 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.554 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.554 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.554 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-03-26T06:41:06.555 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-03-26T06:41:06.555 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.555 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-03-26T06:41:06.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T06:41:06.556 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.556 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.556 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.556 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.556 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.556 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-03-26T06:41:06.556 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.557 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.557 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.557 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.557 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.557 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.557 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.557 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.557 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.558 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.558 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.558 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.558 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.558 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.558 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.558 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.558 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.559 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.559 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.559 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.560 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.560 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.560 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.560 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.561 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.561 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.561 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.561 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.563 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.565 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-03-26T06:41:06.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-03-26T06:41:06.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.566 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.566 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.566 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-03-26T06:41:06.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-03-26T06:41:06.566 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.567 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.567 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-03-26T06:41:06.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.568 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-03-26T06:41:06.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-03-26T06:41:06.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.569 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.569 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.569 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.569 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.569 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.569 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.569 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-03-26T06:41:06.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-03-26T06:41:06.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.570 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.571 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-03-26T06:41:06.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-03-26T06:41:06.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-03-26T06:41:06.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.571 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.571 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.572 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.572 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-03-26T06:41:06.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-03-26T06:41:06.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.573 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.573 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.573 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.574 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.574 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.574 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-03-26T06:41:06.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-03-26T06:41:06.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.575 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-03-26T06:41:06.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-03-26T06:41:06.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.577 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.577 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-03-26T06:41:06.577 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-03-26T06:41:06.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.578 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.578 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-03-26T06:41:06.578 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-03-26T06:41:06.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.579 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.579 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-03-26T06:41:06.579 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-03-26T06:41:06.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.580 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.580 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-03-26T06:41:06.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-03-26T06:41:06.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.582 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.582 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-03-26T06:41:06.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-03-26T06:41:06.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.583 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.583 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.583 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.584 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-03-26T06:41:06.584 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.584 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.584 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.584 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-03-26T06:41:06.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-03-26T06:41:06.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T06:41:06.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.585 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.586 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-03-26T06:41:06.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-03-26T06:41:06.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.586 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-03-26T06:41:06.587 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.587 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.587 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.587 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.588 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.588 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.588 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.588 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.588 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.588 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.588 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.588 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.589 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.589 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.589 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.589 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.589 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.589 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.590 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.590 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.591 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.591 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.592 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.592 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.592 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.592 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.592 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.592 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.592 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.593 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.593 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.593 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.593 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.593 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.593 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.593 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.593 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.593 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.594 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.594 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.594 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.594 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.594 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.595 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.595 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.595 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.595 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.595 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.595 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.595 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-03-26T06:41:06.596 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.596 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.596 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.596 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.597 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.597 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.597 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.598 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.598 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.598 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.598 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.599 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.600 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.600 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.600 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.600 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.601 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.601 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.601 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.601 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.601 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.601 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.601 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.601 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.602 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.602 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.602 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.602 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.602 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.603 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.604 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.604 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-03-26T06:41:06.605 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.605 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.605 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.605 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.606 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.606 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.606 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.606 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.606 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.607 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.607 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.607 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.608 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.608 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.608 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.609 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.609 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.610 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.610 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.610 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.610 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.610 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.610 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.610 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.611 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.611 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.611 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.611 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.611 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.611 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.611 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.613 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.613 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.613 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-03-26T06:41:06.613 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.613 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.613 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.613 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-03-26T06:41:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-03-26T06:41:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T06:41:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.614 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-03-26T06:41:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.616 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.617 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.618 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.622 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.623 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.624 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-03-26T06:41:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-03-26T06:41:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-03-26T06:41:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.625 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T06:41:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.626 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-03-26T06:41:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-03-26T06:41:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.627 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T06:41:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-03-26T06:41:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-03-26T06:41:06.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T06:41:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.630 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-03-26T06:41:06.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-03-26T06:41:06.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.631 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.631 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T06:41:06.631 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.631 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.631 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-03-26T06:41:06.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-03-26T06:41:06.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.632 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.632 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.632 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.632 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.632 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.632 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T06:41:06.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.633 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-03-26T06:41:06.633 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-03-26T06:41:06.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.633 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.633 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.634 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.634 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.634 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.634 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.634 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-03-26T06:41:06.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-03-26T06:41:06.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.635 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.635 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.635 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.636 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.636 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-03-26T06:41:06.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-03-26T06:41:06.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.636 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T06:41:06.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.636 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-03-26T06:41:06.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-03-26T06:41:06.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.638 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.638 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.639 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:06.639 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:06.639 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.639 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.639 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-03-26T06:41:06.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-03-26T06:41:06.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.640 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.640 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.640 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.641 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-03-26T06:41:06.641 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-03-26T06:41:06.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.641 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-03-26T06:41:06.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.642 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.642 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-03-26T06:41:06.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-03-26T06:41:06.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.642 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.643 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.643 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.643 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.643 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.643 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.643 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.643 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.644 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-03-26T06:41:06.644 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-03-26T06:41:06.644 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.645 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.645 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.646 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.646 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.646 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.646 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.646 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.646 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.647 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.647 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.647 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.647 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.647 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.647 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.647 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.648 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.648 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.648 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.648 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.649 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.649 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.649 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.649 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.649 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.650 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.650 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.650 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.650 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.650 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.651 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.651 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.651 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.651 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.651 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.651 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.651 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.652 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.652 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.652 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.653 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-03-26T06:41:06.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-03-26T06:41:06.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.654 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.654 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.655 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-03-26T06:41:06.655 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-03-26T06:41:06.655 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.655 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.655 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.655 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.655 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.655 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.656 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.656 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.656 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.656 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.656 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-03-26T06:41:06.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-03-26T06:41:06.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.657 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.657 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T06:41:06.657 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.657 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.658 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.658 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.658 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-03-26T06:41:06.658 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.658 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.658 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.659 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.660 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.660 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.660 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.660 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.660 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.660 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.660 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.660 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.660 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.661 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.661 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.661 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.661 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.661 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.661 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.662 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.662 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.662 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.662 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.663 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.663 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.663 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.663 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.663 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.663 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.664 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.664 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.664 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.664 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.664 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.664 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.664 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.665 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.665 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.665 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.665 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.666 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.666 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-03-26T06:41:06.667 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.667 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.667 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.667 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.668 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.668 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.668 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.668 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.668 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.668 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.669 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.669 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.669 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.669 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.669 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.669 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.670 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.670 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.670 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.671 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.671 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.672 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.673 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.673 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.674 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.675 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-03-26T06:41:06.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-03-26T06:41:06.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.675 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-03-26T06:41:06.675 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.675 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T06:41:06.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.676 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-03-26T06:41:06.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-03-26T06:41:06.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.677 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-03-26T06:41:06.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.678 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.678 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-03-26T06:41:06.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-03-26T06:41:06.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-03-26T06:41:06.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T06:41:06.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.679 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-03-26T06:41:06.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T06:41:06.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-03-26T06:41:06.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T06:41:06.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.681 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.681 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-03-26T06:41:06.681 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-03-26T06:41:06.682 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-03-26T06:41:06.682 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-03-26T06:41:06.682 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-03-26T06:41:06.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-03-26T06:41:06.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-03-26T06:41:06.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-03-26T06:41:06.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.683 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-03-26T06:41:06.683 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-03-26T06:41:06.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.684 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.684 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.685 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-03-26T06:41:06.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-03-26T06:41:06.685 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-03-26T06:41:06.685 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-03-26T06:41:06.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.686 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.687 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-03-26T06:41:06.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-03-26T06:41:06.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.690 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.690 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.690 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.691 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.691 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.691 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.691 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.691 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.691 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.691 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.692 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-03-26T06:41:06.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-03-26T06:41:06.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.692 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.692 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.693 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.693 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-03-26T06:41:06.694 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-03-26T06:41:06.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-03-26T06:41:06.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-03-26T06:41:06.694 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-03-26T06:41:06.695 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-03-26T06:41:06.695 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-03-26T06:41:06.695 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-03-26T06:41:06.695 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-03-26T06:41:06.695 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-03-26T06:41:06.695 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.695 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.695 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.696 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-03-26T06:41:06.696 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-03-26T06:41:06.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-03-26T06:41:06.696 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-03-26T06:41:06.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-03-26T06:41:06.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.697 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.697 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.697 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.697 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.697 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.697 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-03-26T06:41:06.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-03-26T06:41:06.698 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.698 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.698 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.698 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-03-26T06:41:06.698 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.698 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.698 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.699 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.699 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.699 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.699 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-03-26T06:41:06.699 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-03-26T06:41:06.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-03-26T06:41:06.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-03-26T06:41:06.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.700 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.701 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-03-26T06:41:06.701 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-03-26T06:41:06.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-03-26T06:41:06.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-03-26T06:41:06.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-03-26T06:41:06.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T06:41:06.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-03-26T06:41:06.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-03-26T06:41:06.701 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-03-26T06:41:06.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-03-26T06:41:06.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-03-26T06:41:06.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-03-26T06:41:06.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-03-26T06:41:06.702 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.702 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.702 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.702 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.702 INFO:teuthology.orchestra.run.smithi037.stdout: "services": { 2024-03-26T06:41:06.703 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard": "https://172.21.15.37:8443/" 2024-03-26T06:41:06.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.703 INFO:teuthology.orchestra.run.smithi037.stdout: "always_on_modules": { 2024-03-26T06:41:06.703 INFO:teuthology.orchestra.run.smithi037.stdout: "octopus": [ 2024-03-26T06:41:06.703 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-03-26T06:41:06.703 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-26T06:41:06.703 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-03-26T06:41:06.703 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-03-26T06:41:06.704 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-03-26T06:41:06.704 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-03-26T06:41:06.704 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-03-26T06:41:06.704 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-03-26T06:41:06.704 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-03-26T06:41:06.704 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-03-26T06:41:06.704 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.704 INFO:teuthology.orchestra.run.smithi037.stdout: "pacific": [ 2024-03-26T06:41:06.704 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-03-26T06:41:06.705 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-26T06:41:06.705 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-03-26T06:41:06.705 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-03-26T06:41:06.705 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-03-26T06:41:06.705 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-03-26T06:41:06.705 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-03-26T06:41:06.705 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-03-26T06:41:06.705 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-03-26T06:41:06.706 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-03-26T06:41:06.706 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.706 INFO:teuthology.orchestra.run.smithi037.stdout: "quincy": [ 2024-03-26T06:41:06.706 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-03-26T06:41:06.706 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-26T06:41:06.706 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-03-26T06:41:06.706 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-03-26T06:41:06.706 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-03-26T06:41:06.707 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-03-26T06:41:06.707 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-03-26T06:41:06.707 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-03-26T06:41:06.707 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-03-26T06:41:06.707 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-03-26T06:41:06.707 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.707 INFO:teuthology.orchestra.run.smithi037.stdout: "reef": [ 2024-03-26T06:41:06.707 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-03-26T06:41:06.707 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-26T06:41:06.708 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-03-26T06:41:06.708 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-03-26T06:41:06.708 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-03-26T06:41:06.708 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-03-26T06:41:06.708 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-03-26T06:41:06.708 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-03-26T06:41:06.708 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-03-26T06:41:06.708 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-03-26T06:41:06.708 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-26T06:41:06.709 INFO:teuthology.orchestra.run.smithi037.stdout: "squid": [ 2024-03-26T06:41:06.709 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-03-26T06:41:06.709 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-26T06:41:06.709 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-03-26T06:41:06.709 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-03-26T06:41:06.709 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-03-26T06:41:06.709 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-03-26T06:41:06.709 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-03-26T06:41:06.709 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-03-26T06:41:06.710 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-03-26T06:41:06.710 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-03-26T06:41:06.710 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:41:06.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.710 INFO:teuthology.orchestra.run.smithi037.stdout: "last_failure_osd_epoch": 88, 2024-03-26T06:41:06.710 INFO:teuthology.orchestra.run.smithi037.stdout: "active_clients": [ 2024-03-26T06:41:06.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-03-26T06:41:06.710 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-26T06:41:06.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-26T06:41:06.711 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-26T06:41:06.711 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3076641929 2024-03-26T06:41:06.711 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.711 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:41:06.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "libcephsqlite", 2024-03-26T06:41:06.712 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-26T06:41:06.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-26T06:41:06.712 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-26T06:41:06.712 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2492853550 2024-03-26T06:41:06.712 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.712 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:41:06.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-03-26T06:41:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-26T06:41:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-26T06:41:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-26T06:41:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 498136820 2024-03-26T06:41:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:41:06.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-03-26T06:41:06.714 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-26T06:41:06.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-26T06:41:06.714 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-26T06:41:06.714 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3671224524 2024-03-26T06:41:06.714 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.715 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:41:06.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-03-26T06:41:06.715 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-26T06:41:06.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-26T06:41:06.715 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-26T06:41:06.715 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2217346853 2024-03-26T06:41:06.716 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.716 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:41:06.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:06.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-03-26T06:41:06.716 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-03-26T06:41:06.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-03-26T06:41:06.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-03-26T06:41:06.717 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-03-26T06:41:06.717 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1930399709 2024-03-26T06:41:06.717 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.717 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:41:06.717 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:06.717 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-26T06:41:06.717 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:41:06.719 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:05 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1606326912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:41:06.719 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:05 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1209195006' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T06:41:06.719 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:06 smithi066 ceph-mon[36504]: pgmap v572: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 703 op/s 2024-03-26T06:41:06.719 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:06 smithi066 ceph-mon[36504]: osdmap e115: 12 total, 12 up, 12 in 2024-03-26T06:41:06.719 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:05 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1606326912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:41:06.719 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:05 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1209195006' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T06:41:06.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:06 smithi149 ceph-mon[36681]: pgmap v572: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 703 op/s 2024-03-26T06:41:06.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:06 smithi149 ceph-mon[36681]: osdmap e115: 12 total, 12 up, 12 in 2024-03-26T06:41:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 -- 172.21.15.37:0/1209195006 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2eec0a2970 msgr2=0x7f2eec0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:06.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2eec0a2970 0x7f2eec0a4e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8006010 tx=0x7f2ee8000bc0 comp rx=0 tx=0).stop 2024-03-26T06:41:06.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 -- 172.21.15.37:0/1209195006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f00101520 msgr2=0x7f2f0019d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:06.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f00101520 0x7f2f0019d7f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000d770 tx=0x7f2ef000dc30 comp rx=0 tx=0).stop 2024-03-26T06:41:06.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 -- 172.21.15.37:0/1209195006 shutdown_connections 2024-03-26T06:41:06.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f2eec0a2970 0x7f2eec0a4e30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:06.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f001979a0 0x7f2f00197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:06.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f001065f0 0x7f2f0019dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:06.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 --2- 172.21.15.37:0/1209195006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f00101520 0x7f2f0019d7f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:06.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 -- 172.21.15.37:0/1209195006 >> 172.21.15.37:0/1209195006 conn(0x7f2f000fcd60 msgr2=0x7f2f000ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 -- 172.21.15.37:0/1209195006 shutdown_connections 2024-03-26T06:41:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:04.642+0000 7f2f04f97640 1 -- 172.21.15.37:0/1209195006 wait complete. 2024-03-26T06:41:06.727 DEBUG:tasks.check_counter:Getting stats from a 2024-03-26T06:41:06.727 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-26T06:41:06.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:05 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1606326912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:41:06.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:05 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1209195006' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T06:41:06.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:06 smithi037 ceph-mon[31974]: pgmap v572: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 703 op/s 2024-03-26T06:41:06.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:06 smithi037 ceph-mon[31974]: osdmap e115: 12 total, 12 up, 12 in 2024-03-26T06:41:07.004 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:41:08.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:08 smithi149 ceph-mon[36681]: pgmap v574: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 741 op/s 2024-03-26T06:41:08.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:08 smithi037 ceph-mon[31974]: pgmap v574: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 741 op/s 2024-03-26T06:41:08.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:08 smithi066 ceph-mon[36504]: pgmap v574: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 741 op/s 2024-03-26T06:41:09.110 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:41:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 52099, 2024-03-26T06:41:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 51536, 2024-03-26T06:41:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 16383053, 2024-03-26T06:41:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 138038305, 2024-03-26T06:41:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 18, 2024-03-26T06:41:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 7, 2024-03-26T06:41:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 3.888325846, 2024-03-26T06:41:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 1.588059159, 2024-03-26T06:41:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 1.842565596, 2024-03-26T06:41:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.651605708, 2024-03-26T06:41:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 51531, 2024-03-26T06:41:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 2.459555941, 2024-03-26T06:41:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000047729 2024-03-26T06:41:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 65772, 2024-03-26T06:41:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.012998314, 2024-03-26T06:41:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000197 2024-03-26T06:41:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 4208, 2024-03-26T06:41:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 13312 2024-03-26T06:41:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 339862, 2024-03-26T06:41:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 421634, 2024-03-26T06:41:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 101275062, 2024-03-26T06:41:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 383237509, 2024-03-26T06:41:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 12, 2024-03-26T06:41:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 7, 2024-03-26T06:41:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 31.036190022, 2024-03-26T06:41:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 20.885391983, 2024-03-26T06:41:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 12.605623728, 2024-03-26T06:41:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.757321542, 2024-03-26T06:41:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 421634, 2024-03-26T06:41:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 18.917447902, 2024-03-26T06:41:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000044866 2024-03-26T06:41:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 524081, 2024-03-26T06:41:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.101523085, 2024-03-26T06:41:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000193 2024-03-26T06:41:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.116 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-03-26T06:41:09.116 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 129488 2024-03-26T06:41:09.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.116 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:09.116 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 53037, 2024-03-26T06:41:09.116 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 59594, 2024-03-26T06:41:09.116 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 16404815, 2024-03-26T06:41:09.116 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 159164347, 2024-03-26T06:41:09.116 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 9, 2024-03-26T06:41:09.117 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 6, 2024-03-26T06:41:09.117 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 4.103706787, 2024-03-26T06:41:09.117 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 1.697781832, 2024-03-26T06:41:09.117 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 1.868350993, 2024-03-26T06:41:09.117 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.681603762, 2024-03-26T06:41:09.117 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:09.117 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 59594, 2024-03-26T06:41:09.117 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 2.492727694, 2024-03-26T06:41:09.118 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000041828 2024-03-26T06:41:09.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.118 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:09.118 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 70284, 2024-03-26T06:41:09.118 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.015761018, 2024-03-26T06:41:09.118 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000224 2024-03-26T06:41:09.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.118 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1939328, 2024-03-26T06:41:09.119 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 682672 2024-03-26T06:41:09.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.119 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-03-26T06:41:09.119 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 1, 2024-03-26T06:41:09.119 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-03-26T06:41:09.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.119 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-MDSRank": { 2024-03-26T06:41:09.119 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-03-26T06:41:09.119 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-03-26T06:41:09.120 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 56787, 2024-03-26T06:41:09.120 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 69.808442480, 2024-03-26T06:41:09.120 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001229303 2024-03-26T06:41:09.120 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:09.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.120 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-PurgeQueue": { 2024-03-26T06:41:09.120 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-03-26T06:41:09.120 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-03-26T06:41:09.120 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 37227, 2024-03-26T06:41:09.121 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 5.575190481, 2024-03-26T06:41:09.121 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000149762 2024-03-26T06:41:09.121 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:09.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.121 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-26T06:41:09.121 INFO:teuthology.orchestra.run.smithi037.stdout: "request": 123066, 2024-03-26T06:41:09.121 INFO:teuthology.orchestra.run.smithi037.stdout: "reply": 122704, 2024-03-26T06:41:09.121 INFO:teuthology.orchestra.run.smithi037.stdout: "reply_latency": { 2024-03-26T06:41:09.121 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 122704, 2024-03-26T06:41:09.122 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 474.020959141, 2024-03-26T06:41:09.122 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.003863125 2024-03-26T06:41:09.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.122 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_reply": 0, 2024-03-26T06:41:09.122 INFO:teuthology.orchestra.run.smithi037.stdout: "forward": 362, 2024-03-26T06:41:09.122 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_complete": 0, 2024-03-26T06:41:09.122 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_keys": 0, 2024-03-26T06:41:09.122 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_commit": 75, 2024-03-26T06:41:09.123 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_split": 7, 2024-03-26T06:41:09.123 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_merge": 5, 2024-03-26T06:41:09.123 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes": 1490, 2024-03-26T06:41:09.123 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_top": 602, 2024-03-26T06:41:09.123 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_bottom": 835, 2024-03-26T06:41:09.123 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pin_tail": 53, 2024-03-26T06:41:09.123 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pinned": 629, 2024-03-26T06:41:09.123 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_expired": 179, 2024-03-26T06:41:09.123 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_with_caps": 152, 2024-03-26T06:41:09.124 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": 152, 2024-03-26T06:41:09.124 INFO:teuthology.orchestra.run.smithi037.stdout: "subtrees": 12, 2024-03-26T06:41:09.124 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse": 201475, 2024-03-26T06:41:09.124 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_hit": 111728, 2024-03-26T06:41:09.124 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_forward": 359, 2024-03-26T06:41:09.124 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_discover": 38, 2024-03-26T06:41:09.124 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_dir_fetch": 0, 2024-03-26T06:41:09.124 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_remote_ino": 0, 2024-03-26T06:41:09.125 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_lock": 1861, 2024-03-26T06:41:09.125 INFO:teuthology.orchestra.run.smithi037.stdout: "load_cent": 437935, 2024-03-26T06:41:09.125 INFO:teuthology.orchestra.run.smithi037.stdout: "q": 0, 2024-03-26T06:41:09.125 INFO:teuthology.orchestra.run.smithi037.stdout: "exported": 5, 2024-03-26T06:41:09.125 INFO:teuthology.orchestra.run.smithi037.stdout: "exported_inodes": 162, 2024-03-26T06:41:09.125 INFO:teuthology.orchestra.run.smithi037.stdout: "imported": 8, 2024-03-26T06:41:09.125 INFO:teuthology.orchestra.run.smithi037.stdout: "imported_inodes": 315, 2024-03-26T06:41:09.125 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_dir_fetch": 0, 2024-03-26T06:41:09.125 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_backtrace_fetch": 0, 2024-03-26T06:41:09.126 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_peer_discover": 0, 2024-03-26T06:41:09.126 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rfiles": 729, 2024-03-26T06:41:09.126 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rbytes": 211954046, 2024-03-26T06:41:09.126 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rsnaps": 6, 2024-03-26T06:41:09.126 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T06:41:09.126 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_set_tag": 0, 2024-03-26T06:41:09.126 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T06:41:09.126 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_inotable_repaired": 0, 2024-03-26T06:41:09.126 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_inodes": 0, 2024-03-26T06:41:09.127 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T06:41:09.127 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T06:41:09.127 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_file_inodes": 0, 2024-03-26T06:41:09.127 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_inode_file_caps": 3, 2024-03-26T06:41:09.127 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_revoke": 23982, 2024-03-26T06:41:09.127 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_grant": 72836, 2024-03-26T06:41:09.127 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T06:41:09.127 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T06:41:09.127 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T06:41:09.128 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps": 58006, 2024-03-26T06:41:09.128 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps_dirty": 20114, 2024-03-26T06:41:09.128 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_cap_release": 7309, 2024-03-26T06:41:09.128 INFO:teuthology.orchestra.run.smithi037.stdout: "process_request_cap_release": 86449 2024-03-26T06:41:09.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.128 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_cache": { 2024-03-26T06:41:09.128 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update": 18, 2024-03-26T06:41:09.128 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update_receipt": 154, 2024-03-26T06:41:09.129 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_try_discover": 92, 2024-03-26T06:41:09.129 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_send_discover": 154, 2024-03-26T06:41:09.129 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_handle_discover": 273, 2024-03-26T06:41:09.129 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays": 1, 2024-03-26T06:41:09.129 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_delayed": 0, 2024-03-26T06:41:09.129 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_enqueuing": 0, 2024-03-26T06:41:09.129 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_created": 19418, 2024-03-26T06:41:09.129 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_enqueued": 19417, 2024-03-26T06:41:09.129 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_reintegrated": 0, 2024-03-26T06:41:09.130 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_migrated": 0, 2024-03-26T06:41:09.130 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_processing": 0, 2024-03-26T06:41:09.130 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_enqueued": 0, 2024-03-26T06:41:09.130 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_prioritized": 0, 2024-03-26T06:41:09.130 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_started": 0, 2024-03-26T06:41:09.130 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_completed": 0, 2024-03-26T06:41:09.130 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T06:41:09.130 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_exportdir": 9, 2024-03-26T06:41:09.130 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_flush": 0, 2024-03-26T06:41:09.131 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragmentdir": 12, 2024-03-26T06:41:09.131 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragstats": 0, 2024-03-26T06:41:09.131 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_inodestats": 0 2024-03-26T06:41:09.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.131 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_log": { 2024-03-26T06:41:09.131 INFO:teuthology.orchestra.run.smithi037.stdout: "evlrg": 0, 2024-03-26T06:41:09.131 INFO:teuthology.orchestra.run.smithi037.stdout: "evadd": 103159, 2024-03-26T06:41:09.131 INFO:teuthology.orchestra.run.smithi037.stdout: "evex": 15966, 2024-03-26T06:41:09.131 INFO:teuthology.orchestra.run.smithi037.stdout: "evtrm": 12291, 2024-03-26T06:41:09.132 INFO:teuthology.orchestra.run.smithi037.stdout: "ev": 90868, 2024-03-26T06:41:09.132 INFO:teuthology.orchestra.run.smithi037.stdout: "evexg": 0, 2024-03-26T06:41:09.132 INFO:teuthology.orchestra.run.smithi037.stdout: "evexd": 3675, 2024-03-26T06:41:09.132 INFO:teuthology.orchestra.run.smithi037.stdout: "segadd": 146, 2024-03-26T06:41:09.132 INFO:teuthology.orchestra.run.smithi037.stdout: "segex": 18, 2024-03-26T06:41:09.132 INFO:teuthology.orchestra.run.smithi037.stdout: "segtrm": 14, 2024-03-26T06:41:09.132 INFO:teuthology.orchestra.run.smithi037.stdout: "seg": 132, 2024-03-26T06:41:09.132 INFO:teuthology.orchestra.run.smithi037.stdout: "segmjr": 8, 2024-03-26T06:41:09.132 INFO:teuthology.orchestra.run.smithi037.stdout: "segexg": 0, 2024-03-26T06:41:09.132 INFO:teuthology.orchestra.run.smithi037.stdout: "segexd": 4, 2024-03-26T06:41:09.133 INFO:teuthology.orchestra.run.smithi037.stdout: "expos": 22442169, 2024-03-26T06:41:09.133 INFO:teuthology.orchestra.run.smithi037.stdout: "wrpos": 392343734, 2024-03-26T06:41:09.133 INFO:teuthology.orchestra.run.smithi037.stdout: "rdpos": 0, 2024-03-26T06:41:09.133 INFO:teuthology.orchestra.run.smithi037.stdout: "jlat": { 2024-03-26T06:41:09.133 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 31156, 2024-03-26T06:41:09.133 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 203.922486041, 2024-03-26T06:41:09.133 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.006545207 2024-03-26T06:41:09.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.133 INFO:teuthology.orchestra.run.smithi037.stdout: "replayed": 0 2024-03-26T06:41:09.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.134 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_mem": { 2024-03-26T06:41:09.134 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 748, 2024-03-26T06:41:09.134 INFO:teuthology.orchestra.run.smithi037.stdout: "ino+": 20322, 2024-03-26T06:41:09.134 INFO:teuthology.orchestra.run.smithi037.stdout: "ino-": 19574, 2024-03-26T06:41:09.134 INFO:teuthology.orchestra.run.smithi037.stdout: "dir": 139, 2024-03-26T06:41:09.134 INFO:teuthology.orchestra.run.smithi037.stdout: "dir+": 256, 2024-03-26T06:41:09.134 INFO:teuthology.orchestra.run.smithi037.stdout: "dir-": 117, 2024-03-26T06:41:09.134 INFO:teuthology.orchestra.run.smithi037.stdout: "dn": 1521, 2024-03-26T06:41:09.135 INFO:teuthology.orchestra.run.smithi037.stdout: "dn+": 38908, 2024-03-26T06:41:09.135 INFO:teuthology.orchestra.run.smithi037.stdout: "dn-": 37387, 2024-03-26T06:41:09.135 INFO:teuthology.orchestra.run.smithi037.stdout: "cap": 122, 2024-03-26T06:41:09.135 INFO:teuthology.orchestra.run.smithi037.stdout: "cap+": 21661, 2024-03-26T06:41:09.135 INFO:teuthology.orchestra.run.smithi037.stdout: "cap-": 21539, 2024-03-26T06:41:09.135 INFO:teuthology.orchestra.run.smithi037.stdout: "rss": 77832, 2024-03-26T06:41:09.135 INFO:teuthology.orchestra.run.smithi037.stdout: "heap": 207228 2024-03-26T06:41:09.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.135 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_server": { 2024-03-26T06:41:09.135 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_client_request": 180837, 2024-03-26T06:41:09.136 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_server_request": 0, 2024-03-26T06:41:09.136 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_request": 123066, 2024-03-26T06:41:09.136 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_session": 3354, 2024-03-26T06:41:09.136 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_peer_request": 5019, 2024-03-26T06:41:09.136 INFO:teuthology.orchestra.run.smithi037.stdout: "req_create_latency": { 2024-03-26T06:41:09.136 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 19403, 2024-03-26T06:41:09.136 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 70.920720678, 2024-03-26T06:41:09.136 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.003655142 2024-03-26T06:41:09.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.137 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getattr_latency": { 2024-03-26T06:41:09.137 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 765, 2024-03-26T06:41:09.137 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 5.233846900, 2024-03-26T06:41:09.137 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.006841629 2024-03-26T06:41:09.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.137 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getfilelock_latency": { 2024-03-26T06:41:09.137 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.137 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.138 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.138 INFO:teuthology.orchestra.run.smithi037.stdout: "req_link_latency": { 2024-03-26T06:41:09.138 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.138 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.138 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.138 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookup_latency": { 2024-03-26T06:41:09.138 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 41965, 2024-03-26T06:41:09.139 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 99.641401162, 2024-03-26T06:41:09.139 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.002374392 2024-03-26T06:41:09.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.139 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookuphash_latency": { 2024-03-26T06:41:09.139 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.139 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.139 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.139 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupino_latency": { 2024-03-26T06:41:09.140 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.140 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.140 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.140 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupname_latency": { 2024-03-26T06:41:09.140 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.140 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.140 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.141 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupparent_latency": { 2024-03-26T06:41:09.141 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.141 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.141 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.141 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupsnap_latency": { 2024-03-26T06:41:09.141 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.141 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.141 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.142 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lssnap_latency": { 2024-03-26T06:41:09.143 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.143 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.143 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.143 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mkdir_latency": { 2024-03-26T06:41:09.143 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.143 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.143 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.144 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mknod_latency": { 2024-03-26T06:41:09.144 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.144 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.144 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.144 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mksnap_latency": { 2024-03-26T06:41:09.144 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.144 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.145 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.145 INFO:teuthology.orchestra.run.smithi037.stdout: "req_open_latency": { 2024-03-26T06:41:09.145 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.145 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.145 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.145 INFO:teuthology.orchestra.run.smithi037.stdout: "req_readdir_latency": { 2024-03-26T06:41:09.145 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 36586, 2024-03-26T06:41:09.145 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 220.723483924, 2024-03-26T06:41:09.146 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.006033003 2024-03-26T06:41:09.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.146 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rename_latency": { 2024-03-26T06:41:09.146 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 4037, 2024-03-26T06:41:09.146 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 16.982352069, 2024-03-26T06:41:09.146 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.004206676 2024-03-26T06:41:09.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.146 INFO:teuthology.orchestra.run.smithi037.stdout: "req_renamesnap_latency": { 2024-03-26T06:41:09.147 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.147 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.147 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.147 INFO:teuthology.orchestra.run.smithi037.stdout: "req_snapdiff_latency": { 2024-03-26T06:41:09.147 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.147 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.147 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.148 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmdir_latency": { 2024-03-26T06:41:09.148 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.148 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.148 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.148 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmsnap_latency": { 2024-03-26T06:41:09.148 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.148 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.148 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.149 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmxattr_latency": { 2024-03-26T06:41:09.149 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.149 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.149 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.149 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setattr_latency": { 2024-03-26T06:41:09.149 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 14, 2024-03-26T06:41:09.149 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.185855509, 2024-03-26T06:41:09.150 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.013275393 2024-03-26T06:41:09.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.150 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setdirlayout_latency": { 2024-03-26T06:41:09.150 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.150 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.150 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.150 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setfilelock_latency": { 2024-03-26T06:41:09.151 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 516, 2024-03-26T06:41:09.151 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.807428180, 2024-03-26T06:41:09.151 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001564783 2024-03-26T06:41:09.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.151 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setlayout_latency": { 2024-03-26T06:41:09.151 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.151 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.151 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.151 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setxattr_latency": { 2024-03-26T06:41:09.152 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.152 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.152 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.152 INFO:teuthology.orchestra.run.smithi037.stdout: "req_symlink_latency": { 2024-03-26T06:41:09.152 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.152 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.152 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.153 INFO:teuthology.orchestra.run.smithi037.stdout: "req_unlink_latency": { 2024-03-26T06:41:09.153 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 19418, 2024-03-26T06:41:09.153 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 59.525870719, 2024-03-26T06:41:09.153 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.003065499 2024-03-26T06:41:09.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.153 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_revoke_eviction": 0, 2024-03-26T06:41:09.153 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_acquisition_throttle": 0, 2024-03-26T06:41:09.153 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getvxattr_latency": { 2024-03-26T06:41:09.153 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.154 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.154 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.154 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:09.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.154 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_sessions": { 2024-03-26T06:41:09.154 INFO:teuthology.orchestra.run.smithi037.stdout: "session_count": 2, 2024-03-26T06:41:09.154 INFO:teuthology.orchestra.run.smithi037.stdout: "session_add": 4, 2024-03-26T06:41:09.154 INFO:teuthology.orchestra.run.smithi037.stdout: "session_remove": 2, 2024-03-26T06:41:09.154 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_open": 2, 2024-03-26T06:41:09.155 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_stale": 0, 2024-03-26T06:41:09.155 INFO:teuthology.orchestra.run.smithi037.stdout: "total_load": 27626, 2024-03-26T06:41:09.155 INFO:teuthology.orchestra.run.smithi037.stdout: "average_load": 13813, 2024-03-26T06:41:09.155 INFO:teuthology.orchestra.run.smithi037.stdout: "avg_session_uptime": 1026, 2024-03-26T06:41:09.155 INFO:teuthology.orchestra.run.smithi037.stdout: "mdthresh_evicted": 0 2024-03-26T06:41:09.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.155 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-03-26T06:41:09.155 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-03-26T06:41:09.156 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-03-26T06:41:09.156 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:09.156 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:09.156 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:09.156 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:09.156 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:09.156 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:09.156 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:09.156 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:09.157 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:09.157 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:09.157 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:09.157 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:09.157 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:09.157 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:09.157 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:09.157 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:09.157 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:09.157 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:09.158 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:09.158 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:09.158 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:09.158 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:09.158 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:09.158 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:09.158 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:09.158 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:09.159 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:09.159 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:09.159 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:09.159 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-03-26T06:41:09.159 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:09.159 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:09.159 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:09.159 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:09.159 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 365881, 2024-03-26T06:41:09.160 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 482, 2024-03-26T06:41:09.160 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 264, 2024-03-26T06:41:09.160 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 3, 2024-03-26T06:41:09.160 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-03-26T06:41:09.160 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-03-26T06:41:09.160 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:09.160 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:09.160 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:09.160 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:09.161 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 8152, 2024-03-26T06:41:09.161 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 170, 2024-03-26T06:41:09.161 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:09.161 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:09.161 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:09.161 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-03-26T06:41:09.161 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 3140552, 2024-03-26T06:41:09.161 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 28592, 2024-03-26T06:41:09.161 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:09.162 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-03-26T06:41:09.162 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:09.162 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-03-26T06:41:09.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.162 INFO:teuthology.orchestra.run.smithi037.stdout: "objecter": { 2024-03-26T06:41:09.162 INFO:teuthology.orchestra.run.smithi037.stdout: "op_active": 0, 2024-03-26T06:41:09.162 INFO:teuthology.orchestra.run.smithi037.stdout: "op_laggy": 0, 2024-03-26T06:41:09.162 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send": 53976, 2024-03-26T06:41:09.162 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send_bytes": 392622400, 2024-03-26T06:41:09.163 INFO:teuthology.orchestra.run.smithi037.stdout: "op_resend": 33, 2024-03-26T06:41:09.163 INFO:teuthology.orchestra.run.smithi037.stdout: "op_reply": 53976, 2024-03-26T06:41:09.163 INFO:teuthology.orchestra.run.smithi037.stdout: "op_latency": { 2024-03-26T06:41:09.163 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 53976, 2024-03-26T06:41:09.163 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 385.162654066, 2024-03-26T06:41:09.163 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.007135813 2024-03-26T06:41:09.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.163 INFO:teuthology.orchestra.run.smithi037.stdout: "op_inflight": 0, 2024-03-26T06:41:09.164 INFO:teuthology.orchestra.run.smithi037.stdout: "oplen_avg": { 2024-03-26T06:41:09.164 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 53976, 2024-03-26T06:41:09.164 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 55258 2024-03-26T06:41:09.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.164 INFO:teuthology.orchestra.run.smithi037.stdout: "op": 53976, 2024-03-26T06:41:09.164 INFO:teuthology.orchestra.run.smithi037.stdout: "op_r": 941, 2024-03-26T06:41:09.164 INFO:teuthology.orchestra.run.smithi037.stdout: "op_w": 53035, 2024-03-26T06:41:09.164 INFO:teuthology.orchestra.run.smithi037.stdout: "op_rmw": 0, 2024-03-26T06:41:09.165 INFO:teuthology.orchestra.run.smithi037.stdout: "op_pg": 0, 2024-03-26T06:41:09.165 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_stat": 1, 2024-03-26T06:41:09.165 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_create": 21, 2024-03-26T06:41:09.165 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_read": 941, 2024-03-26T06:41:09.165 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_write": 32097, 2024-03-26T06:41:09.165 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writefull": 557, 2024-03-26T06:41:09.165 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writesame": 0, 2024-03-26T06:41:09.165 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_append": 0, 2024-03-26T06:41:09.165 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_zero": 0, 2024-03-26T06:41:09.166 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_truncate": 7, 2024-03-26T06:41:09.166 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_delete": 19565, 2024-03-26T06:41:09.166 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_mapext": 0, 2024-03-26T06:41:09.166 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_sparse_read": 0, 2024-03-26T06:41:09.166 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_clonerange": 0, 2024-03-26T06:41:09.166 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_getxattr": 0, 2024-03-26T06:41:09.166 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_setxattr": 42, 2024-03-26T06:41:09.166 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_cmpxattr": 0, 2024-03-26T06:41:09.167 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_rmxattr": 0, 2024-03-26T06:41:09.167 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_resetxattrs": 0, 2024-03-26T06:41:09.167 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_call": 0, 2024-03-26T06:41:09.167 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_watch": 0, 2024-03-26T06:41:09.167 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_notify": 0, 2024-03-26T06:41:09.167 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T06:41:09.167 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls": 0, 2024-03-26T06:41:09.167 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls_filter": 0, 2024-03-26T06:41:09.167 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_other": 0, 2024-03-26T06:41:09.168 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_active": 0, 2024-03-26T06:41:09.168 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_send": 0, 2024-03-26T06:41:09.168 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_resend": 0, 2024-03-26T06:41:09.168 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_ping": 0, 2024-03-26T06:41:09.168 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_active": 0, 2024-03-26T06:41:09.168 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_send": 0, 2024-03-26T06:41:09.168 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_resend": 0, 2024-03-26T06:41:09.169 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_active": 0, 2024-03-26T06:41:09.169 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_send": 0, 2024-03-26T06:41:09.169 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_resend": 0, 2024-03-26T06:41:09.169 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_active": 0, 2024-03-26T06:41:09.169 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_send": 0, 2024-03-26T06:41:09.169 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_resend": 0, 2024-03-26T06:41:09.169 INFO:teuthology.orchestra.run.smithi037.stdout: "command_active": 0, 2024-03-26T06:41:09.169 INFO:teuthology.orchestra.run.smithi037.stdout: "command_send": 0, 2024-03-26T06:41:09.169 INFO:teuthology.orchestra.run.smithi037.stdout: "command_resend": 0, 2024-03-26T06:41:09.170 INFO:teuthology.orchestra.run.smithi037.stdout: "map_epoch": 115, 2024-03-26T06:41:09.170 INFO:teuthology.orchestra.run.smithi037.stdout: "map_full": 0, 2024-03-26T06:41:09.170 INFO:teuthology.orchestra.run.smithi037.stdout: "map_inc": 30, 2024-03-26T06:41:09.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_sessions": 12, 2024-03-26T06:41:09.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_open": 12, 2024-03-26T06:41:09.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_close": 0, 2024-03-26T06:41:09.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_laggy": 0, 2024-03-26T06:41:09.170 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_wr": 1504, 2024-03-26T06:41:09.171 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_rd": 0, 2024-03-26T06:41:09.171 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_del": 523 2024-03-26T06:41:09.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.171 INFO:teuthology.orchestra.run.smithi037.stdout: "oft": { 2024-03-26T06:41:09.171 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_objs": 1, 2024-03-26T06:41:09.171 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_kv_pairs": 0, 2024-03-26T06:41:09.171 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_updates": 0, 2024-03-26T06:41:09.171 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_removes": 0 2024-03-26T06:41:09.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.172 INFO:teuthology.orchestra.run.smithi037.stdout: "osd-slow-ops": { 2024-03-26T06:41:09.172 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_ops_count": 4 2024-03-26T06:41:09.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.172 INFO:teuthology.orchestra.run.smithi037.stdout: "purge_queue": { 2024-03-26T06:41:09.172 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops": 0, 2024-03-26T06:41:09.172 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops_high_water": 6, 2024-03-26T06:41:09.172 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing": 0, 2024-03-26T06:41:09.172 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_high_water": 6, 2024-03-26T06:41:09.173 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed_ops": 19417, 2024-03-26T06:41:09.173 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed": 19417, 2024-03-26T06:41:09.173 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_item_in_journal": 0 2024-03-26T06:41:09.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.173 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:09.173 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-26T06:41:09.173 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-03-26T06:41:09.173 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:09.173 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 444994, 2024-03-26T06:41:09.173 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 97079260, 2024-03-26T06:41:09.174 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:09.174 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 444994, 2024-03-26T06:41:09.174 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-26T06:41:09.174 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-26T06:41:09.174 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 444994, 2024-03-26T06:41:09.174 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 97079260, 2024-03-26T06:41:09.174 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:09.174 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.174 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.175 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.175 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:09.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.175 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_bytes": { 2024-03-26T06:41:09.175 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-26T06:41:09.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-03-26T06:41:09.175 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:09.175 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-03-26T06:41:09.175 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-03-26T06:41:09.176 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:09.176 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:09.176 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 53976, 2024-03-26T06:41:09.176 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 394585173, 2024-03-26T06:41:09.176 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 53976, 2024-03-26T06:41:09.176 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 394585173, 2024-03-26T06:41:09.176 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:09.176 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.176 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.177 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.177 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:09.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.177 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_ops": { 2024-03-26T06:41:09.177 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-26T06:41:09.177 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 1024, 2024-03-26T06:41:09.177 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:09.177 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-03-26T06:41:09.177 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-03-26T06:41:09.178 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:09.178 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:09.178 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 53976, 2024-03-26T06:41:09.178 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 53976, 2024-03-26T06:41:09.178 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 53976, 2024-03-26T06:41:09.178 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 53976, 2024-03-26T06:41:09.178 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:09.178 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.179 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.179 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.179 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:09.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.179 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle": { 2024-03-26T06:41:09.179 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-26T06:41:09.179 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-03-26T06:41:09.179 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:09.179 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 19417, 2024-03-26T06:41:09.179 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1962773, 2024-03-26T06:41:09.180 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:09.180 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 19417, 2024-03-26T06:41:09.180 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-26T06:41:09.180 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-26T06:41:09.180 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 941, 2024-03-26T06:41:09.180 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1962773, 2024-03-26T06:41:09.180 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:09.180 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.180 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.181 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.181 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:09.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:09.181 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle-0x55ef642bc6e0": { 2024-03-26T06:41:09.181 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 228267, 2024-03-26T06:41:09.181 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-03-26T06:41:09.181 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:09.181 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 103159, 2024-03-26T06:41:09.181 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 388149430, 2024-03-26T06:41:09.182 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:09.182 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 103159, 2024-03-26T06:41:09.182 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-26T06:41:09.182 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-26T06:41:09.182 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 31156, 2024-03-26T06:41:09.182 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 387921163, 2024-03-26T06:41:09.182 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:09.182 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:09.182 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:09.183 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:09.183 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:09.183 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:09.183 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:41:10.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:10 smithi037 ceph-mon[31974]: pgmap v575: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 741 op/s 2024-03-26T06:41:10.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:10 smithi149 ceph-mon[36681]: pgmap v575: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 741 op/s 2024-03-26T06:41:10.463 INFO:tasks.check_counter:Daemon mds.a mds.exported=5 2024-03-26T06:41:10.464 INFO:tasks.check_counter:Daemon mds.a mds.imported=8 2024-03-26T06:41:10.464 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-03-26T06:41:10.464 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-26T06:41:10.464 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T06:41:10.464 DEBUG:tasks.check_counter:Getting stats from d 2024-03-26T06:41:10.464 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-26T06:41:10.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:10 smithi066 ceph-mon[36504]: pgmap v575: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 741 op/s 2024-03-26T06:41:10.713 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:41:12.360 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:12 smithi037 ceph-mon[31974]: pgmap v576: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-03-26T06:41:12.360 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:41:12.361 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:12.361 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-03-26T06:41:12.361 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-03-26T06:41:12.361 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-03-26T06:41:12.361 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-03-26T06:41:12.361 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-03-26T06:41:12.361 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-03-26T06:41:12.362 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001087599, 2024-03-26T06:41:12.362 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T06:41:12.362 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000155756, 2024-03-26T06:41:12.362 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T06:41:12.362 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:12.362 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:12.362 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:12.362 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:12.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:12.363 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:12.363 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:12.363 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:12.363 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:12.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:12.363 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T06:41:12.363 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T06:41:12.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:12.364 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:12.364 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 3, 2024-03-26T06:41:12.364 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 254, 2024-03-26T06:41:12.364 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 480, 2024-03-26T06:41:12.364 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 277408, 2024-03-26T06:41:12.364 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 6, 2024-03-26T06:41:12.364 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-03-26T06:41:12.364 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.019746346, 2024-03-26T06:41:12.364 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.014396402, 2024-03-26T06:41:12.365 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002123999, 2024-03-26T06:41:12.365 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T06:41:12.365 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:12.365 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 251, 2024-03-26T06:41:12.365 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.017952843, 2024-03-26T06:41:12.365 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000071525 2024-03-26T06:41:12.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:12.365 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:12.366 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:12.366 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:12.366 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:12.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:12.366 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-26T06:41:12.366 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 277408 2024-03-26T06:41:12.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:12.366 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:12.367 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 346, 2024-03-26T06:41:12.367 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 333, 2024-03-26T06:41:12.367 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1611824, 2024-03-26T06:41:12.367 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 200160, 2024-03-26T06:41:12.367 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-03-26T06:41:12.367 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-03-26T06:41:12.367 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.057777870, 2024-03-26T06:41:12.367 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.019292083, 2024-03-26T06:41:12.368 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.041288067, 2024-03-26T06:41:12.368 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.006376592, 2024-03-26T06:41:12.368 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:12.368 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 333, 2024-03-26T06:41:12.368 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.014246061, 2024-03-26T06:41:12.368 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000042780 2024-03-26T06:41:12.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:12.368 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:12.369 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:12.369 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:12.369 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:12.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:12.369 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1611824, 2024-03-26T06:41:12.369 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 200160 2024-03-26T06:41:12.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:12.369 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-03-26T06:41:12.369 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-03-26T06:41:12.370 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-03-26T06:41:12.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:12.370 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-03-26T06:41:12.370 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-03-26T06:41:12.370 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-03-26T06:41:12.370 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:12.370 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:12.370 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:12.371 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:12.371 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:12.371 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:12.371 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:12.371 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:12.371 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:12.371 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:12.371 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:12.371 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:12.372 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:12.372 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:12.372 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:12.372 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:12.372 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:12.372 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:12.372 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:12.372 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:12.372 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:12.373 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:12.373 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:12.373 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:12.373 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:12.373 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:12.373 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:12.373 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:12.373 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:12.373 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-03-26T06:41:12.374 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:12.374 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:12.374 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:12.374 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:12.374 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148670, 2024-03-26T06:41:12.374 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-03-26T06:41:12.374 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-03-26T06:41:12.374 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-03-26T06:41:12.375 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-03-26T06:41:12.375 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-03-26T06:41:12.375 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:12.375 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:12.375 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:12.375 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:12.375 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-03-26T06:41:12.375 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-03-26T06:41:12.376 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:12.376 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:12.376 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:12.376 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-03-26T06:41:12.376 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-03-26T06:41:12.376 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-03-26T06:41:12.376 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:12.376 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-03-26T06:41:12.376 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:12.377 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-03-26T06:41:12.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:12.377 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:12.377 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-26T06:41:12.377 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-03-26T06:41:12.377 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:12.377 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 345, 2024-03-26T06:41:12.378 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1560561, 2024-03-26T06:41:12.378 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:12.378 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 345, 2024-03-26T06:41:12.378 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-26T06:41:12.378 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-26T06:41:12.378 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 345, 2024-03-26T06:41:12.378 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1560561, 2024-03-26T06:41:12.378 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:12.379 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:12.379 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:12.379 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:12.379 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:12.379 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:12.379 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:41:12.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:12 smithi149 ceph-mon[36681]: pgmap v576: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-03-26T06:41:12.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:12 smithi066 ceph-mon[36504]: pgmap v576: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-03-26T06:41:13.115 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-26T06:41:13.115 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-26T06:41:13.115 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-26T06:41:13.115 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-26T06:41:13.115 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-26T06:41:13.115 DEBUG:tasks.check_counter:Getting stats from g 2024-03-26T06:41:13.116 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-26T06:41:13.361 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:41:14.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:14 smithi149 ceph-mon[36681]: pgmap v577: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-03-26T06:41:14.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:14 smithi037 ceph-mon[31974]: pgmap v577: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-03-26T06:41:14.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:14 smithi066 ceph-mon[36504]: pgmap v577: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-03-26T06:41:15.178 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:41:15.178 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:15.178 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2277, 2024-03-26T06:41:15.178 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2222, 2024-03-26T06:41:15.178 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 90573471, 2024-03-26T06:41:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 794508, 2024-03-26T06:41:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 7, 2024-03-26T06:41:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-03-26T06:41:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.269295181, 2024-03-26T06:41:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.085452487, 2024-03-26T06:41:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.121313691, 2024-03-26T06:41:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.057703362, 2024-03-26T06:41:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2221, 2024-03-26T06:41:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.081865585, 2024-03-26T06:41:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000036859 2024-03-26T06:41:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1921200, 2024-03-26T06:41:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 203664 2024-03-26T06:41:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 1873, 2024-03-26T06:41:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2120, 2024-03-26T06:41:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 130502428, 2024-03-26T06:41:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 1285461, 2024-03-26T06:41:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 10, 2024-03-26T06:41:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 7, 2024-03-26T06:41:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.236764765, 2024-03-26T06:41:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.048203528, 2024-03-26T06:41:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.130877633, 2024-03-26T06:41:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.044494418, 2024-03-26T06:41:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2118, 2024-03-26T06:41:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.064851645, 2024-03-26T06:41:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000030619 2024-03-26T06:41:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-03-26T06:41:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 692016 2024-03-26T06:41:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 860, 2024-03-26T06:41:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 862, 2024-03-26T06:41:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 170287546, 2024-03-26T06:41:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 273492, 2024-03-26T06:41:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 7, 2024-03-26T06:41:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 5, 2024-03-26T06:41:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.173221518, 2024-03-26T06:41:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.017814419, 2024-03-26T06:41:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.127291469, 2024-03-26T06:41:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.021951546, 2024-03-26T06:41:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 860, 2024-03-26T06:41:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.027624265, 2024-03-26T06:41:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000032121 2024-03-26T06:41:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T06:41:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T06:41:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-03-26T06:41:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 1, 2024-03-26T06:41:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-03-26T06:41:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-MDSRank": { 2024-03-26T06:41:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-03-26T06:41:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-03-26T06:41:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 7586, 2024-03-26T06:41:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.170417443, 2024-03-26T06:41:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000022464 2024-03-26T06:41:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-PurgeQueue": { 2024-03-26T06:41:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-03-26T06:41:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-03-26T06:41:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-26T06:41:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "request": 0, 2024-03-26T06:41:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "reply": 0, 2024-03-26T06:41:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "reply_latency": { 2024-03-26T06:41:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_reply": 0, 2024-03-26T06:41:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "forward": 0, 2024-03-26T06:41:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_complete": 0, 2024-03-26T06:41:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_keys": 0, 2024-03-26T06:41:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_commit": 0, 2024-03-26T06:41:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_split": 0, 2024-03-26T06:41:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_merge": 0, 2024-03-26T06:41:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes": 20262, 2024-03-26T06:41:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_top": 1060, 2024-03-26T06:41:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_bottom": 19202, 2024-03-26T06:41:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pin_tail": 0, 2024-03-26T06:41:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pinned": 18748, 2024-03-26T06:41:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_expired": 0, 2024-03-26T06:41:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_with_caps": 0, 2024-03-26T06:41:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": 0, 2024-03-26T06:41:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "subtrees": 5, 2024-03-26T06:41:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse": 0, 2024-03-26T06:41:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_hit": 0, 2024-03-26T06:41:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_forward": 0, 2024-03-26T06:41:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_discover": 0, 2024-03-26T06:41:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_dir_fetch": 0, 2024-03-26T06:41:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_remote_ino": 0, 2024-03-26T06:41:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_lock": 0, 2024-03-26T06:41:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "load_cent": 0, 2024-03-26T06:41:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "q": 0, 2024-03-26T06:41:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "exported": 0, 2024-03-26T06:41:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "exported_inodes": 0, 2024-03-26T06:41:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "imported": 0, 2024-03-26T06:41:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "imported_inodes": 0, 2024-03-26T06:41:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_dir_fetch": 0, 2024-03-26T06:41:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_backtrace_fetch": 0, 2024-03-26T06:41:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_peer_discover": 0, 2024-03-26T06:41:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rfiles": 0, 2024-03-26T06:41:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rbytes": 0, 2024-03-26T06:41:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rsnaps": 0, 2024-03-26T06:41:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T06:41:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_set_tag": 0, 2024-03-26T06:41:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T06:41:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_inotable_repaired": 0, 2024-03-26T06:41:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_inodes": 0, 2024-03-26T06:41:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T06:41:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T06:41:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_file_inodes": 0, 2024-03-26T06:41:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_inode_file_caps": 0, 2024-03-26T06:41:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T06:41:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_grant": 0, 2024-03-26T06:41:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T06:41:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T06:41:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T06:41:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps": 0, 2024-03-26T06:41:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps_dirty": 0, 2024-03-26T06:41:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_cap_release": 0, 2024-03-26T06:41:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "process_request_cap_release": 0 2024-03-26T06:41:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_cache": { 2024-03-26T06:41:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update": 0, 2024-03-26T06:41:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update_receipt": 0, 2024-03-26T06:41:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_try_discover": 0, 2024-03-26T06:41:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_send_discover": 0, 2024-03-26T06:41:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_handle_discover": 0, 2024-03-26T06:41:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays": 0, 2024-03-26T06:41:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_delayed": 0, 2024-03-26T06:41:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_enqueuing": 0, 2024-03-26T06:41:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_created": 0, 2024-03-26T06:41:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_enqueued": 0, 2024-03-26T06:41:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_reintegrated": 0, 2024-03-26T06:41:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_migrated": 0, 2024-03-26T06:41:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_processing": 0, 2024-03-26T06:41:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_enqueued": 0, 2024-03-26T06:41:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_prioritized": 0, 2024-03-26T06:41:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_started": 0, 2024-03-26T06:41:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_completed": 0, 2024-03-26T06:41:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T06:41:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_exportdir": 0, 2024-03-26T06:41:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_flush": 0, 2024-03-26T06:41:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragmentdir": 0, 2024-03-26T06:41:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragstats": 0, 2024-03-26T06:41:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_inodestats": 0 2024-03-26T06:41:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_log": { 2024-03-26T06:41:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "evlrg": 0, 2024-03-26T06:41:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "evadd": 0, 2024-03-26T06:41:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "evex": 0, 2024-03-26T06:41:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "evtrm": 0, 2024-03-26T06:41:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "ev": 103161, 2024-03-26T06:41:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "evexg": 0, 2024-03-26T06:41:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "evexd": 0, 2024-03-26T06:41:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "segadd": 0, 2024-03-26T06:41:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "segex": 0, 2024-03-26T06:41:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "segtrm": 0, 2024-03-26T06:41:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "seg": 132, 2024-03-26T06:41:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "segmjr": 10, 2024-03-26T06:41:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "segexg": 0, 2024-03-26T06:41:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "segexd": 0, 2024-03-26T06:41:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "expos": 22442169, 2024-03-26T06:41:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "wrpos": 392355212, 2024-03-26T06:41:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "rdpos": 392349473, 2024-03-26T06:41:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "jlat": { 2024-03-26T06:41:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "replayed": 103161 2024-03-26T06:41:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_mem": { 2024-03-26T06:41:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 699, 2024-03-26T06:41:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "ino+": 20266, 2024-03-26T06:41:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "ino-": 19567, 2024-03-26T06:41:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "dir": 58, 2024-03-26T06:41:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "dir+": 110, 2024-03-26T06:41:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "dir-": 52, 2024-03-26T06:41:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "dn": 20262, 2024-03-26T06:41:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "dn+": 20433, 2024-03-26T06:41:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "dn-": 171, 2024-03-26T06:41:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "cap": 0, 2024-03-26T06:41:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "cap+": 0, 2024-03-26T06:41:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "cap-": 0, 2024-03-26T06:41:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "rss": 78060, 2024-03-26T06:41:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "heap": 215420 2024-03-26T06:41:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_server": { 2024-03-26T06:41:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_client_request": 0, 2024-03-26T06:41:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_server_request": 0, 2024-03-26T06:41:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_request": 0, 2024-03-26T06:41:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_session": 0, 2024-03-26T06:41:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_peer_request": 0, 2024-03-26T06:41:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "req_create_latency": { 2024-03-26T06:41:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getattr_latency": { 2024-03-26T06:41:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getfilelock_latency": { 2024-03-26T06:41:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: "req_link_latency": { 2024-03-26T06:41:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookup_latency": { 2024-03-26T06:41:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookuphash_latency": { 2024-03-26T06:41:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupino_latency": { 2024-03-26T06:41:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupname_latency": { 2024-03-26T06:41:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupparent_latency": { 2024-03-26T06:41:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupsnap_latency": { 2024-03-26T06:41:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lssnap_latency": { 2024-03-26T06:41:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mkdir_latency": { 2024-03-26T06:41:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mknod_latency": { 2024-03-26T06:41:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mksnap_latency": { 2024-03-26T06:41:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "req_open_latency": { 2024-03-26T06:41:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "req_readdir_latency": { 2024-03-26T06:41:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rename_latency": { 2024-03-26T06:41:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: "req_renamesnap_latency": { 2024-03-26T06:41:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "req_snapdiff_latency": { 2024-03-26T06:41:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmdir_latency": { 2024-03-26T06:41:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmsnap_latency": { 2024-03-26T06:41:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmxattr_latency": { 2024-03-26T06:41:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setattr_latency": { 2024-03-26T06:41:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setdirlayout_latency": { 2024-03-26T06:41:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setfilelock_latency": { 2024-03-26T06:41:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setlayout_latency": { 2024-03-26T06:41:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setxattr_latency": { 2024-03-26T06:41:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "req_symlink_latency": { 2024-03-26T06:41:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "req_unlink_latency": { 2024-03-26T06:41:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_revoke_eviction": 0, 2024-03-26T06:41:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_acquisition_throttle": 0, 2024-03-26T06:41:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getvxattr_latency": { 2024-03-26T06:41:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_sessions": { 2024-03-26T06:41:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "session_count": 2, 2024-03-26T06:41:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "session_add": 4, 2024-03-26T06:41:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "session_remove": 2, 2024-03-26T06:41:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_open": 2, 2024-03-26T06:41:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_stale": 0, 2024-03-26T06:41:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "total_load": 0, 2024-03-26T06:41:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "average_load": 0, 2024-03-26T06:41:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "avg_session_uptime": 4588, 2024-03-26T06:41:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "mdthresh_evicted": 0 2024-03-26T06:41:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-03-26T06:41:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-03-26T06:41:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-03-26T06:41:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-03-26T06:41:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 269004, 2024-03-26T06:41:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 878, 2024-03-26T06:41:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-03-26T06:41:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-03-26T06:41:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-03-26T06:41:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-03-26T06:41:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 8152, 2024-03-26T06:41:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 170, 2024-03-26T06:41:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-03-26T06:41:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 16126970, 2024-03-26T06:41:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 99692, 2024-03-26T06:41:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-03-26T06:41:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-03-26T06:41:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "objecter": { 2024-03-26T06:41:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "op_active": 0, 2024-03-26T06:41:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "op_laggy": 0, 2024-03-26T06:41:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send": 4584, 2024-03-26T06:41:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send_bytes": 21, 2024-03-26T06:41:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "op_resend": 3, 2024-03-26T06:41:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "op_reply": 4584, 2024-03-26T06:41:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "op_latency": { 2024-03-26T06:41:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 4584, 2024-03-26T06:41:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 7.440896634, 2024-03-26T06:41:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001623232 2024-03-26T06:41:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "op_inflight": 0, 2024-03-26T06:41:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: "oplen_avg": { 2024-03-26T06:41:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 4584, 2024-03-26T06:41:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 4585 2024-03-26T06:41:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: "op": 4584, 2024-03-26T06:41:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "op_r": 4584, 2024-03-26T06:41:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "op_w": 0, 2024-03-26T06:41:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "op_rmw": 0, 2024-03-26T06:41:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "op_pg": 0, 2024-03-26T06:41:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_stat": 2645, 2024-03-26T06:41:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_create": 0, 2024-03-26T06:41:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_read": 1937, 2024-03-26T06:41:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_write": 0, 2024-03-26T06:41:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writefull": 0, 2024-03-26T06:41:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writesame": 0, 2024-03-26T06:41:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_append": 0, 2024-03-26T06:41:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_zero": 0, 2024-03-26T06:41:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_truncate": 0, 2024-03-26T06:41:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_delete": 0, 2024-03-26T06:41:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_mapext": 0, 2024-03-26T06:41:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_sparse_read": 0, 2024-03-26T06:41:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_clonerange": 0, 2024-03-26T06:41:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_getxattr": 1, 2024-03-26T06:41:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_setxattr": 0, 2024-03-26T06:41:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_cmpxattr": 0, 2024-03-26T06:41:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_rmxattr": 0, 2024-03-26T06:41:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_resetxattrs": 0, 2024-03-26T06:41:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_call": 0, 2024-03-26T06:41:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_watch": 0, 2024-03-26T06:41:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_notify": 0, 2024-03-26T06:41:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T06:41:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls": 0, 2024-03-26T06:41:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls_filter": 0, 2024-03-26T06:41:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_other": 0, 2024-03-26T06:41:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_active": 0, 2024-03-26T06:41:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_send": 0, 2024-03-26T06:41:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_resend": 0, 2024-03-26T06:41:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_ping": 0, 2024-03-26T06:41:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_active": 0, 2024-03-26T06:41:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_send": 0, 2024-03-26T06:41:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_resend": 0, 2024-03-26T06:41:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_active": 0, 2024-03-26T06:41:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_send": 0, 2024-03-26T06:41:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_resend": 0, 2024-03-26T06:41:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_active": 0, 2024-03-26T06:41:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_send": 0, 2024-03-26T06:41:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_resend": 0, 2024-03-26T06:41:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "command_active": 0, 2024-03-26T06:41:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "command_send": 0, 2024-03-26T06:41:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "command_resend": 0, 2024-03-26T06:41:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "map_epoch": 116, 2024-03-26T06:41:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "map_full": 0, 2024-03-26T06:41:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "map_inc": 31, 2024-03-26T06:41:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_sessions": 11, 2024-03-26T06:41:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_open": 12, 2024-03-26T06:41:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_close": 1, 2024-03-26T06:41:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_laggy": 0, 2024-03-26T06:41:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_wr": 0, 2024-03-26T06:41:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_rd": 2, 2024-03-26T06:41:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_del": 0 2024-03-26T06:41:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "oft": { 2024-03-26T06:41:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_objs": 0, 2024-03-26T06:41:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_kv_pairs": 0, 2024-03-26T06:41:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_updates": 0, 2024-03-26T06:41:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_removes": 0 2024-03-26T06:41:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "osd-slow-ops": { 2024-03-26T06:41:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_ops_count": 0 2024-03-26T06:41:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "purge_queue": { 2024-03-26T06:41:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops": 0, 2024-03-26T06:41:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T06:41:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing": 0, 2024-03-26T06:41:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_high_water": 0, 2024-03-26T06:41:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed_ops": 0, 2024-03-26T06:41:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed": 0, 2024-03-26T06:41:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_item_in_journal": 0 2024-03-26T06:41:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-26T06:41:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-03-26T06:41:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 5006, 2024-03-26T06:41:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 390888795, 2024-03-26T06:41:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 5006, 2024-03-26T06:41:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-26T06:41:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-26T06:41:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 5006, 2024-03-26T06:41:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 390888795, 2024-03-26T06:41:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_bytes": { 2024-03-26T06:41:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-26T06:41:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-03-26T06:41:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-03-26T06:41:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-03-26T06:41:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 4584, 2024-03-26T06:41:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 388173649, 2024-03-26T06:41:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 4584, 2024-03-26T06:41:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 388173649, 2024-03-26T06:41:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_ops": { 2024-03-26T06:41:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-26T06:41:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 1024, 2024-03-26T06:41:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-03-26T06:41:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-03-26T06:41:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 4584, 2024-03-26T06:41:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 4584, 2024-03-26T06:41:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 4584, 2024-03-26T06:41:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 4584, 2024-03-26T06:41:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle": { 2024-03-26T06:41:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-26T06:41:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-03-26T06:41:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-03-26T06:41:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-03-26T06:41:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-26T06:41:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-26T06:41:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 0, 2024-03-26T06:41:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 0, 2024-03-26T06:41:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle-0x5642cd7b4960": { 2024-03-26T06:41:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-26T06:41:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-03-26T06:41:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-03-26T06:41:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-03-26T06:41:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-26T06:41:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-26T06:41:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 0, 2024-03-26T06:41:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 0, 2024-03-26T06:41:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:15.255 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:15.255 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:15.255 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:41:16.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:15 smithi066 ceph-mon[36504]: osdmap e116: 12 total, 12 up, 12 in 2024-03-26T06:41:16.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:15 smithi149 ceph-mon[36681]: osdmap e116: 12 total, 12 up, 12 in 2024-03-26T06:41:16.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:15 smithi037 ceph-mon[31974]: osdmap e116: 12 total, 12 up, 12 in 2024-03-26T06:41:16.259 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-26T06:41:16.259 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-26T06:41:16.259 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-03-26T06:41:16.260 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-26T06:41:16.260 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T06:41:16.260 DEBUG:tasks.check_counter:Getting stats from j 2024-03-26T06:41:16.260 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-26T06:41:16.510 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T06:41:16.907 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:16 smithi037 ceph-mon[31974]: pgmap v579: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-03-26T06:41:17.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:16 smithi066 ceph-mon[36504]: pgmap v579: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-03-26T06:41:17.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:16 smithi149 ceph-mon[36681]: pgmap v579: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-03-26T06:41:18.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:17 smithi037 ceph-mon[31974]: pgmap v580: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 24 op/s 2024-03-26T06:41:18.141 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-26T06:41:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 345, 2024-03-26T06:41:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 334, 2024-03-26T06:41:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1609712, 2024-03-26T06:41:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 201120, 2024-03-26T06:41:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-03-26T06:41:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-03-26T06:41:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.059630725, 2024-03-26T06:41:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.020025310, 2024-03-26T06:41:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.039920782, 2024-03-26T06:41:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.006832633, 2024-03-26T06:41:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 333, 2024-03-26T06:41:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.013033989, 2024-03-26T06:41:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000039141 2024-03-26T06:41:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1609712, 2024-03-26T06:41:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 201120 2024-03-26T06:41:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 7, 2024-03-26T06:41:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 258, 2024-03-26T06:41:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3472, 2024-03-26T06:41:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 278208, 2024-03-26T06:41:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 6, 2024-03-26T06:41:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-03-26T06:41:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.020580700, 2024-03-26T06:41:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.013999718, 2024-03-26T06:41:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.003383928, 2024-03-26T06:41:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T06:41:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 256, 2024-03-26T06:41:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.021557143, 2024-03-26T06:41:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000084207 2024-03-26T06:41:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-03-26T06:41:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 278208 2024-03-26T06:41:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-03-26T06:41:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-03-26T06:41:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-03-26T06:41:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-03-26T06:41:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-03-26T06:41:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-03-26T06:41:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.000426444, 2024-03-26T06:41:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T06:41:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000349447, 2024-03-26T06:41:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T06:41:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T06:41:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T06:41:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-03-26T06:41:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-03-26T06:41:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-03-26T06:41:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-03-26T06:41:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-03-26T06:41:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-03-26T06:41:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-03-26T06:41:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148714, 2024-03-26T06:41:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-03-26T06:41:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-03-26T06:41:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-03-26T06:41:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-03-26T06:41:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-03-26T06:41:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-03-26T06:41:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-03-26T06:41:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-03-26T06:41:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-03-26T06:41:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-03-26T06:41:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-03-26T06:41:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-03-26T06:41:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-26T06:41:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-03-26T06:41:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-03-26T06:41:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-03-26T06:41:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 348, 2024-03-26T06:41:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1561015, 2024-03-26T06:41:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 348, 2024-03-26T06:41:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-03-26T06:41:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-03-26T06:41:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 348, 2024-03-26T06:41:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1561015, 2024-03-26T06:41:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-03-26T06:41:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-03-26T06:41:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-03-26T06:41:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-03-26T06:41:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-26T06:41:18.159 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-26T06:41:18.161 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:17 smithi149 ceph-mon[36681]: pgmap v580: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 24 op/s 2024-03-26T06:41:18.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:17 smithi066 ceph-mon[36504]: pgmap v580: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 24 op/s 2024-03-26T06:41:18.824 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-26T06:41:18.824 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-26T06:41:18.824 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-26T06:41:18.825 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-26T06:41:18.825 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-26T06:41:18.825 DEBUG:tasks.check_counter:Getting stats from b 2024-03-26T06:41:18.825 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-26T06:41:19.242 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:41:20.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:20 smithi066 ceph-mon[36504]: pgmap v581: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 24 op/s 2024-03-26T06:41:20.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:20 smithi149 ceph-mon[36681]: pgmap v581: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 24 op/s 2024-03-26T06:41:20.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:20 smithi037 ceph-mon[31974]: pgmap v581: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 24 op/s 2024-03-26T06:41:21.328 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-26T06:41:21.329 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:21.329 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 115943, 2024-03-26T06:41:21.329 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 132240, 2024-03-26T06:41:21.329 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 30695365, 2024-03-26T06:41:21.329 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 238495527, 2024-03-26T06:41:21.329 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 21, 2024-03-26T06:41:21.330 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-03-26T06:41:21.330 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 10.016353664, 2024-03-26T06:41:21.330 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 4.358706842, 2024-03-26T06:41:21.330 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 4.807156908, 2024-03-26T06:41:21.330 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 1.352889270, 2024-03-26T06:41:21.330 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:21.330 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 132240, 2024-03-26T06:41:21.330 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 16.819193908, 2024-03-26T06:41:21.330 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000127186 2024-03-26T06:41:21.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.331 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:21.331 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 181186, 2024-03-26T06:41:21.331 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.039835249, 2024-03-26T06:41:21.331 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000219 2024-03-26T06:41:21.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.331 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 554928, 2024-03-26T06:41:21.331 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 178688 2024-03-26T06:41:21.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.332 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:21.332 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 360800, 2024-03-26T06:41:21.332 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 444491, 2024-03-26T06:41:21.332 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 102410689, 2024-03-26T06:41:21.332 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 453703345, 2024-03-26T06:41:21.332 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 171, 2024-03-26T06:41:21.332 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 5, 2024-03-26T06:41:21.332 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 22.308041543, 2024-03-26T06:41:21.332 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 9.592825047, 2024-03-26T06:41:21.333 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 13.378686582, 2024-03-26T06:41:21.333 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 1.775412129, 2024-03-26T06:41:21.333 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:21.333 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 444489, 2024-03-26T06:41:21.333 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 47.644721258, 2024-03-26T06:41:21.333 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000107189 2024-03-26T06:41:21.333 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.333 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:21.333 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 480436, 2024-03-26T06:41:21.334 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.127903285, 2024-03-26T06:41:21.334 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000266 2024-03-26T06:41:21.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.334 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 98176, 2024-03-26T06:41:21.334 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 162096 2024-03-26T06:41:21.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.334 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:21.334 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 188204, 2024-03-26T06:41:21.334 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 234603, 2024-03-26T06:41:21.335 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 50848273, 2024-03-26T06:41:21.335 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 184678680, 2024-03-26T06:41:21.335 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 9, 2024-03-26T06:41:21.335 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 8, 2024-03-26T06:41:21.335 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 14.669704406, 2024-03-26T06:41:21.335 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 6.405614308, 2024-03-26T06:41:21.335 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 8.440939964, 2024-03-26T06:41:21.335 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 1.044903988, 2024-03-26T06:41:21.335 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:21.335 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 234600, 2024-03-26T06:41:21.336 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 18.541589538, 2024-03-26T06:41:21.336 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000079034 2024-03-26T06:41:21.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.336 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:21.336 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 382343, 2024-03-26T06:41:21.336 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.075659341, 2024-03-26T06:41:21.336 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000197 2024-03-26T06:41:21.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.337 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1963504, 2024-03-26T06:41:21.337 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 699792 2024-03-26T06:41:21.337 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.337 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-03-26T06:41:21.337 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-03-26T06:41:21.337 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-03-26T06:41:21.337 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.337 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-03-26T06:41:21.337 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-03-26T06:41:21.338 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-03-26T06:41:21.338 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 94422, 2024-03-26T06:41:21.338 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 88.460797263, 2024-03-26T06:41:21.338 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000936866 2024-03-26T06:41:21.338 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:21.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.338 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-03-26T06:41:21.338 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-03-26T06:41:21.338 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-03-26T06:41:21.339 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 26613, 2024-03-26T06:41:21.339 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 5.393637451, 2024-03-26T06:41:21.339 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000202669 2024-03-26T06:41:21.339 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:21.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.339 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-03-26T06:41:21.339 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 163600, 2024-03-26T06:41:21.339 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 140662, 2024-03-26T06:41:21.339 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-03-26T06:41:21.339 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 140662, 2024-03-26T06:41:21.340 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 714.733888538, 2024-03-26T06:41:21.340 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.005081215 2024-03-26T06:41:21.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.340 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-03-26T06:41:21.340 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 22938, 2024-03-26T06:41:21.340 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 14, 2024-03-26T06:41:21.340 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-03-26T06:41:21.340 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 2083, 2024-03-26T06:41:21.341 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 9, 2024-03-26T06:41:21.341 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 5, 2024-03-26T06:41:21.341 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 1619, 2024-03-26T06:41:21.341 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 735, 2024-03-26T06:41:21.341 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 830, 2024-03-26T06:41:21.341 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 54, 2024-03-26T06:41:21.341 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 569, 2024-03-26T06:41:21.341 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 1545, 2024-03-26T06:41:21.341 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 178, 2024-03-26T06:41:21.341 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 178, 2024-03-26T06:41:21.342 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 22, 2024-03-26T06:41:21.342 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 244087, 2024-03-26T06:41:21.342 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 156157, 2024-03-26T06:41:21.342 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 22921, 2024-03-26T06:41:21.342 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 46, 2024-03-26T06:41:21.342 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-03-26T06:41:21.342 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-03-26T06:41:21.342 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 1401, 2024-03-26T06:41:21.343 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 64694, 2024-03-26T06:41:21.343 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 22, 2024-03-26T06:41:21.343 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 31, 2024-03-26T06:41:21.343 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 1211, 2024-03-26T06:41:21.343 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 16, 2024-03-26T06:41:21.343 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 648, 2024-03-26T06:41:21.343 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-03-26T06:41:21.343 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-03-26T06:41:21.343 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-03-26T06:41:21.344 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 701, 2024-03-26T06:41:21.344 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 221176120, 2024-03-26T06:41:21.344 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 6, 2024-03-26T06:41:21.344 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T06:41:21.344 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-03-26T06:41:21.344 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T06:41:21.344 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-03-26T06:41:21.344 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-03-26T06:41:21.344 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T06:41:21.345 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T06:41:21.345 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-03-26T06:41:21.345 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 16, 2024-03-26T06:41:21.345 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 37753, 2024-03-26T06:41:21.345 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 58839, 2024-03-26T06:41:21.345 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T06:41:21.345 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T06:41:21.345 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T06:41:21.345 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 62090, 2024-03-26T06:41:21.346 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 13435, 2024-03-26T06:41:21.346 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 6223, 2024-03-26T06:41:21.346 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 87608 2024-03-26T06:41:21.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.346 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-03-26T06:41:21.346 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 118, 2024-03-26T06:41:21.346 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 39, 2024-03-26T06:41:21.346 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 26, 2024-03-26T06:41:21.347 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 59, 2024-03-26T06:41:21.347 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 276, 2024-03-26T06:41:21.347 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-03-26T06:41:21.347 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-03-26T06:41:21.347 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-03-26T06:41:21.347 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 15225, 2024-03-26T06:41:21.347 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 15225, 2024-03-26T06:41:21.347 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-03-26T06:41:21.347 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-03-26T06:41:21.348 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-03-26T06:41:21.348 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-03-26T06:41:21.348 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-03-26T06:41:21.348 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-03-26T06:41:21.348 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-03-26T06:41:21.348 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T06:41:21.348 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 36, 2024-03-26T06:41:21.348 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-03-26T06:41:21.349 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 14, 2024-03-26T06:41:21.349 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-03-26T06:41:21.349 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-03-26T06:41:21.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.349 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-03-26T06:41:21.349 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-03-26T06:41:21.349 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 103437, 2024-03-26T06:41:21.349 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 102717, 2024-03-26T06:41:21.350 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 102717, 2024-03-26T06:41:21.350 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 720, 2024-03-26T06:41:21.350 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-03-26T06:41:21.350 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 2386, 2024-03-26T06:41:21.350 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 228, 2024-03-26T06:41:21.350 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 227, 2024-03-26T06:41:21.350 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 227, 2024-03-26T06:41:21.350 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 1, 2024-03-26T06:41:21.351 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 1, 2024-03-26T06:41:21.351 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-03-26T06:41:21.351 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 4, 2024-03-26T06:41:21.351 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 459044566, 2024-03-26T06:41:21.351 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 460992286, 2024-03-26T06:41:21.351 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 0, 2024-03-26T06:41:21.351 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-03-26T06:41:21.351 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 49380, 2024-03-26T06:41:21.351 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 260.430674495, 2024-03-26T06:41:21.352 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.005274011 2024-03-26T06:41:21.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.352 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 0 2024-03-26T06:41:21.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.352 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-03-26T06:41:21.352 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 882, 2024-03-26T06:41:21.352 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 17272, 2024-03-26T06:41:21.352 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 16390, 2024-03-26T06:41:21.353 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 162, 2024-03-26T06:41:21.353 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 315, 2024-03-26T06:41:21.353 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 153, 2024-03-26T06:41:21.353 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 1620, 2024-03-26T06:41:21.353 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 35550, 2024-03-26T06:41:21.353 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 33930, 2024-03-26T06:41:21.353 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 178, 2024-03-26T06:41:21.353 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 18187, 2024-03-26T06:41:21.354 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 18009, 2024-03-26T06:41:21.354 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 77132, 2024-03-26T06:41:21.354 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 215420 2024-03-26T06:41:21.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.354 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-03-26T06:41:21.354 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 231233, 2024-03-26T06:41:21.354 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 22768, 2024-03-26T06:41:21.354 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 163600, 2024-03-26T06:41:21.354 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 2137, 2024-03-26T06:41:21.355 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 47014, 2024-03-26T06:41:21.355 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-03-26T06:41:21.355 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 15941, 2024-03-26T06:41:21.355 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 93.086167760, 2024-03-26T06:41:21.355 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.005839418 2024-03-26T06:41:21.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.355 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-03-26T06:41:21.355 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 28571, 2024-03-26T06:41:21.356 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 196.177509489, 2024-03-26T06:41:21.356 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.006866315 2024-03-26T06:41:21.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.356 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-03-26T06:41:21.356 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.356 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.356 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.356 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-03-26T06:41:21.357 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.357 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.357 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.357 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-03-26T06:41:21.357 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 30287, 2024-03-26T06:41:21.357 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 128.496781648, 2024-03-26T06:41:21.357 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.004242638 2024-03-26T06:41:21.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.358 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-03-26T06:41:21.358 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.358 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.358 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.358 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-03-26T06:41:21.358 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1, 2024-03-26T06:41:21.358 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000186957, 2024-03-26T06:41:21.358 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000186957 2024-03-26T06:41:21.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.359 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-03-26T06:41:21.359 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.359 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.359 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.359 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-03-26T06:41:21.359 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.359 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.360 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.360 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-03-26T06:41:21.360 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.360 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.361 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.361 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-03-26T06:41:21.361 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 18, 2024-03-26T06:41:21.361 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.009170554, 2024-03-26T06:41:21.361 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000509475 2024-03-26T06:41:21.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.362 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-03-26T06:41:21.362 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 119, 2024-03-26T06:41:21.362 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.636208087, 2024-03-26T06:41:21.362 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.005346286 2024-03-26T06:41:21.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.362 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-03-26T06:41:21.362 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.363 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.363 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.363 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-03-26T06:41:21.363 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 18, 2024-03-26T06:41:21.363 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 2.938298470, 2024-03-26T06:41:21.363 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.163238803 2024-03-26T06:41:21.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.363 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-03-26T06:41:21.364 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 9, 2024-03-26T06:41:21.364 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.011902305, 2024-03-26T06:41:21.364 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001322478 2024-03-26T06:41:21.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.364 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-03-26T06:41:21.364 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 47232, 2024-03-26T06:41:21.364 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 175.916944364, 2024-03-26T06:41:21.364 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.003724528 2024-03-26T06:41:21.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.365 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-03-26T06:41:21.365 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2770, 2024-03-26T06:41:21.365 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 31.013609907, 2024-03-26T06:41:21.365 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.011196249 2024-03-26T06:41:21.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.365 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-03-26T06:41:21.365 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.366 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.366 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.366 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-03-26T06:41:21.366 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.366 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.366 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.367 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-03-26T06:41:21.367 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1, 2024-03-26T06:41:21.367 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000917178, 2024-03-26T06:41:21.367 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000917178 2024-03-26T06:41:21.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.367 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-03-26T06:41:21.367 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 12, 2024-03-26T06:41:21.367 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 10.231810954, 2024-03-26T06:41:21.367 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.852650912 2024-03-26T06:41:21.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.368 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-03-26T06:41:21.368 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.368 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.368 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.368 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-03-26T06:41:21.368 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 7, 2024-03-26T06:41:21.368 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.174840083, 2024-03-26T06:41:21.369 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.024977154 2024-03-26T06:41:21.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.369 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-03-26T06:41:21.369 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.369 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.369 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.369 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-03-26T06:41:21.370 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 444, 2024-03-26T06:41:21.370 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.652571701, 2024-03-26T06:41:21.370 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001469756 2024-03-26T06:41:21.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.370 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-03-26T06:41:21.370 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.370 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.370 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.371 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-03-26T06:41:21.371 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 7, 2024-03-26T06:41:21.371 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.019703337, 2024-03-26T06:41:21.371 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.002814762 2024-03-26T06:41:21.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.371 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-03-26T06:41:21.371 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.371 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.372 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.372 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-03-26T06:41:21.372 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 15225, 2024-03-26T06:41:21.372 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 75.367265744, 2024-03-26T06:41:21.372 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.004950230 2024-03-26T06:41:21.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.372 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-03-26T06:41:21.373 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-03-26T06:41:21.373 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-03-26T06:41:21.373 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.373 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.373 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.373 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:21.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.373 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-03-26T06:41:21.373 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 2, 2024-03-26T06:41:21.374 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 4, 2024-03-26T06:41:21.374 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 2, 2024-03-26T06:41:21.374 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 2, 2024-03-26T06:41:21.374 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-03-26T06:41:21.374 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 24800, 2024-03-26T06:41:21.374 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 12400, 2024-03-26T06:41:21.374 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 1074, 2024-03-26T06:41:21.374 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-03-26T06:41:21.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.375 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-03-26T06:41:21.375 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 360, 2024-03-26T06:41:21.375 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 360, 2024-03-26T06:41:21.375 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:21.375 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:21.375 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:21.375 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:21.375 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:21.375 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:21.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:21.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:21.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:21.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:21.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:21.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:21.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:21.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:21.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:21.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:21.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:21.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:21.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:21.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:21.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:21.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:21.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:21.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:21.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:21.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:21.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:21.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:21.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:21.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-03-26T06:41:21.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:21.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:21.379 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:21.379 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:21.379 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 330086, 2024-03-26T06:41:21.379 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 710, 2024-03-26T06:41:21.379 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 176, 2024-03-26T06:41:21.379 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 2, 2024-03-26T06:41:21.379 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-03-26T06:41:21.379 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-03-26T06:41:21.379 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:21.379 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:21.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:21.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:21.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 8032, 2024-03-26T06:41:21.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 167, 2024-03-26T06:41:21.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:21.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:21.380 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:21.380 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-03-26T06:41:21.380 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 3714757, 2024-03-26T06:41:21.381 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 35528, 2024-03-26T06:41:21.381 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:21.381 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-03-26T06:41:21.381 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:21.381 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-03-26T06:41:21.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.381 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-03-26T06:41:21.381 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-03-26T06:41:21.381 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-03-26T06:41:21.382 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 79139, 2024-03-26T06:41:21.382 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 476412431, 2024-03-26T06:41:21.382 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 37, 2024-03-26T06:41:21.382 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 79139, 2024-03-26T06:41:21.382 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-03-26T06:41:21.382 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 79139, 2024-03-26T06:41:21.382 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 1144.107260729, 2024-03-26T06:41:21.382 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.014456933 2024-03-26T06:41:21.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.383 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-03-26T06:41:21.383 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-03-26T06:41:21.383 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 79139, 2024-03-26T06:41:21.383 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 102618 2024-03-26T06:41:21.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.383 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 79139, 2024-03-26T06:41:21.383 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 636, 2024-03-26T06:41:21.384 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 78503, 2024-03-26T06:41:21.384 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-03-26T06:41:21.384 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-03-26T06:41:21.384 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 1962, 2024-03-26T06:41:21.384 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 8949, 2024-03-26T06:41:21.384 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 602, 2024-03-26T06:41:21.384 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 49982, 2024-03-26T06:41:21.384 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 659, 2024-03-26T06:41:21.384 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-03-26T06:41:21.385 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-03-26T06:41:21.385 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-03-26T06:41:21.385 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 9, 2024-03-26T06:41:21.385 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 15993, 2024-03-26T06:41:21.385 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-03-26T06:41:21.385 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-03-26T06:41:21.385 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-03-26T06:41:21.385 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 2, 2024-03-26T06:41:21.386 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 17898, 2024-03-26T06:41:21.386 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-03-26T06:41:21.386 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-03-26T06:41:21.386 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-03-26T06:41:21.386 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-03-26T06:41:21.386 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-03-26T06:41:21.386 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-03-26T06:41:21.386 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T06:41:21.386 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-03-26T06:41:21.387 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-03-26T06:41:21.387 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-03-26T06:41:21.387 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-03-26T06:41:21.387 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-03-26T06:41:21.387 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-03-26T06:41:21.387 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-03-26T06:41:21.387 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-03-26T06:41:21.387 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-03-26T06:41:21.387 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-03-26T06:41:21.388 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-03-26T06:41:21.388 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-03-26T06:41:21.388 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-03-26T06:41:21.388 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-03-26T06:41:21.388 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-03-26T06:41:21.388 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-03-26T06:41:21.388 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-03-26T06:41:21.388 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-03-26T06:41:21.388 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-03-26T06:41:21.389 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 116, 2024-03-26T06:41:21.389 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-03-26T06:41:21.389 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 32, 2024-03-26T06:41:21.389 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 12, 2024-03-26T06:41:21.389 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 12, 2024-03-26T06:41:21.389 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2024-03-26T06:41:21.389 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-03-26T06:41:21.389 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 5265, 2024-03-26T06:41:21.389 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 48, 2024-03-26T06:41:21.390 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 1249 2024-03-26T06:41:21.390 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.390 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-03-26T06:41:21.390 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 1, 2024-03-26T06:41:21.390 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-03-26T06:41:21.390 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-03-26T06:41:21.390 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-03-26T06:41:21.390 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.391 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-03-26T06:41:21.391 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-03-26T06:41:21.391 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.391 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-03-26T06:41:21.391 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-03-26T06:41:21.391 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 6, 2024-03-26T06:41:21.391 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-03-26T06:41:21.391 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 6, 2024-03-26T06:41:21.391 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 15225, 2024-03-26T06:41:21.391 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 15225, 2024-03-26T06:41:21.392 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-03-26T06:41:21.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.392 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:21.392 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:21.392 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-03-26T06:41:21.392 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:21.392 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 664943, 2024-03-26T06:41:21.392 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 127848464, 2024-03-26T06:41:21.392 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:21.393 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 664943, 2024-03-26T06:41:21.393 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:21.393 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:21.393 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 664943, 2024-03-26T06:41:21.393 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 127848464, 2024-03-26T06:41:21.393 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:21.393 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.393 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.394 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.394 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:21.394 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.394 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-03-26T06:41:21.394 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:21.394 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-03-26T06:41:21.394 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:21.394 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-03-26T06:41:21.395 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-03-26T06:41:21.395 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:21.395 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:21.395 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 79139, 2024-03-26T06:41:21.395 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 477950678, 2024-03-26T06:41:21.395 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 79139, 2024-03-26T06:41:21.395 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 477950678, 2024-03-26T06:41:21.395 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:21.395 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.395 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.396 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.396 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:21.396 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.396 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-03-26T06:41:21.396 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:21.396 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-03-26T06:41:21.396 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:21.396 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-03-26T06:41:21.396 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-03-26T06:41:21.397 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:21.397 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:21.397 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 79139, 2024-03-26T06:41:21.397 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 79139, 2024-03-26T06:41:21.397 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 79139, 2024-03-26T06:41:21.397 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 79139, 2024-03-26T06:41:21.397 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:21.397 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.397 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.398 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.398 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:21.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.398 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-03-26T06:41:21.398 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:21.398 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-03-26T06:41:21.398 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:21.398 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 15225, 2024-03-26T06:41:21.398 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1539061, 2024-03-26T06:41:21.399 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:21.399 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 15225, 2024-03-26T06:41:21.399 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:21.399 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:21.399 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 602, 2024-03-26T06:41:21.399 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1539061, 2024-03-26T06:41:21.399 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:21.399 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.399 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.400 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.400 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:21.400 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:21.400 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x5572a078c6e0": { 2024-03-26T06:41:21.400 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:21.400 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-03-26T06:41:21.400 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:21.400 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 103437, 2024-03-26T06:41:21.400 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 456797982, 2024-03-26T06:41:21.401 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:21.401 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 103437, 2024-03-26T06:41:21.401 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:21.401 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:21.401 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 49380, 2024-03-26T06:41:21.401 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 456797982, 2024-03-26T06:41:21.401 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:21.401 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:21.401 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:21.401 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:21.402 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:21.402 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:21.402 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-26T06:41:22.350 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:22 smithi066 ceph-mon[36504]: pgmap v582: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-03-26T06:41:22.413 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:22 smithi149 ceph-mon[36681]: pgmap v582: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-03-26T06:41:22.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:22 smithi037 ceph-mon[31974]: pgmap v582: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-03-26T06:41:23.591 INFO:tasks.check_counter:Daemon mds.b mds.exported=31 2024-03-26T06:41:23.591 INFO:tasks.check_counter:Daemon mds.b mds.imported=16 2024-03-26T06:41:23.591 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-26T06:41:23.592 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=18 2024-03-26T06:41:23.592 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=12 2024-03-26T06:41:23.592 DEBUG:tasks.check_counter:Getting stats from e 2024-03-26T06:41:23.592 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-26T06:41:23.860 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:41:24.235 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:24 smithi066 ceph-mon[36504]: pgmap v583: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-03-26T06:41:24.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:24 smithi149 ceph-mon[36681]: pgmap v583: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-03-26T06:41:24.453 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:24 smithi037 ceph-mon[31974]: pgmap v583: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-03-26T06:41:25.608 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-26T06:41:25.609 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:25.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 122533, 2024-03-26T06:41:25.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 101198, 2024-03-26T06:41:25.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 38744916, 2024-03-26T06:41:25.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 144258164, 2024-03-26T06:41:25.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 12, 2024-03-26T06:41:25.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 6, 2024-03-26T06:41:25.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 9.160587561, 2024-03-26T06:41:25.610 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 4.157651272, 2024-03-26T06:41:25.610 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 5.241007603, 2024-03-26T06:41:25.610 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.769652447, 2024-03-26T06:41:25.610 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:25.610 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 101196, 2024-03-26T06:41:25.610 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 6.739396176, 2024-03-26T06:41:25.610 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000066597 2024-03-26T06:41:25.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.611 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:25.611 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 190809, 2024-03-26T06:41:25.611 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.035944989, 2024-03-26T06:41:25.611 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000188 2024-03-26T06:41:25.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.611 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 5216, 2024-03-26T06:41:25.611 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 21792 2024-03-26T06:41:25.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.612 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:25.612 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 50659, 2024-03-26T06:41:25.612 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 51109, 2024-03-26T06:41:25.612 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 14462204, 2024-03-26T06:41:25.612 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 176657529, 2024-03-26T06:41:25.612 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 13, 2024-03-26T06:41:25.612 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 7, 2024-03-26T06:41:25.612 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 5.763652026, 2024-03-26T06:41:25.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 2.070026973, 2024-03-26T06:41:25.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 1.976593400, 2024-03-26T06:41:25.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 1.810559144, 2024-03-26T06:41:25.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:25.613 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 51109, 2024-03-26T06:41:25.613 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 6.466142905, 2024-03-26T06:41:25.613 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000126516 2024-03-26T06:41:25.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:25.614 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 55339, 2024-03-26T06:41:25.614 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.014171427, 2024-03-26T06:41:25.614 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000256 2024-03-26T06:41:25.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.614 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 4240, 2024-03-26T06:41:25.614 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 611648 2024-03-26T06:41:25.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.614 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:25.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 277844, 2024-03-26T06:41:25.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 378032, 2024-03-26T06:41:25.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 88210605, 2024-03-26T06:41:25.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 306668914, 2024-03-26T06:41:25.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 15, 2024-03-26T06:41:25.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 8, 2024-03-26T06:41:25.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 17.978514176, 2024-03-26T06:41:25.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 8.267011514, 2024-03-26T06:41:25.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 11.355281745, 2024-03-26T06:41:25.616 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.741302600, 2024-03-26T06:41:25.616 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:25.616 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 378029, 2024-03-26T06:41:25.616 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 44.316364421, 2024-03-26T06:41:25.616 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000117230 2024-03-26T06:41:25.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.616 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:25.616 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 405228, 2024-03-26T06:41:25.616 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.108667834, 2024-03-26T06:41:25.617 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000268 2024-03-26T06:41:25.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.617 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1953296, 2024-03-26T06:41:25.617 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 214848 2024-03-26T06:41:25.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.617 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-03-26T06:41:25.617 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-03-26T06:41:25.617 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-03-26T06:41:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-03-26T06:41:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-03-26T06:41:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-03-26T06:41:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 48570, 2024-03-26T06:41:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 75.099209099, 2024-03-26T06:41:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001546205 2024-03-26T06:41:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-03-26T06:41:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-03-26T06:41:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-03-26T06:41:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 34666, 2024-03-26T06:41:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 6.519143457, 2024-03-26T06:41:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000188055 2024-03-26T06:41:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-03-26T06:41:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 129582, 2024-03-26T06:41:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 119480, 2024-03-26T06:41:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-03-26T06:41:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 119480, 2024-03-26T06:41:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 793.304036373, 2024-03-26T06:41:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.006639638 2024-03-26T06:41:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-03-26T06:41:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 10102, 2024-03-26T06:41:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-03-26T06:41:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-03-26T06:41:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 38, 2024-03-26T06:41:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 3, 2024-03-26T06:41:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 1, 2024-03-26T06:41:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 1135, 2024-03-26T06:41:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 282, 2024-03-26T06:41:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 803, 2024-03-26T06:41:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 50, 2024-03-26T06:41:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 723, 2024-03-26T06:41:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 949, 2024-03-26T06:41:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 156, 2024-03-26T06:41:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 157, 2024-03-26T06:41:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 18, 2024-03-26T06:41:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 197289, 2024-03-26T06:41:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 101572, 2024-03-26T06:41:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 10095, 2024-03-26T06:41:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 260, 2024-03-26T06:41:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-03-26T06:41:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-03-26T06:41:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 1126, 2024-03-26T06:41:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 48061, 2024-03-26T06:41:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-03-26T06:41:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 6, 2024-03-26T06:41:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 187, 2024-03-26T06:41:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 14, 2024-03-26T06:41:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 406, 2024-03-26T06:41:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-03-26T06:41:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-03-26T06:41:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-03-26T06:41:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 709, 2024-03-26T06:41:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 220974521, 2024-03-26T06:41:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 6, 2024-03-26T06:41:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T06:41:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-03-26T06:41:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T06:41:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-03-26T06:41:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-03-26T06:41:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T06:41:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T06:41:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-03-26T06:41:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 2, 2024-03-26T06:41:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 30294, 2024-03-26T06:41:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 84223, 2024-03-26T06:41:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T06:41:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-03-26T06:41:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T06:41:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 52429, 2024-03-26T06:41:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 12105, 2024-03-26T06:41:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 5088, 2024-03-26T06:41:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 86473 2024-03-26T06:41:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-03-26T06:41:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 42, 2024-03-26T06:41:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 145, 2024-03-26T06:41:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 88, 2024-03-26T06:41:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 365, 2024-03-26T06:41:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 94, 2024-03-26T06:41:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 3, 2024-03-26T06:41:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-03-26T06:41:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-03-26T06:41:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 19377, 2024-03-26T06:41:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 19374, 2024-03-26T06:41:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-03-26T06:41:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-03-26T06:41:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-03-26T06:41:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-03-26T06:41:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-03-26T06:41:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-03-26T06:41:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-03-26T06:41:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T06:41:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 7, 2024-03-26T06:41:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-03-26T06:41:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 4, 2024-03-26T06:41:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-03-26T06:41:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-03-26T06:41:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-03-26T06:41:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-03-26T06:41:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 92762, 2024-03-26T06:41:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-03-26T06:41:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-03-26T06:41:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 92762, 2024-03-26T06:41:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-03-26T06:41:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-03-26T06:41:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 127, 2024-03-26T06:41:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-03-26T06:41:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-03-26T06:41:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 127, 2024-03-26T06:41:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 9, 2024-03-26T06:41:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-03-26T06:41:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-03-26T06:41:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2024-03-26T06:41:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 349671629, 2024-03-26T06:41:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 0, 2024-03-26T06:41:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-03-26T06:41:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 26207, 2024-03-26T06:41:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 169.633571243, 2024-03-26T06:41:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.006472834 2024-03-26T06:41:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 0 2024-03-26T06:41:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-03-26T06:41:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 824, 2024-03-26T06:41:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 20840, 2024-03-26T06:41:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 20016, 2024-03-26T06:41:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 147, 2024-03-26T06:41:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 259, 2024-03-26T06:41:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 112, 2024-03-26T06:41:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 1135, 2024-03-26T06:41:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 39292, 2024-03-26T06:41:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 38157, 2024-03-26T06:41:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 157, 2024-03-26T06:41:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 21590, 2024-03-26T06:41:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 21433, 2024-03-26T06:41:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 73376, 2024-03-26T06:41:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 207228 2024-03-26T06:41:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-03-26T06:41:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 180161, 2024-03-26T06:41:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 547, 2024-03-26T06:41:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 129582, 2024-03-26T06:41:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 2041, 2024-03-26T06:41:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 6576, 2024-03-26T06:41:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-03-26T06:41:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 19369, 2024-03-26T06:41:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 72.864179090, 2024-03-26T06:41:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.003761896 2024-03-26T06:41:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-03-26T06:41:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 165, 2024-03-26T06:41:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.234182310, 2024-03-26T06:41:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001419286 2024-03-26T06:41:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-03-26T06:41:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-03-26T06:41:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-03-26T06:41:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 37156, 2024-03-26T06:41:25.638 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 409.585903719, 2024-03-26T06:41:25.638 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.011023412 2024-03-26T06:41:25.638 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.638 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-03-26T06:41:25.638 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.638 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.638 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.638 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-03-26T06:41:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1, 2024-03-26T06:41:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000103309, 2024-03-26T06:41:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000103309 2024-03-26T06:41:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-03-26T06:41:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-03-26T06:41:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-03-26T06:41:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-03-26T06:41:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-03-26T06:41:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-03-26T06:41:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-03-26T06:41:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-03-26T06:41:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-03-26T06:41:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 40039, 2024-03-26T06:41:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 228.153780313, 2024-03-26T06:41:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.005698288 2024-03-26T06:41:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-03-26T06:41:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2921, 2024-03-26T06:41:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 12.217643998, 2024-03-26T06:41:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.004182692 2024-03-26T06:41:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-03-26T06:41:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-03-26T06:41:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-03-26T06:41:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-03-26T06:41:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-03-26T06:41:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-03-26T06:41:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 8, 2024-03-26T06:41:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.115837237, 2024-03-26T06:41:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.014479654 2024-03-26T06:41:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-03-26T06:41:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-03-26T06:41:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 444, 2024-03-26T06:41:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 1.462088472, 2024-03-26T06:41:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.003292992 2024-03-26T06:41:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-03-26T06:41:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-03-26T06:41:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-03-26T06:41:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-03-26T06:41:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 19377, 2024-03-26T06:41:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 68.670317925, 2024-03-26T06:41:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.003543908 2024-03-26T06:41:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-03-26T06:41:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-03-26T06:41:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-03-26T06:41:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-03-26T06:41:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 2, 2024-03-26T06:41:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 5, 2024-03-26T06:41:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 3, 2024-03-26T06:41:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 2, 2024-03-26T06:41:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-03-26T06:41:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 23565, 2024-03-26T06:41:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 11782, 2024-03-26T06:41:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 1074, 2024-03-26T06:41:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-03-26T06:41:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-03-26T06:41:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-03-26T06:41:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-03-26T06:41:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-03-26T06:41:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 359166, 2024-03-26T06:41:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 578, 2024-03-26T06:41:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 176, 2024-03-26T06:41:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 2, 2024-03-26T06:41:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-03-26T06:41:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-03-26T06:41:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 8032, 2024-03-26T06:41:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 167, 2024-03-26T06:41:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-03-26T06:41:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 3129452, 2024-03-26T06:41:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 26953, 2024-03-26T06:41:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-03-26T06:41:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-03-26T06:41:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-03-26T06:41:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-03-26T06:41:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-03-26T06:41:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 48670, 2024-03-26T06:41:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 349713309, 2024-03-26T06:41:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 33, 2024-03-26T06:41:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 48670, 2024-03-26T06:41:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-03-26T06:41:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 48670, 2024-03-26T06:41:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 318.992844050, 2024-03-26T06:41:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.006554198 2024-03-26T06:41:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-03-26T06:41:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-03-26T06:41:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 48670, 2024-03-26T06:41:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 49839 2024-03-26T06:41:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 48670, 2024-03-26T06:41:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 857, 2024-03-26T06:41:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 47813, 2024-03-26T06:41:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-03-26T06:41:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-03-26T06:41:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 0, 2024-03-26T06:41:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 10, 2024-03-26T06:41:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 857, 2024-03-26T06:41:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 27064, 2024-03-26T06:41:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 498, 2024-03-26T06:41:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-03-26T06:41:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-03-26T06:41:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-03-26T06:41:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 3, 2024-03-26T06:41:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 19476, 2024-03-26T06:41:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-03-26T06:41:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-03-26T06:41:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-03-26T06:41:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 0, 2024-03-26T06:41:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 20, 2024-03-26T06:41:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-03-26T06:41:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-03-26T06:41:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-03-26T06:41:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-03-26T06:41:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-03-26T06:41:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-03-26T06:41:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T06:41:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-03-26T06:41:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-03-26T06:41:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-03-26T06:41:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-03-26T06:41:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-03-26T06:41:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-03-26T06:41:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-03-26T06:41:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-03-26T06:41:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-03-26T06:41:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-03-26T06:41:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-03-26T06:41:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-03-26T06:41:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-03-26T06:41:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-03-26T06:41:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-03-26T06:41:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-03-26T06:41:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-03-26T06:41:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-03-26T06:41:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-03-26T06:41:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 116, 2024-03-26T06:41:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-03-26T06:41:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 31, 2024-03-26T06:41:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 12, 2024-03-26T06:41:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 12, 2024-03-26T06:41:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2024-03-26T06:41:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-03-26T06:41:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 1428, 2024-03-26T06:41:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 0, 2024-03-26T06:41:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 483 2024-03-26T06:41:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-03-26T06:41:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 1, 2024-03-26T06:41:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-03-26T06:41:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-03-26T06:41:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-03-26T06:41:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-03-26T06:41:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 4 2024-03-26T06:41:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-03-26T06:41:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-03-26T06:41:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 6, 2024-03-26T06:41:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-03-26T06:41:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 6, 2024-03-26T06:41:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 19374, 2024-03-26T06:41:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 19374, 2024-03-26T06:41:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-03-26T06:41:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-03-26T06:41:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 451032, 2024-03-26T06:41:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 103899174, 2024-03-26T06:41:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 451032, 2024-03-26T06:41:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 451032, 2024-03-26T06:41:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 103899174, 2024-03-26T06:41:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-03-26T06:41:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-03-26T06:41:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-03-26T06:41:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-03-26T06:41:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 48670, 2024-03-26T06:41:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 351671755, 2024-03-26T06:41:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 48670, 2024-03-26T06:41:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 351671755, 2024-03-26T06:41:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-03-26T06:41:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-03-26T06:41:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-03-26T06:41:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-03-26T06:41:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 48670, 2024-03-26T06:41:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 48670, 2024-03-26T06:41:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 48670, 2024-03-26T06:41:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 48670, 2024-03-26T06:41:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-03-26T06:41:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-03-26T06:41:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 19374, 2024-03-26T06:41:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1958446, 2024-03-26T06:41:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 19374, 2024-03-26T06:41:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 857, 2024-03-26T06:41:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1958446, 2024-03-26T06:41:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x5600cc26c6e0": { 2024-03-26T06:41:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 5424, 2024-03-26T06:41:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-03-26T06:41:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 92762, 2024-03-26T06:41:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 345477325, 2024-03-26T06:41:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 92762, 2024-03-26T06:41:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 26207, 2024-03-26T06:41:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 345471901, 2024-03-26T06:41:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:25.684 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-26T06:41:26.343 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:26 smithi066 ceph-mon[36504]: pgmap v584: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 19 op/s 2024-03-26T06:41:26.345 INFO:tasks.check_counter:Daemon mds.e mds.exported=6 2024-03-26T06:41:26.345 INFO:tasks.check_counter:Daemon mds.e mds.imported=14 2024-03-26T06:41:26.345 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-03-26T06:41:26.345 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-26T06:41:26.345 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T06:41:26.345 DEBUG:tasks.check_counter:Getting stats from h 2024-03-26T06:41:26.346 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-26T06:41:26.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:26 smithi149 ceph-mon[36681]: pgmap v584: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 19 op/s 2024-03-26T06:41:26.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:26 smithi037 ceph-mon[31974]: pgmap v584: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 19 op/s 2024-03-26T06:41:26.603 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:41:28.189 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:28 smithi066 ceph-mon[36504]: pgmap v585: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-03-26T06:41:28.190 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:28 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:41:28.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:28 smithi149 ceph-mon[36681]: pgmap v585: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-03-26T06:41:28.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:28 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:41:28.443 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-26T06:41:28.443 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:28.443 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 143408, 2024-03-26T06:41:28.443 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 125800, 2024-03-26T06:41:28.444 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 45779691, 2024-03-26T06:41:28.444 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 155121288, 2024-03-26T06:41:28.444 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 11, 2024-03-26T06:41:28.444 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 4, 2024-03-26T06:41:28.444 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 11.021926828, 2024-03-26T06:41:28.444 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 4.732522186, 2024-03-26T06:41:28.444 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 6.289809976, 2024-03-26T06:41:28.444 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 1.182578770, 2024-03-26T06:41:28.444 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:28.445 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 125798, 2024-03-26T06:41:28.445 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 11.081320256, 2024-03-26T06:41:28.445 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000088088 2024-03-26T06:41:28.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.445 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:28.445 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 236146, 2024-03-26T06:41:28.445 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.048323987, 2024-03-26T06:41:28.445 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000204 2024-03-26T06:41:28.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.446 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 2368, 2024-03-26T06:41:28.446 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 10432 2024-03-26T06:41:28.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.446 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:28.446 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 43233, 2024-03-26T06:41:28.446 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 43197, 2024-03-26T06:41:28.446 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 14460702, 2024-03-26T06:41:28.446 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 103469045, 2024-03-26T06:41:28.446 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 15, 2024-03-26T06:41:28.447 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 5, 2024-03-26T06:41:28.447 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 4.701084673, 2024-03-26T06:41:28.447 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 2.146489707, 2024-03-26T06:41:28.447 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 1.918296228, 2024-03-26T06:41:28.447 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.792333464, 2024-03-26T06:41:28.447 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:28.447 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 43196, 2024-03-26T06:41:28.447 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 5.657750482, 2024-03-26T06:41:28.447 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000130978 2024-03-26T06:41:28.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.448 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:28.448 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 55366, 2024-03-26T06:41:28.448 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.014389252, 2024-03-26T06:41:28.448 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000259 2024-03-26T06:41:28.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.448 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1956160, 2024-03-26T06:41:28.448 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 780592 2024-03-26T06:41:28.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.449 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:28.449 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 233847, 2024-03-26T06:41:28.449 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 329207, 2024-03-26T06:41:28.449 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 69125760, 2024-03-26T06:41:28.449 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 380244677, 2024-03-26T06:41:28.449 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 13, 2024-03-26T06:41:28.449 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 8, 2024-03-26T06:41:28.449 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 16.217529519, 2024-03-26T06:41:28.450 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 7.242585626, 2024-03-26T06:41:28.450 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 9.333886482, 2024-03-26T06:41:28.450 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 1.227364933, 2024-03-26T06:41:28.450 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:28.450 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 329205, 2024-03-26T06:41:28.450 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 41.026591364, 2024-03-26T06:41:28.450 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000124623 2024-03-26T06:41:28.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.451 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:28.451 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 310731, 2024-03-26T06:41:28.451 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.103859456, 2024-03-26T06:41:28.451 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000334 2024-03-26T06:41:28.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.451 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 4080, 2024-03-26T06:41:28.451 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 130000 2024-03-26T06:41:28.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.452 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-03-26T06:41:28.452 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-03-26T06:41:28.452 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-03-26T06:41:28.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.452 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-03-26T06:41:28.452 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-03-26T06:41:28.452 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-03-26T06:41:28.452 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 48638, 2024-03-26T06:41:28.452 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 80.469680513, 2024-03-26T06:41:28.453 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001654461 2024-03-26T06:41:28.453 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:28.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.453 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-03-26T06:41:28.453 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-03-26T06:41:28.453 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-03-26T06:41:28.453 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 32294, 2024-03-26T06:41:28.454 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 7.264355435, 2024-03-26T06:41:28.454 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000224944 2024-03-26T06:41:28.454 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:28.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.454 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-03-26T06:41:28.454 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 122367, 2024-03-26T06:41:28.454 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 114122, 2024-03-26T06:41:28.454 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-03-26T06:41:28.455 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 114122, 2024-03-26T06:41:28.455 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 504.655666370, 2024-03-26T06:41:28.455 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.004422071 2024-03-26T06:41:28.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.455 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-03-26T06:41:28.455 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 8245, 2024-03-26T06:41:28.455 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-03-26T06:41:28.455 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-03-26T06:41:28.455 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 77, 2024-03-26T06:41:28.456 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 7, 2024-03-26T06:41:28.456 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 7, 2024-03-26T06:41:28.456 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 1536, 2024-03-26T06:41:28.456 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 592, 2024-03-26T06:41:28.456 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 929, 2024-03-26T06:41:28.456 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 15, 2024-03-26T06:41:28.456 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 690, 2024-03-26T06:41:28.456 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 139, 2024-03-26T06:41:28.457 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 139, 2024-03-26T06:41:28.457 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 139, 2024-03-26T06:41:28.457 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 12, 2024-03-26T06:41:28.457 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 198710, 2024-03-26T06:41:28.457 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 107653, 2024-03-26T06:41:28.457 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 8244, 2024-03-26T06:41:28.457 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 32, 2024-03-26T06:41:28.457 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-03-26T06:41:28.457 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-03-26T06:41:28.458 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 2627, 2024-03-26T06:41:28.458 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 47502, 2024-03-26T06:41:28.458 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-03-26T06:41:28.458 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 2, 2024-03-26T06:41:28.458 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 107, 2024-03-26T06:41:28.458 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 5, 2024-03-26T06:41:28.458 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 246, 2024-03-26T06:41:28.458 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-03-26T06:41:28.459 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-03-26T06:41:28.459 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 1, 2024-03-26T06:41:28.459 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 709, 2024-03-26T06:41:28.459 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 220974521, 2024-03-26T06:41:28.459 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 6, 2024-03-26T06:41:28.459 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T06:41:28.459 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-03-26T06:41:28.459 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T06:41:28.460 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-03-26T06:41:28.460 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-03-26T06:41:28.460 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T06:41:28.460 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T06:41:28.460 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-03-26T06:41:28.460 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 2, 2024-03-26T06:41:28.460 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 22609, 2024-03-26T06:41:28.460 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 77240, 2024-03-26T06:41:28.460 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T06:41:28.461 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-03-26T06:41:28.461 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T06:41:28.461 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 49270, 2024-03-26T06:41:28.461 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 14875, 2024-03-26T06:41:28.461 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 6779, 2024-03-26T06:41:28.461 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 81414 2024-03-26T06:41:28.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.462 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-03-26T06:41:28.462 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 32, 2024-03-26T06:41:28.462 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 142, 2024-03-26T06:41:28.462 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 85, 2024-03-26T06:41:28.462 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 137, 2024-03-26T06:41:28.462 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 63, 2024-03-26T06:41:28.462 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-03-26T06:41:28.462 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-03-26T06:41:28.462 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-03-26T06:41:28.463 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 18433, 2024-03-26T06:41:28.463 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 18433, 2024-03-26T06:41:28.463 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-03-26T06:41:28.463 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-03-26T06:41:28.463 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-03-26T06:41:28.463 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-03-26T06:41:28.463 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-03-26T06:41:28.463 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-03-26T06:41:28.464 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-03-26T06:41:28.464 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T06:41:28.464 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 5, 2024-03-26T06:41:28.464 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-03-26T06:41:28.464 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 14, 2024-03-26T06:41:28.464 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-03-26T06:41:28.464 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-03-26T06:41:28.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.464 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-03-26T06:41:28.465 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-03-26T06:41:28.465 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 96273, 2024-03-26T06:41:28.465 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 5467, 2024-03-26T06:41:28.465 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 1, 2024-03-26T06:41:28.465 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 96272, 2024-03-26T06:41:28.465 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-03-26T06:41:28.465 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 5466, 2024-03-26T06:41:28.465 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 135, 2024-03-26T06:41:28.465 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 7, 2024-03-26T06:41:28.466 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 1, 2024-03-26T06:41:28.466 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 134, 2024-03-26T06:41:28.466 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 8, 2024-03-26T06:41:28.466 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-03-26T06:41:28.466 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 6, 2024-03-26T06:41:28.466 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194352, 2024-03-26T06:41:28.466 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 362156430, 2024-03-26T06:41:28.466 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 0, 2024-03-26T06:41:28.466 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-03-26T06:41:28.467 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 26473, 2024-03-26T06:41:28.467 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 172.055478352, 2024-03-26T06:41:28.467 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.006499281 2024-03-26T06:41:28.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.467 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 0 2024-03-26T06:41:28.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.467 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-03-26T06:41:28.467 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 810, 2024-03-26T06:41:28.468 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 19365, 2024-03-26T06:41:28.468 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 18555, 2024-03-26T06:41:28.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 139, 2024-03-26T06:41:28.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 257, 2024-03-26T06:41:28.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 118, 2024-03-26T06:41:28.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 1535, 2024-03-26T06:41:28.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 36676, 2024-03-26T06:41:28.468 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 35141, 2024-03-26T06:41:28.468 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 139, 2024-03-26T06:41:28.469 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 20523, 2024-03-26T06:41:28.469 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 20384, 2024-03-26T06:41:28.469 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 72100, 2024-03-26T06:41:28.469 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 207228 2024-03-26T06:41:28.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.469 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-03-26T06:41:28.469 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 172853, 2024-03-26T06:41:28.469 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 4, 2024-03-26T06:41:28.470 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 122367, 2024-03-26T06:41:28.470 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 2686, 2024-03-26T06:41:28.470 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 6912, 2024-03-26T06:41:28.470 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-03-26T06:41:28.470 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 18443, 2024-03-26T06:41:28.470 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 73.357372928, 2024-03-26T06:41:28.470 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.003977518 2024-03-26T06:41:28.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.470 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-03-26T06:41:28.471 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 21, 2024-03-26T06:41:28.471 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.169090270, 2024-03-26T06:41:28.471 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.008051917 2024-03-26T06:41:28.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.471 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-03-26T06:41:28.471 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.471 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.471 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.472 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-03-26T06:41:28.472 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.472 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.472 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.472 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-03-26T06:41:28.472 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 39047, 2024-03-26T06:41:28.472 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 119.984382325, 2024-03-26T06:41:28.473 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.003072819 2024-03-26T06:41:28.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.473 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-03-26T06:41:28.473 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.473 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.473 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.473 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-03-26T06:41:28.473 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.474 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.474 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.474 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-03-26T06:41:28.474 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.474 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.474 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.475 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-03-26T06:41:28.475 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.475 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.475 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.475 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-03-26T06:41:28.475 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.475 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.476 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.476 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:28 smithi037 ceph-mon[31974]: pgmap v585: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-03-26T06:41:28.476 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:28 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T06:41:28.477 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-03-26T06:41:28.477 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.477 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.477 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.477 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-03-26T06:41:28.477 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.477 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.478 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.478 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-03-26T06:41:28.478 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.478 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.478 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.478 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-03-26T06:41:28.479 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.479 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.479 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.479 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-03-26T06:41:28.479 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.479 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.479 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.480 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-03-26T06:41:28.480 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 33188, 2024-03-26T06:41:28.480 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 228.703852763, 2024-03-26T06:41:28.480 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.006891161 2024-03-26T06:41:28.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.480 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-03-26T06:41:28.480 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 4522, 2024-03-26T06:41:28.481 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 17.168652603, 2024-03-26T06:41:28.481 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.003796694 2024-03-26T06:41:28.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.481 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-03-26T06:41:28.481 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.481 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.481 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.482 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-03-26T06:41:28.482 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.482 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.482 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.482 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-03-26T06:41:28.482 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.482 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.483 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.483 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-03-26T06:41:28.483 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.483 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.483 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.483 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-03-26T06:41:28.484 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.484 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.484 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.484 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-03-26T06:41:28.484 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 20, 2024-03-26T06:41:28.484 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.301332111, 2024-03-26T06:41:28.484 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.015066605 2024-03-26T06:41:28.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.485 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-03-26T06:41:28.485 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.485 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.485 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.485 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-03-26T06:41:28.485 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 448, 2024-03-26T06:41:28.486 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 1.030696735, 2024-03-26T06:41:28.486 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.002300662 2024-03-26T06:41:28.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.486 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-03-26T06:41:28.486 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.486 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.486 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.487 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-03-26T06:41:28.487 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.487 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.487 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.487 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-03-26T06:41:28.487 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.487 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.488 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.488 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-03-26T06:41:28.488 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 18433, 2024-03-26T06:41:28.488 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 63.940286635, 2024-03-26T06:41:28.488 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.003468794 2024-03-26T06:41:28.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.488 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-03-26T06:41:28.489 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-03-26T06:41:28.489 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-03-26T06:41:28.489 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.489 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.489 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.489 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:28.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.489 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-03-26T06:41:28.490 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 2, 2024-03-26T06:41:28.490 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 3, 2024-03-26T06:41:28.490 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 1, 2024-03-26T06:41:28.490 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 2, 2024-03-26T06:41:28.490 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-03-26T06:41:28.490 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 24727, 2024-03-26T06:41:28.490 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 12363, 2024-03-26T06:41:28.490 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 1072, 2024-03-26T06:41:28.490 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-03-26T06:41:28.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.491 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-03-26T06:41:28.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-03-26T06:41:28.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-03-26T06:41:28.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:28.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:28.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:28.491 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:28.492 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:28.492 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:28.492 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:28.492 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:28.492 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:28.492 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:28.492 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:28.492 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:28.492 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:28.493 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:28.493 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:28.493 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:28.493 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:28.493 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:28.493 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:28.493 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:28.493 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:28.494 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:28.494 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:28.494 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:28.494 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:28.494 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:28.494 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:28.494 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:28.494 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:28.495 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-03-26T06:41:28.495 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:28.495 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:28.495 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:28.495 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:28.495 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 344077, 2024-03-26T06:41:28.495 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 538, 2024-03-26T06:41:28.495 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 88, 2024-03-26T06:41:28.495 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 1, 2024-03-26T06:41:28.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-03-26T06:41:28.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-03-26T06:41:28.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:28.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:28.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:28.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:28.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 8032, 2024-03-26T06:41:28.496 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 167, 2024-03-26T06:41:28.497 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:28.497 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:28.497 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:28.497 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-03-26T06:41:28.497 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 3377199, 2024-03-26T06:41:28.497 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 30534, 2024-03-26T06:41:28.497 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:28.497 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-03-26T06:41:28.497 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:28.498 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-03-26T06:41:28.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.498 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-03-26T06:41:28.498 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-03-26T06:41:28.498 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-03-26T06:41:28.498 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 48002, 2024-03-26T06:41:28.498 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 362536778, 2024-03-26T06:41:28.498 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 33, 2024-03-26T06:41:28.498 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 48002, 2024-03-26T06:41:28.499 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-03-26T06:41:28.499 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 48002, 2024-03-26T06:41:28.499 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 315.692867541, 2024-03-26T06:41:28.499 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.006576660 2024-03-26T06:41:28.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.499 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-03-26T06:41:28.499 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-03-26T06:41:28.499 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 48002, 2024-03-26T06:41:28.499 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 49169 2024-03-26T06:41:28.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.500 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 48002, 2024-03-26T06:41:28.500 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 852, 2024-03-26T06:41:28.500 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 47150, 2024-03-26T06:41:28.500 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-03-26T06:41:28.500 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-03-26T06:41:28.500 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 1, 2024-03-26T06:41:28.500 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 17, 2024-03-26T06:41:28.500 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 852, 2024-03-26T06:41:28.501 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 27325, 2024-03-26T06:41:28.501 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 468, 2024-03-26T06:41:28.501 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-03-26T06:41:28.501 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-03-26T06:41:28.501 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-03-26T06:41:28.501 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 7, 2024-03-26T06:41:28.501 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 18572, 2024-03-26T06:41:28.501 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-03-26T06:41:28.502 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-03-26T06:41:28.502 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-03-26T06:41:28.502 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 0, 2024-03-26T06:41:28.502 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 34, 2024-03-26T06:41:28.502 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-03-26T06:41:28.502 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-03-26T06:41:28.502 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-03-26T06:41:28.502 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-03-26T06:41:28.503 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-03-26T06:41:28.503 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-03-26T06:41:28.503 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T06:41:28.503 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-03-26T06:41:28.503 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-03-26T06:41:28.503 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-03-26T06:41:28.503 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-03-26T06:41:28.503 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-03-26T06:41:28.503 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-03-26T06:41:28.504 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-03-26T06:41:28.504 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-03-26T06:41:28.504 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-03-26T06:41:28.504 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-03-26T06:41:28.504 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-03-26T06:41:28.504 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-03-26T06:41:28.504 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-03-26T06:41:28.504 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-03-26T06:41:28.505 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-03-26T06:41:28.505 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-03-26T06:41:28.505 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-03-26T06:41:28.505 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-03-26T06:41:28.505 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-03-26T06:41:28.505 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 116, 2024-03-26T06:41:28.505 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-03-26T06:41:28.505 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 31, 2024-03-26T06:41:28.505 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 12, 2024-03-26T06:41:28.506 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 12, 2024-03-26T06:41:28.506 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2024-03-26T06:41:28.506 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-03-26T06:41:28.506 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 1448, 2024-03-26T06:41:28.506 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 0, 2024-03-26T06:41:28.506 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 445 2024-03-26T06:41:28.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.506 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-03-26T06:41:28.506 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 1, 2024-03-26T06:41:28.507 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-03-26T06:41:28.507 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-03-26T06:41:28.507 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-03-26T06:41:28.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.507 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-03-26T06:41:28.507 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 1 2024-03-26T06:41:28.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.507 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-03-26T06:41:28.507 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-03-26T06:41:28.508 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 6, 2024-03-26T06:41:28.508 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-03-26T06:41:28.508 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 6, 2024-03-26T06:41:28.508 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 18433, 2024-03-26T06:41:28.508 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 18433, 2024-03-26T06:41:28.508 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-03-26T06:41:28.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.508 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:28.508 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:28.508 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-03-26T06:41:28.509 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:28.509 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 420484, 2024-03-26T06:41:28.509 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 94422921, 2024-03-26T06:41:28.509 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:28.509 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 420484, 2024-03-26T06:41:28.509 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:28.509 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:28.509 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 420484, 2024-03-26T06:41:28.509 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 94422921, 2024-03-26T06:41:28.510 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:28.510 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.510 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.510 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.510 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:28.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.510 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-03-26T06:41:28.510 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:28.511 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-03-26T06:41:28.511 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:28.511 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-03-26T06:41:28.511 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-03-26T06:41:28.511 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:28.511 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:28.511 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 48002, 2024-03-26T06:41:28.511 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 364400735, 2024-03-26T06:41:28.511 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 48002, 2024-03-26T06:41:28.512 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 364400735, 2024-03-26T06:41:28.512 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:28.512 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.512 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.512 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.512 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:28.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.512 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-03-26T06:41:28.512 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:28.513 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-03-26T06:41:28.513 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:28.513 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-03-26T06:41:28.513 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-03-26T06:41:28.513 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:28.513 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:28.513 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 48002, 2024-03-26T06:41:28.513 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 48002, 2024-03-26T06:41:28.514 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 48002, 2024-03-26T06:41:28.514 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 48002, 2024-03-26T06:41:28.514 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:28.514 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.514 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.514 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.514 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:28.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.514 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-03-26T06:41:28.515 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:28.515 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-03-26T06:41:28.515 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:28.515 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 18433, 2024-03-26T06:41:28.515 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1863957, 2024-03-26T06:41:28.515 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:28.515 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 18433, 2024-03-26T06:41:28.515 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:28.515 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:28.515 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 852, 2024-03-26T06:41:28.516 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1863957, 2024-03-26T06:41:28.516 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:28.516 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.516 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.516 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.516 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:28.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:28.516 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x55d4a181e6e0": { 2024-03-26T06:41:28.516 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 4890, 2024-03-26T06:41:28.517 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-03-26T06:41:28.517 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:28.517 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 96273, 2024-03-26T06:41:28.517 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 357962126, 2024-03-26T06:41:28.517 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:28.517 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 96273, 2024-03-26T06:41:28.517 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:28.517 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:28.517 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 26473, 2024-03-26T06:41:28.518 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 357957236, 2024-03-26T06:41:28.518 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:28.518 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:28.518 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:28.518 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:28.518 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:28.518 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:28.518 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-26T06:41:29.208 INFO:tasks.check_counter:Daemon mds.h mds.exported=2 2024-03-26T06:41:29.208 INFO:tasks.check_counter:Daemon mds.h mds.imported=5 2024-03-26T06:41:29.208 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-26T06:41:29.208 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-26T06:41:29.208 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T06:41:29.208 DEBUG:tasks.check_counter:Getting stats from k 2024-03-26T06:41:29.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-26T06:41:29.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:29 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:41:29.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:29 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:41:29.349 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:29 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:41:29.411 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:29 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:41:29.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:29 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:41:29.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:29 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:41:29.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:29 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T06:41:29.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:29 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T06:41:29.451 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:29 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' 2024-03-26T06:41:29.461 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T06:41:30.599 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:30 smithi066 ceph-mon[36504]: pgmap v586: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-03-26T06:41:30.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:30 smithi149 ceph-mon[36681]: pgmap v586: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-03-26T06:41:30.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:30 smithi037 ceph-mon[31974]: pgmap v586: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 11 op/s 2024-03-26T06:41:31.143 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-03-26T06:41:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 1100, 2024-03-26T06:41:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 1040, 2024-03-26T06:41:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 127943766, 2024-03-26T06:41:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 421528, 2024-03-26T06:41:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 8, 2024-03-26T06:41:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-03-26T06:41:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.197416008, 2024-03-26T06:41:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.034116173, 2024-03-26T06:41:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.139867825, 2024-03-26T06:41:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.027067935, 2024-03-26T06:41:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1040, 2024-03-26T06:41:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.049826156, 2024-03-26T06:41:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000047909 2024-03-26T06:41:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1944624, 2024-03-26T06:41:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 210688 2024-03-26T06:41:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 1165, 2024-03-26T06:41:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 1421, 2024-03-26T06:41:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 84190226, 2024-03-26T06:41:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 980415, 2024-03-26T06:41:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 9, 2024-03-26T06:41:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 7, 2024-03-26T06:41:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.168006310, 2024-03-26T06:41:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.036464374, 2024-03-26T06:41:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.085894895, 2024-03-26T06:41:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.033342068, 2024-03-26T06:41:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1418, 2024-03-26T06:41:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.056149660, 2024-03-26T06:41:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000039597 2024-03-26T06:41:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T06:41:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 610576 2024-03-26T06:41:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2796, 2024-03-26T06:41:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2798, 2024-03-26T06:41:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 136672574, 2024-03-26T06:41:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 889186, 2024-03-26T06:41:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 6, 2024-03-26T06:41:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 5, 2024-03-26T06:41:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.326309785, 2024-03-26T06:41:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.076229561, 2024-03-26T06:41:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.146950092, 2024-03-26T06:41:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.080584389, 2024-03-26T06:41:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2796, 2024-03-26T06:41:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.112630835, 2024-03-26T06:41:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000040282 2024-03-26T06:41:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T06:41:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T06:41:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-03-26T06:41:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-03-26T06:41:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-03-26T06:41:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-03-26T06:41:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-03-26T06:41:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-03-26T06:41:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 7596, 2024-03-26T06:41:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.193065832, 2024-03-26T06:41:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000025416 2024-03-26T06:41:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-03-26T06:41:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-03-26T06:41:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-03-26T06:41:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-03-26T06:41:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-03-26T06:41:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 0, 2024-03-26T06:41:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-03-26T06:41:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-03-26T06:41:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 0, 2024-03-26T06:41:31.168 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-03-26T06:41:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-03-26T06:41:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 0, 2024-03-26T06:41:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-03-26T06:41:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-03-26T06:41:31.169 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 20315, 2024-03-26T06:41:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 44, 2024-03-26T06:41:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 20271, 2024-03-26T06:41:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2024-03-26T06:41:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 20253, 2024-03-26T06:41:31.170 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 0, 2024-03-26T06:41:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 0, 2024-03-26T06:41:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 0, 2024-03-26T06:41:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 10, 2024-03-26T06:41:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 0, 2024-03-26T06:41:31.171 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 0, 2024-03-26T06:41:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 0, 2024-03-26T06:41:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 0, 2024-03-26T06:41:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-03-26T06:41:31.172 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-03-26T06:41:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-03-26T06:41:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 0, 2024-03-26T06:41:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-03-26T06:41:31.173 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2024-03-26T06:41:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2024-03-26T06:41:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2024-03-26T06:41:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2024-03-26T06:41:31.174 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-03-26T06:41:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-03-26T06:41:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-03-26T06:41:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 0, 2024-03-26T06:41:31.175 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 0, 2024-03-26T06:41:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-03-26T06:41:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T06:41:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-03-26T06:41:31.176 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T06:41:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-03-26T06:41:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-03-26T06:41:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T06:41:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T06:41:31.177 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-03-26T06:41:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-03-26T06:41:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T06:41:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 0, 2024-03-26T06:41:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T06:41:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T06:41:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T06:41:31.178 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 0, 2024-03-26T06:41:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 0, 2024-03-26T06:41:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2024-03-26T06:41:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 0 2024-03-26T06:41:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-03-26T06:41:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2024-03-26T06:41:31.179 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 0, 2024-03-26T06:41:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 0, 2024-03-26T06:41:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 0, 2024-03-26T06:41:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 0, 2024-03-26T06:41:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-03-26T06:41:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-03-26T06:41:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-03-26T06:41:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2024-03-26T06:41:31.180 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2024-03-26T06:41:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-03-26T06:41:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-03-26T06:41:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-03-26T06:41:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-03-26T06:41:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-03-26T06:41:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-03-26T06:41:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-03-26T06:41:31.181 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T06:41:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2024-03-26T06:41:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-03-26T06:41:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-03-26T06:41:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-03-26T06:41:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-03-26T06:41:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.182 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-03-26T06:41:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-03-26T06:41:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 0, 2024-03-26T06:41:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-03-26T06:41:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-03-26T06:41:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 92774, 2024-03-26T06:41:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-03-26T06:41:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-03-26T06:41:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 0, 2024-03-26T06:41:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-03-26T06:41:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-03-26T06:41:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 127, 2024-03-26T06:41:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 9, 2024-03-26T06:41:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-03-26T06:41:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-03-26T06:41:31.184 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2024-03-26T06:41:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 349767726, 2024-03-26T06:41:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 349761931, 2024-03-26T06:41:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-03-26T06:41:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 92774 2024-03-26T06:41:31.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-03-26T06:41:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 778, 2024-03-26T06:41:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 20278, 2024-03-26T06:41:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 19500, 2024-03-26T06:41:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 63, 2024-03-26T06:41:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 82, 2024-03-26T06:41:31.186 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 19, 2024-03-26T06:41:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 20315, 2024-03-26T06:41:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 20507, 2024-03-26T06:41:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 192, 2024-03-26T06:41:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 0, 2024-03-26T06:41:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 0, 2024-03-26T06:41:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 0, 2024-03-26T06:41:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 70400, 2024-03-26T06:41:31.187 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 215420 2024-03-26T06:41:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-03-26T06:41:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 0, 2024-03-26T06:41:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2024-03-26T06:41:31.188 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 0, 2024-03-26T06:41:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 0, 2024-03-26T06:41:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 0, 2024-03-26T06:41:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-03-26T06:41:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.189 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-03-26T06:41:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-03-26T06:41:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.190 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-03-26T06:41:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-03-26T06:41:31.191 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-03-26T06:41:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.192 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-03-26T06:41:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-03-26T06:41:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-03-26T06:41:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-03-26T06:41:31.194 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-03-26T06:41:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.195 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-03-26T06:41:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.196 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-03-26T06:41:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.197 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-03-26T06:41:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-03-26T06:41:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.199 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-03-26T06:41:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-03-26T06:41:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.200 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-03-26T06:41:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-03-26T06:41:31.201 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-03-26T06:41:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-03-26T06:41:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-03-26T06:41:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.203 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-03-26T06:41:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.204 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-03-26T06:41:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-03-26T06:41:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.205 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-03-26T06:41:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-03-26T06:41:31.206 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-03-26T06:41:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-03-26T06:41:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-03-26T06:41:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-03-26T06:41:31.208 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-03-26T06:41:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.209 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-03-26T06:41:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 2, 2024-03-26T06:41:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 5, 2024-03-26T06:41:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 3, 2024-03-26T06:41:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 4, 2024-03-26T06:41:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-03-26T06:41:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 0, 2024-03-26T06:41:31.210 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 0, 2024-03-26T06:41:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 3838, 2024-03-26T06:41:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-03-26T06:41:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-03-26T06:41:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-03-26T06:41:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-03-26T06:41:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:31.211 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:31.212 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:31.213 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:31.214 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-03-26T06:41:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:31.215 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 274353, 2024-03-26T06:41:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 965, 2024-03-26T06:41:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-03-26T06:41:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-03-26T06:41:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-03-26T06:41:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-03-26T06:41:31.216 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 8032, 2024-03-26T06:41:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 167, 2024-03-26T06:41:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:31.217 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-03-26T06:41:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 16350202, 2024-03-26T06:41:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 101884, 2024-03-26T06:41:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-03-26T06:41:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-03-26T06:41:31.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-03-26T06:41:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-03-26T06:41:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-03-26T06:41:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 4622, 2024-03-26T06:41:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 21, 2024-03-26T06:41:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 3, 2024-03-26T06:41:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 4622, 2024-03-26T06:41:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-03-26T06:41:31.219 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 4622, 2024-03-26T06:41:31.220 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 6.934952946, 2024-03-26T06:41:31.220 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001500422 2024-03-26T06:41:31.220 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.220 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-03-26T06:41:31.220 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-03-26T06:41:31.221 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 4622, 2024-03-26T06:41:31.221 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 4623 2024-03-26T06:41:31.221 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.221 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 4622, 2024-03-26T06:41:31.221 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 4622, 2024-03-26T06:41:31.221 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 0, 2024-03-26T06:41:31.221 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-03-26T06:41:31.221 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-03-26T06:41:31.221 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 2663, 2024-03-26T06:41:31.222 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 0, 2024-03-26T06:41:31.222 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 1957, 2024-03-26T06:41:31.222 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 0, 2024-03-26T06:41:31.222 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 0, 2024-03-26T06:41:31.222 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-03-26T06:41:31.222 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-03-26T06:41:31.222 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-03-26T06:41:31.222 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-03-26T06:41:31.223 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 0, 2024-03-26T06:41:31.223 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-03-26T06:41:31.223 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-03-26T06:41:31.223 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-03-26T06:41:31.223 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 1, 2024-03-26T06:41:31.223 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 0, 2024-03-26T06:41:31.223 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-03-26T06:41:31.223 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-03-26T06:41:31.223 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-03-26T06:41:31.224 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-03-26T06:41:31.224 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-03-26T06:41:31.224 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-03-26T06:41:31.224 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T06:41:31.224 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-03-26T06:41:31.224 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-03-26T06:41:31.224 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-03-26T06:41:31.224 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-03-26T06:41:31.224 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-03-26T06:41:31.224 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-03-26T06:41:31.225 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-03-26T06:41:31.225 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-03-26T06:41:31.225 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-03-26T06:41:31.225 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-03-26T06:41:31.225 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-03-26T06:41:31.225 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-03-26T06:41:31.225 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-03-26T06:41:31.225 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-03-26T06:41:31.226 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-03-26T06:41:31.226 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-03-26T06:41:31.226 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-03-26T06:41:31.226 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-03-26T06:41:31.226 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-03-26T06:41:31.226 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 116, 2024-03-26T06:41:31.226 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-03-26T06:41:31.227 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 31, 2024-03-26T06:41:31.227 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 11, 2024-03-26T06:41:31.227 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 12, 2024-03-26T06:41:31.227 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 1, 2024-03-26T06:41:31.227 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-03-26T06:41:31.227 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 0, 2024-03-26T06:41:31.227 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 2, 2024-03-26T06:41:31.228 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 0 2024-03-26T06:41:31.228 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.228 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-03-26T06:41:31.228 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 0, 2024-03-26T06:41:31.228 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-03-26T06:41:31.228 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-03-26T06:41:31.228 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-03-26T06:41:31.228 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.229 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-03-26T06:41:31.229 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-03-26T06:41:31.229 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.229 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-03-26T06:41:31.229 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-03-26T06:41:31.229 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T06:41:31.229 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-03-26T06:41:31.229 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2024-03-26T06:41:31.230 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 0, 2024-03-26T06:41:31.230 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2024-03-26T06:41:31.230 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-03-26T06:41:31.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.230 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:31.230 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:31.231 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-03-26T06:41:31.231 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:31.231 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 5057, 2024-03-26T06:41:31.231 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 348326681, 2024-03-26T06:41:31.231 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:31.232 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 5057, 2024-03-26T06:41:31.232 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:31.232 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:31.232 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 5057, 2024-03-26T06:41:31.232 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 348326681, 2024-03-26T06:41:31.232 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:31.232 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.232 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.232 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.233 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:31.233 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.233 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-03-26T06:41:31.233 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:31.233 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-03-26T06:41:31.233 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:31.233 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-03-26T06:41:31.233 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-03-26T06:41:31.233 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:31.234 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:31.234 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 4622, 2024-03-26T06:41:31.234 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 345585252, 2024-03-26T06:41:31.234 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 4622, 2024-03-26T06:41:31.234 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 345585252, 2024-03-26T06:41:31.234 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:31.234 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.234 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.235 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.235 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:31.235 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.235 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-03-26T06:41:31.235 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:31.235 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-03-26T06:41:31.235 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:31.235 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-03-26T06:41:31.235 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-03-26T06:41:31.236 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:31.236 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:31.236 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 4622, 2024-03-26T06:41:31.236 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 4622, 2024-03-26T06:41:31.236 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 4622, 2024-03-26T06:41:31.236 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 4622, 2024-03-26T06:41:31.236 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:31.236 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.236 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.237 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.237 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:31.237 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.237 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-03-26T06:41:31.237 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:31.237 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-03-26T06:41:31.237 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:31.237 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-03-26T06:41:31.237 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-03-26T06:41:31.238 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:31.238 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:31.238 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:31.238 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:31.238 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-03-26T06:41:31.238 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-03-26T06:41:31.238 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:31.238 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.239 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.239 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.239 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:31.239 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-03-26T06:41:31.239 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x5584df430960": { 2024-03-26T06:41:31.239 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-03-26T06:41:31.239 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-03-26T06:41:31.239 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-03-26T06:41:31.239 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-03-26T06:41:31.240 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-03-26T06:41:31.240 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:31.240 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:31.240 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-03-26T06:41:31.240 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-03-26T06:41:31.240 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-03-26T06:41:31.240 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-03-26T06:41:31.240 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-03-26T06:41:31.240 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-03-26T06:41:31.240 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-03-26T06:41:31.241 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-03-26T06:41:31.241 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:31.241 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-03-26T06:41:31.241 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-03-26T06:41:31.808 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-26T06:41:31.808 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-26T06:41:31.808 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-03-26T06:41:31.808 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-26T06:41:31.808 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T06:41:31.808 DEBUG:tasks.check_counter:Getting stats from c 2024-03-26T06:41:31.809 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-26T06:41:32.241 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:41:32.648 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:32 smithi149 ceph-mon[36681]: pgmap v587: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 140 KiB/s wr, 19 op/s 2024-03-26T06:41:32.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:32 smithi066 ceph-mon[36504]: pgmap v587: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 140 KiB/s wr, 19 op/s 2024-03-26T06:41:32.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:32 smithi037 ceph-mon[31974]: pgmap v587: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 140 KiB/s wr, 19 op/s 2024-03-26T06:41:34.731 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-26T06:41:34.731 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:34.731 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 402, 2024-03-26T06:41:34.731 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 402, 2024-03-26T06:41:34.732 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 41931507, 2024-03-26T06:41:34.732 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 127022, 2024-03-26T06:41:34.732 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 7, 2024-03-26T06:41:34.732 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-03-26T06:41:34.732 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.060060437, 2024-03-26T06:41:34.732 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.009450285, 2024-03-26T06:41:34.732 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.034574229, 2024-03-26T06:41:34.732 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.012119177, 2024-03-26T06:41:34.732 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:34.733 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 401, 2024-03-26T06:41:34.733 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.017749156, 2024-03-26T06:41:34.733 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000044262 2024-03-26T06:41:34.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.733 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:34.733 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.733 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.733 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.734 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-26T06:41:34.734 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T06:41:34.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.734 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:34.734 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 276, 2024-03-26T06:41:34.734 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 265, 2024-03-26T06:41:34.734 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 42233499, 2024-03-26T06:41:34.735 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 96799, 2024-03-26T06:41:34.735 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2024-03-26T06:41:34.735 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-26T06:41:34.735 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.051023182, 2024-03-26T06:41:34.735 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.007569825, 2024-03-26T06:41:34.735 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.037350825, 2024-03-26T06:41:34.735 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.006318138, 2024-03-26T06:41:34.735 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:34.735 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 264, 2024-03-26T06:41:34.736 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.010800748, 2024-03-26T06:41:34.736 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000040911 2024-03-26T06:41:34.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.736 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:34.736 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.736 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.736 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.736 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 239440, 2024-03-26T06:41:34.737 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 34096 2024-03-26T06:41:34.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.737 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:34.737 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 208, 2024-03-26T06:41:34.737 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 248, 2024-03-26T06:41:34.737 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 37758221, 2024-03-26T06:41:34.737 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 220348, 2024-03-26T06:41:34.737 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2024-03-26T06:41:34.737 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2024-03-26T06:41:34.738 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.038355367, 2024-03-26T06:41:34.738 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.006518747, 2024-03-26T06:41:34.738 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.024975598, 2024-03-26T06:41:34.738 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.005223619, 2024-03-26T06:41:34.738 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:34.738 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 246, 2024-03-26T06:41:34.738 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.017194195, 2024-03-26T06:41:34.738 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000069895 2024-03-26T06:41:34.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.739 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:34.739 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.739 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.739 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.739 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-26T06:41:34.739 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 154512 2024-03-26T06:41:34.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.739 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-26T06:41:34.740 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-26T06:41:34.740 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-26T06:41:34.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.740 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-26T06:41:34.740 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-26T06:41:34.740 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-26T06:41:34.740 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1311, 2024-03-26T06:41:34.740 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.091230917, 2024-03-26T06:41:34.740 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000069588 2024-03-26T06:41:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-26T06:41:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-26T06:41:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-26T06:41:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-26T06:41:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-03-26T06:41:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-03-26T06:41:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-26T06:41:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.743 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-26T06:41:34.743 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-03-26T06:41:34.743 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-26T06:41:34.743 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-26T06:41:34.743 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2024-03-26T06:41:34.743 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-26T06:41:34.744 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-26T06:41:34.744 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 7284, 2024-03-26T06:41:34.744 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 4782, 2024-03-26T06:41:34.744 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 2502, 2024-03-26T06:41:34.744 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-03-26T06:41:34.744 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 453, 2024-03-26T06:41:34.744 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-03-26T06:41:34.744 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-03-26T06:41:34.744 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-03-26T06:41:34.745 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 14, 2024-03-26T06:41:34.745 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2024-03-26T06:41:34.745 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2024-03-26T06:41:34.746 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-26T06:41:34.746 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-03-26T06:41:34.746 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-26T06:41:34.746 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-26T06:41:34.746 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-26T06:41:34.746 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-26T06:41:34.746 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-26T06:41:34.747 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-03-26T06:41:34.747 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-03-26T06:41:34.747 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-03-26T06:41:34.747 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-03-26T06:41:34.747 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-26T06:41:34.747 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-03-26T06:41:34.747 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-26T06:41:34.747 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 705, 2024-03-26T06:41:34.747 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 221259469, 2024-03-26T06:41:34.748 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 6, 2024-03-26T06:41:34.748 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T06:41:34.748 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-26T06:41:34.748 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T06:41:34.748 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-26T06:41:34.748 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-03-26T06:41:34.748 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T06:41:34.748 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T06:41:34.748 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-03-26T06:41:34.748 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-26T06:41:34.749 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T06:41:34.749 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-03-26T06:41:34.749 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T06:41:34.749 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T06:41:34.749 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T06:41:34.749 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-03-26T06:41:34.749 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-03-26T06:41:34.749 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-03-26T06:41:34.749 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-03-26T06:41:34.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.750 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-26T06:41:34.750 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-26T06:41:34.750 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-26T06:41:34.750 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-26T06:41:34.750 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-03-26T06:41:34.750 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-03-26T06:41:34.751 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-03-26T06:41:34.751 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-26T06:41:34.751 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-26T06:41:34.751 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-03-26T06:41:34.751 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-03-26T06:41:34.751 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-26T06:41:34.751 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-26T06:41:34.751 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-26T06:41:34.751 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-26T06:41:34.752 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-26T06:41:34.752 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-26T06:41:34.752 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-26T06:41:34.752 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T06:41:34.752 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-03-26T06:41:34.752 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-26T06:41:34.752 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-26T06:41:34.752 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-26T06:41:34.752 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-26T06:41:34.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.753 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-26T06:41:34.753 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-26T06:41:34.753 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2024-03-26T06:41:34.753 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-26T06:41:34.753 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-26T06:41:34.753 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 34421, 2024-03-26T06:41:34.753 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-26T06:41:34.753 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-26T06:41:34.753 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2024-03-26T06:41:34.754 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-26T06:41:34.754 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-26T06:41:34.754 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 5, 2024-03-26T06:41:34.754 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 12, 2024-03-26T06:41:34.754 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-26T06:41:34.754 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-26T06:41:34.754 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 459044566, 2024-03-26T06:41:34.754 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 461328608, 2024-03-26T06:41:34.755 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 461320153, 2024-03-26T06:41:34.755 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-26T06:41:34.755 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.755 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.755 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.755 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 34421 2024-03-26T06:41:34.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.755 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-26T06:41:34.756 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 341, 2024-03-26T06:41:34.756 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 7188, 2024-03-26T06:41:34.756 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 6847, 2024-03-26T06:41:34.756 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 72, 2024-03-26T06:41:34.756 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 107, 2024-03-26T06:41:34.756 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 35, 2024-03-26T06:41:34.756 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 7291, 2024-03-26T06:41:34.756 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 7345, 2024-03-26T06:41:34.756 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 54, 2024-03-26T06:41:34.757 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-03-26T06:41:34.757 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-03-26T06:41:34.757 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-03-26T06:41:34.757 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 58400, 2024-03-26T06:41:34.757 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215396 2024-03-26T06:41:34.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.757 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-26T06:41:34.757 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-03-26T06:41:34.757 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-03-26T06:41:34.758 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-03-26T06:41:34.758 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-03-26T06:41:34.758 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-03-26T06:41:34.758 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-26T06:41:34.758 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.758 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.758 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.759 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-26T06:41:34.759 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.759 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.759 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.759 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-26T06:41:34.759 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.759 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.760 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.760 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-26T06:41:34.760 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.760 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.760 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.760 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-26T06:41:34.760 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.761 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.761 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.761 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-26T06:41:34.761 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.761 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.761 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.761 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-26T06:41:34.761 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.762 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.762 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.762 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-26T06:41:34.762 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.762 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.762 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.763 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-26T06:41:34.763 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.763 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.763 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.763 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-26T06:41:34.763 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.763 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.764 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.764 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-26T06:41:34.764 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.764 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.764 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.764 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-26T06:41:34.765 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.765 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.765 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.766 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-26T06:41:34.766 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.766 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.766 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.766 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-26T06:41:34.766 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.767 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.767 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.767 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-26T06:41:34.767 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.767 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.767 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.767 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-26T06:41:34.768 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.768 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.768 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.768 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-26T06:41:34.768 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.768 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.768 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.769 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-26T06:41:34.769 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.769 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.769 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.769 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-26T06:41:34.769 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.769 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.769 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.770 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-26T06:41:34.770 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.770 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.770 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.770 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-26T06:41:34.770 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.770 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.771 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.771 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-26T06:41:34.771 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.771 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.771 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.771 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-26T06:41:34.771 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.772 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.772 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.772 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-26T06:41:34.772 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.772 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.772 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.773 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-26T06:41:34.773 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.773 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.773 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.773 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-26T06:41:34.773 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.773 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.774 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.774 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-26T06:41:34.774 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.774 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.774 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.774 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-26T06:41:34.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.775 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.775 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-26T06:41:34.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.775 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.776 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-26T06:41:34.776 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-26T06:41:34.776 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-26T06:41:34.776 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.776 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.776 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.776 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:34.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.776 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-26T06:41:34.777 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 2, 2024-03-26T06:41:34.777 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 2, 2024-03-26T06:41:34.777 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 0, 2024-03-26T06:41:34.777 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 0, 2024-03-26T06:41:34.777 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-26T06:41:34.777 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-03-26T06:41:34.777 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-03-26T06:41:34.777 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 2564, 2024-03-26T06:41:34.777 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-26T06:41:34.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.778 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-26T06:41:34.778 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-26T06:41:34.778 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-26T06:41:34.778 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:34.778 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:34.778 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:34.778 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:34.778 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:34.779 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:34.779 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:34.779 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:34.779 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:34.779 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:34.779 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:34.779 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:34.779 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:34.780 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:34.780 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:34.780 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:34.780 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:34.780 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:34.780 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:34.780 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:34.780 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:34.780 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:34.781 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:34.781 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:34.781 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:34.781 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:34.781 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:34.781 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:34.781 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:34.781 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-26T06:41:34.781 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:34.782 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:34.782 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:34.782 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:34.782 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 240187, 2024-03-26T06:41:34.782 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 568, 2024-03-26T06:41:34.782 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-26T06:41:34.782 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-26T06:41:34.782 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-26T06:41:34.782 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-26T06:41:34.783 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:34.783 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:34.783 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:34.783 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:34.783 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8032, 2024-03-26T06:41:34.783 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 167, 2024-03-26T06:41:34.783 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:34.783 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:34.784 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:34.784 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-26T06:41:34.784 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 6171588, 2024-03-26T06:41:34.784 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 40469, 2024-03-26T06:41:34.784 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:34.784 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-26T06:41:34.784 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:34.784 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-26T06:41:34.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.785 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-26T06:41:34.785 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-26T06:41:34.785 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-26T06:41:34.785 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 802, 2024-03-26T06:41:34.785 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 26, 2024-03-26T06:41:34.785 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 4, 2024-03-26T06:41:34.785 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 802, 2024-03-26T06:41:34.786 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-26T06:41:34.786 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 802, 2024-03-26T06:41:34.786 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 5.385891059, 2024-03-26T06:41:34.786 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.006715574 2024-03-26T06:41:34.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.786 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-26T06:41:34.786 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-26T06:41:34.786 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 802, 2024-03-26T06:41:34.786 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 803 2024-03-26T06:41:34.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.787 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 802, 2024-03-26T06:41:34.787 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 802, 2024-03-26T06:41:34.787 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2024-03-26T06:41:34.787 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-26T06:41:34.787 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-26T06:41:34.787 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 424, 2024-03-26T06:41:34.787 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2024-03-26T06:41:34.787 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 375, 2024-03-26T06:41:34.788 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2024-03-26T06:41:34.788 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2024-03-26T06:41:34.788 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-26T06:41:34.788 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-26T06:41:34.788 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-26T06:41:34.788 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-26T06:41:34.788 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2024-03-26T06:41:34.788 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-26T06:41:34.788 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-26T06:41:34.789 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-26T06:41:34.789 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 2, 2024-03-26T06:41:34.789 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2024-03-26T06:41:34.789 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-26T06:41:34.789 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-26T06:41:34.789 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-26T06:41:34.789 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-26T06:41:34.789 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-26T06:41:34.790 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-26T06:41:34.790 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T06:41:34.790 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-26T06:41:34.790 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-26T06:41:34.790 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-26T06:41:34.790 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-26T06:41:34.790 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-26T06:41:34.790 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-26T06:41:34.791 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-26T06:41:34.791 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-26T06:41:34.791 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-26T06:41:34.791 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-26T06:41:34.791 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-26T06:41:34.791 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-26T06:41:34.791 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-26T06:41:34.791 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-26T06:41:34.791 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-26T06:41:34.792 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-26T06:41:34.792 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-26T06:41:34.792 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-26T06:41:34.792 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-26T06:41:34.792 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 116, 2024-03-26T06:41:34.792 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-26T06:41:34.792 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 6, 2024-03-26T06:41:34.792 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 9, 2024-03-26T06:41:34.793 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 9, 2024-03-26T06:41:34.793 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-26T06:41:34.793 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-26T06:41:34.793 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2024-03-26T06:41:34.793 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2024-03-26T06:41:34.793 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2024-03-26T06:41:34.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.793 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-26T06:41:34.794 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2024-03-26T06:41:34.794 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-26T06:41:34.794 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-26T06:41:34.794 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-26T06:41:34.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.794 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-26T06:41:34.794 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-26T06:41:34.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.794 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-26T06:41:34.795 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-26T06:41:34.795 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T06:41:34.795 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-26T06:41:34.795 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-03-26T06:41:34.795 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-03-26T06:41:34.795 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-03-26T06:41:34.795 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-26T06:41:34.795 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.796 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:34.796 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:34.796 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-26T06:41:34.796 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:34.796 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 882, 2024-03-26T06:41:34.796 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 121836901, 2024-03-26T06:41:34.796 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:34.796 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 882, 2024-03-26T06:41:34.796 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:34.797 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:34.797 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 882, 2024-03-26T06:41:34.797 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 121836901, 2024-03-26T06:41:34.797 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:34.797 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.797 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.797 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.797 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:34.797 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.797 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-26T06:41:34.798 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:34.798 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-26T06:41:34.798 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:34.798 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:34.798 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:34.798 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:34.798 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:34.798 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 802, 2024-03-26T06:41:34.799 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 121455442, 2024-03-26T06:41:34.799 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 802, 2024-03-26T06:41:34.799 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 121455442, 2024-03-26T06:41:34.799 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:34.799 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.799 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.799 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.799 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:34.799 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.800 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-26T06:41:34.800 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:34.800 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-26T06:41:34.800 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:34.800 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:34.800 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:34.800 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:34.800 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:34.800 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 802, 2024-03-26T06:41:34.801 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 802, 2024-03-26T06:41:34.801 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 802, 2024-03-26T06:41:34.801 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 802, 2024-03-26T06:41:34.801 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:34.801 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.801 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.801 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.801 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:34.801 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.801 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-26T06:41:34.802 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:34.802 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-26T06:41:34.802 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:34.802 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:34.802 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:34.802 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:34.802 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:34.802 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:34.803 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:34.803 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-26T06:41:34.803 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-26T06:41:34.803 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:34.803 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.803 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.803 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.803 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:34.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:34.804 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55c3ec1ac960": { 2024-03-26T06:41:34.804 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:34.804 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-26T06:41:34.804 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:34.804 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:34.804 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:34.804 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:34.804 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:34.805 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:34.805 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:34.805 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-26T06:41:34.805 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-26T06:41:34.805 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:34.805 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:34.805 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:34.806 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:34.806 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:34.806 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:34.806 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-26T06:41:34.806 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:34 smithi149 ceph-mon[36681]: pgmap v588: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 124 KiB/s wr, 12 op/s 2024-03-26T06:41:34.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:34 smithi066 ceph-mon[36504]: pgmap v588: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 124 KiB/s wr, 12 op/s 2024-03-26T06:41:34.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:34 smithi037 ceph-mon[31974]: pgmap v588: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 124 KiB/s wr, 12 op/s 2024-03-26T06:41:36.241 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-26T06:41:36.241 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-26T06:41:36.241 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-03-26T06:41:36.241 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-26T06:41:36.242 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T06:41:36.242 DEBUG:tasks.check_counter:Getting stats from f 2024-03-26T06:41:36.242 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-26T06:41:36.543 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:41:36.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:36 smithi066 ceph-mon[36504]: pgmap v589: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 136 KiB/s wr, 14 op/s 2024-03-26T06:41:36.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:36 smithi149 ceph-mon[36681]: pgmap v589: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 136 KiB/s wr, 14 op/s 2024-03-26T06:41:36.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:36 smithi037 ceph-mon[31974]: pgmap v589: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 136 KiB/s wr, 14 op/s 2024-03-26T06:41:38.758 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:38 smithi149 ceph-mon[36681]: pgmap v590: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 215 KiB/s wr, 21 op/s 2024-03-26T06:41:38.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:38 smithi066 ceph-mon[36504]: pgmap v590: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 215 KiB/s wr, 21 op/s 2024-03-26T06:41:38.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:38 smithi037 ceph-mon[31974]: pgmap v590: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 215 KiB/s wr, 21 op/s 2024-03-26T06:41:39.069 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-26T06:41:39.069 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:39.069 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 2262, 2024-03-26T06:41:39.069 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2204, 2024-03-26T06:41:39.069 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 90587638, 2024-03-26T06:41:39.070 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 791556, 2024-03-26T06:41:39.070 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 7, 2024-03-26T06:41:39.070 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-03-26T06:41:39.070 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.286158346, 2024-03-26T06:41:39.070 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.069645547, 2024-03-26T06:41:39.070 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.146346683, 2024-03-26T06:41:39.070 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.064721826, 2024-03-26T06:41:39.070 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:39.071 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2203, 2024-03-26T06:41:39.071 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.089197002, 2024-03-26T06:41:39.071 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000040488 2024-03-26T06:41:39.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.071 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:39.071 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.071 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.071 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.072 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1941360, 2024-03-26T06:41:39.072 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 209616 2024-03-26T06:41:39.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.072 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:39.072 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 802, 2024-03-26T06:41:39.072 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1057, 2024-03-26T06:41:39.072 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 81380459, 2024-03-26T06:41:39.072 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 866462, 2024-03-26T06:41:39.072 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 8, 2024-03-26T06:41:39.073 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-03-26T06:41:39.073 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.130129694, 2024-03-26T06:41:39.073 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.026386935, 2024-03-26T06:41:39.073 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.075187707, 2024-03-26T06:41:39.073 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.020305812, 2024-03-26T06:41:39.073 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:39.073 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1054, 2024-03-26T06:41:39.073 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.040807490, 2024-03-26T06:41:39.073 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000038716 2024-03-26T06:41:39.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.074 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:39.074 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.074 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.074 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.074 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T06:41:39.074 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 612000 2024-03-26T06:41:39.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.075 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:39.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1955, 2024-03-26T06:41:39.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1955, 2024-03-26T06:41:39.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 189252730, 2024-03-26T06:41:39.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 620873, 2024-03-26T06:41:39.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 7, 2024-03-26T06:41:39.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2024-03-26T06:41:39.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.229708081, 2024-03-26T06:41:39.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.042428980, 2024-03-26T06:41:39.076 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.129858184, 2024-03-26T06:41:39.076 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.046399342, 2024-03-26T06:41:39.076 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:39.076 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1954, 2024-03-26T06:41:39.076 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.087487972, 2024-03-26T06:41:39.076 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000044773 2024-03-26T06:41:39.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.076 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:39.076 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.077 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.077 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.077 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-26T06:41:39.077 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T06:41:39.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.077 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-26T06:41:39.077 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-26T06:41:39.077 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-26T06:41:39.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.078 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-26T06:41:39.078 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-26T06:41:39.078 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-26T06:41:39.078 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7591, 2024-03-26T06:41:39.078 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.176258030, 2024-03-26T06:41:39.078 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000023219 2024-03-26T06:41:39.078 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:39.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.079 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-26T06:41:39.079 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-26T06:41:39.079 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-26T06:41:39.079 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.079 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.079 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.079 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:39.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.080 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-26T06:41:39.080 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-03-26T06:41:39.080 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-03-26T06:41:39.080 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-26T06:41:39.080 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.080 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.080 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.081 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-26T06:41:39.081 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-03-26T06:41:39.081 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-26T06:41:39.081 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-26T06:41:39.081 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2024-03-26T06:41:39.081 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-26T06:41:39.081 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-26T06:41:39.081 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 19325, 2024-03-26T06:41:39.081 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 2, 2024-03-26T06:41:39.082 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 19323, 2024-03-26T06:41:39.082 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-03-26T06:41:39.082 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 19323, 2024-03-26T06:41:39.082 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-03-26T06:41:39.082 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-03-26T06:41:39.082 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-03-26T06:41:39.082 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 5, 2024-03-26T06:41:39.082 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2024-03-26T06:41:39.083 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2024-03-26T06:41:39.083 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-26T06:41:39.083 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-03-26T06:41:39.083 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-26T06:41:39.083 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-26T06:41:39.083 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-26T06:41:39.083 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-26T06:41:39.083 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-26T06:41:39.084 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-03-26T06:41:39.084 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-03-26T06:41:39.084 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-03-26T06:41:39.084 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-03-26T06:41:39.084 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-26T06:41:39.084 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-03-26T06:41:39.084 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-26T06:41:39.084 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 0, 2024-03-26T06:41:39.084 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 0, 2024-03-26T06:41:39.085 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-03-26T06:41:39.085 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T06:41:39.085 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-26T06:41:39.085 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T06:41:39.085 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-26T06:41:39.085 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-03-26T06:41:39.085 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T06:41:39.085 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T06:41:39.085 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-03-26T06:41:39.086 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-26T06:41:39.086 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T06:41:39.086 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-03-26T06:41:39.086 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T06:41:39.086 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T06:41:39.086 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T06:41:39.086 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-03-26T06:41:39.086 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-03-26T06:41:39.086 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-03-26T06:41:39.087 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-03-26T06:41:39.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.087 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-26T06:41:39.087 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-26T06:41:39.087 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-26T06:41:39.087 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-26T06:41:39.087 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-03-26T06:41:39.087 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-03-26T06:41:39.087 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-03-26T06:41:39.088 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-26T06:41:39.088 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-26T06:41:39.088 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-03-26T06:41:39.088 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-03-26T06:41:39.088 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-26T06:41:39.088 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-26T06:41:39.088 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-26T06:41:39.088 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-26T06:41:39.089 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-26T06:41:39.089 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-26T06:41:39.089 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-26T06:41:39.089 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T06:41:39.089 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-03-26T06:41:39.089 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-26T06:41:39.089 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-26T06:41:39.089 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-26T06:41:39.089 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-26T06:41:39.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.090 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-26T06:41:39.090 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-26T06:41:39.090 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2024-03-26T06:41:39.090 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-26T06:41:39.090 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-26T06:41:39.090 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 96279, 2024-03-26T06:41:39.090 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-26T06:41:39.091 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-26T06:41:39.091 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2024-03-26T06:41:39.091 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-26T06:41:39.091 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-26T06:41:39.091 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 134, 2024-03-26T06:41:39.091 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 9, 2024-03-26T06:41:39.091 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-26T06:41:39.091 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-26T06:41:39.091 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194352, 2024-03-26T06:41:39.092 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 362187366, 2024-03-26T06:41:39.092 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 362182105, 2024-03-26T06:41:39.092 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-26T06:41:39.092 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.092 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.092 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.092 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 96279 2024-03-26T06:41:39.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.093 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-26T06:41:39.093 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 768, 2024-03-26T06:41:39.093 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 19286, 2024-03-26T06:41:39.093 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 18518, 2024-03-26T06:41:39.093 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 58, 2024-03-26T06:41:39.093 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 109, 2024-03-26T06:41:39.093 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 51, 2024-03-26T06:41:39.094 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 19325, 2024-03-26T06:41:39.094 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 19434, 2024-03-26T06:41:39.094 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 109, 2024-03-26T06:41:39.094 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-03-26T06:41:39.094 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-03-26T06:41:39.094 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-03-26T06:41:39.094 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 71364, 2024-03-26T06:41:39.094 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215420 2024-03-26T06:41:39.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.095 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-26T06:41:39.095 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-03-26T06:41:39.095 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-03-26T06:41:39.095 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-03-26T06:41:39.095 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-03-26T06:41:39.095 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-03-26T06:41:39.096 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-26T06:41:39.096 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.096 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.096 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.096 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-26T06:41:39.096 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.096 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.096 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.097 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-26T06:41:39.097 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.097 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.097 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.097 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-26T06:41:39.097 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.097 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.097 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.098 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-26T06:41:39.098 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.098 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.098 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.098 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-26T06:41:39.098 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.099 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.099 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.099 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-26T06:41:39.099 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.099 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.099 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.099 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-26T06:41:39.100 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.100 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.100 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.100 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-26T06:41:39.100 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.100 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.100 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.101 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-26T06:41:39.101 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.101 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.101 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.101 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-26T06:41:39.101 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.101 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.101 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.102 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-26T06:41:39.102 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.102 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.103 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.103 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-26T06:41:39.103 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.103 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.103 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.103 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-26T06:41:39.104 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.104 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.104 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.104 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-26T06:41:39.104 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.104 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.104 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.105 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-26T06:41:39.105 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.105 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.105 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.105 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.105 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-26T06:41:39.105 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.105 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.105 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.106 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-26T06:41:39.106 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.106 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.106 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.106 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-26T06:41:39.106 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.107 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.107 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.107 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-26T06:41:39.107 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.107 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.107 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.107 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-26T06:41:39.108 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.108 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.108 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.108 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-26T06:41:39.108 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.108 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.108 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.109 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-26T06:41:39.109 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.109 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.109 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.109 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-26T06:41:39.109 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.109 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.109 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.110 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.110 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-26T06:41:39.110 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.110 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.110 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.110 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.110 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-26T06:41:39.110 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.111 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.111 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.111 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-26T06:41:39.111 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.111 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.111 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.111 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-26T06:41:39.112 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.112 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.112 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.112 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-26T06:41:39.112 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.112 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.112 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.113 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-26T06:41:39.113 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-26T06:41:39.113 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-26T06:41:39.113 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.113 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.113 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.113 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:39.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.113 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-26T06:41:39.113 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 2, 2024-03-26T06:41:39.114 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 3, 2024-03-26T06:41:39.114 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 1, 2024-03-26T06:41:39.114 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 3, 2024-03-26T06:41:39.114 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-26T06:41:39.114 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-03-26T06:41:39.114 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-03-26T06:41:39.114 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 3298, 2024-03-26T06:41:39.114 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-26T06:41:39.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.115 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-26T06:41:39.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-26T06:41:39.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-26T06:41:39.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:39.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:39.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:39.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:39.115 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:39.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:39.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:39.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:39.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:39.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:39.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:39.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:39.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:39.116 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:39.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:39.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:39.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:39.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:39.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:39.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:39.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:39.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:39.117 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:39.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:39.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:39.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:39.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:39.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:39.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:39.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-26T06:41:39.118 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:39.119 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:39.119 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:39.119 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:39.119 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 265529, 2024-03-26T06:41:39.119 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 960, 2024-03-26T06:41:39.119 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-26T06:41:39.119 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-26T06:41:39.119 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-26T06:41:39.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-26T06:41:39.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:39.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:39.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:39.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:39.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8032, 2024-03-26T06:41:39.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 167, 2024-03-26T06:41:39.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:39.120 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:39.120 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:39.121 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-26T06:41:39.121 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 15625349, 2024-03-26T06:41:39.121 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 97660, 2024-03-26T06:41:39.121 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:39.121 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-26T06:41:39.121 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:39.121 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-26T06:41:39.121 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.121 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-26T06:41:39.122 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-26T06:41:39.122 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-26T06:41:39.122 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 4581, 2024-03-26T06:41:39.122 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 21, 2024-03-26T06:41:39.122 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 3, 2024-03-26T06:41:39.122 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 4581, 2024-03-26T06:41:39.122 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-26T06:41:39.122 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4581, 2024-03-26T06:41:39.122 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 7.433849434, 2024-03-26T06:41:39.123 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001622756 2024-03-26T06:41:39.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.123 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-26T06:41:39.123 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-26T06:41:39.123 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4581, 2024-03-26T06:41:39.123 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 4582 2024-03-26T06:41:39.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.123 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 4581, 2024-03-26T06:41:39.123 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 4581, 2024-03-26T06:41:39.124 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2024-03-26T06:41:39.124 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-26T06:41:39.124 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-26T06:41:39.124 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 2620, 2024-03-26T06:41:39.124 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2024-03-26T06:41:39.124 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 1959, 2024-03-26T06:41:39.124 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2024-03-26T06:41:39.124 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2024-03-26T06:41:39.124 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-26T06:41:39.125 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-26T06:41:39.125 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-26T06:41:39.125 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-26T06:41:39.125 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2024-03-26T06:41:39.125 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-26T06:41:39.125 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-26T06:41:39.125 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-26T06:41:39.125 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 1, 2024-03-26T06:41:39.126 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2024-03-26T06:41:39.126 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-26T06:41:39.126 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-26T06:41:39.126 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-26T06:41:39.126 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-26T06:41:39.126 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-26T06:41:39.126 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-26T06:41:39.126 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T06:41:39.126 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-26T06:41:39.127 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-26T06:41:39.127 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-26T06:41:39.127 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-26T06:41:39.127 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-26T06:41:39.127 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-26T06:41:39.127 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-26T06:41:39.127 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-26T06:41:39.127 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-26T06:41:39.127 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-26T06:41:39.127 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-26T06:41:39.128 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-26T06:41:39.128 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-26T06:41:39.128 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-26T06:41:39.128 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-26T06:41:39.128 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-26T06:41:39.128 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-26T06:41:39.128 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-26T06:41:39.128 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-26T06:41:39.128 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 116, 2024-03-26T06:41:39.129 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-26T06:41:39.129 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 31, 2024-03-26T06:41:39.129 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 10, 2024-03-26T06:41:39.129 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 11, 2024-03-26T06:41:39.129 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 1, 2024-03-26T06:41:39.129 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-26T06:41:39.129 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2024-03-26T06:41:39.129 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2024-03-26T06:41:39.129 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2024-03-26T06:41:39.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.130 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-26T06:41:39.130 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2024-03-26T06:41:39.130 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-26T06:41:39.130 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-26T06:41:39.130 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-26T06:41:39.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.130 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-26T06:41:39.130 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-26T06:41:39.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.131 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-26T06:41:39.131 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-26T06:41:39.131 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T06:41:39.131 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-26T06:41:39.131 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-03-26T06:41:39.131 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-03-26T06:41:39.131 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-03-26T06:41:39.131 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-26T06:41:39.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.132 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:39.132 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:39.132 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-26T06:41:39.132 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:39.132 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 5015, 2024-03-26T06:41:39.132 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 360744774, 2024-03-26T06:41:39.132 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:39.132 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 5015, 2024-03-26T06:41:39.133 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:39.133 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:39.133 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 5015, 2024-03-26T06:41:39.133 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 360744774, 2024-03-26T06:41:39.133 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:39.133 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.133 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.133 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.133 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:39.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.134 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-26T06:41:39.134 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:39.134 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-26T06:41:39.134 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:39.134 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:39.134 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:39.134 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:39.135 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:39.135 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 4581, 2024-03-26T06:41:39.135 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 358008939, 2024-03-26T06:41:39.135 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4581, 2024-03-26T06:41:39.135 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 358008939, 2024-03-26T06:41:39.135 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:39.135 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.135 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.136 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.136 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:39.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.136 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-26T06:41:39.136 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:39.136 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-26T06:41:39.136 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:39.136 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:39.137 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:39.137 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:39.137 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:39.137 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 4581, 2024-03-26T06:41:39.137 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 4581, 2024-03-26T06:41:39.137 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4581, 2024-03-26T06:41:39.137 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 4581, 2024-03-26T06:41:39.137 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:39.137 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.138 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.138 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.138 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:39.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.138 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-26T06:41:39.138 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:39.138 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-26T06:41:39.138 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:39.138 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:39.139 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:39.139 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:39.139 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:39.139 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:39.139 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:39.139 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-26T06:41:39.139 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-26T06:41:39.139 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:39.139 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.140 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.140 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.140 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:39.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:39.140 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x562452c50960": { 2024-03-26T06:41:39.140 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:39.140 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-26T06:41:39.140 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:39.140 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:39.141 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:39.141 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:39.141 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:39.141 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:39.141 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:39.141 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-26T06:41:39.141 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-26T06:41:39.141 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:39.141 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:39.141 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:39.142 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:39.142 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:39.142 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:39.142 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-26T06:41:39.973 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-26T06:41:39.974 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-26T06:41:39.974 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-26T06:41:39.974 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-26T06:41:39.974 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T06:41:39.974 DEBUG:tasks.check_counter:Getting stats from i 2024-03-26T06:41:39.974 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-26T06:41:40.245 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:41:40.625 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:40 smithi149 ceph-mon[36681]: pgmap v591: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 206 KiB/s wr, 17 op/s 2024-03-26T06:41:40.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:40 smithi066 ceph-mon[36504]: pgmap v591: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 206 KiB/s wr, 17 op/s 2024-03-26T06:41:40.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:40 smithi037 ceph-mon[31974]: pgmap v591: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 206 KiB/s wr, 17 op/s 2024-03-26T06:41:41.881 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-26T06:41:41.882 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:41.882 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 253441, 2024-03-26T06:41:41.882 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 360506, 2024-03-26T06:41:41.882 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 75073647, 2024-03-26T06:41:41.883 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 396150528, 2024-03-26T06:41:41.883 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 16, 2024-03-26T06:41:41.883 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-03-26T06:41:41.883 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 13.939805853, 2024-03-26T06:41:41.883 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 6.656444228, 2024-03-26T06:41:41.884 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 8.061494467, 2024-03-26T06:41:41.884 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.934783453, 2024-03-26T06:41:41.884 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:41.884 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 360502, 2024-03-26T06:41:41.884 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 14.724232342, 2024-03-26T06:41:41.885 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000040843 2024-03-26T06:41:41.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.885 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:41.885 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 345155, 2024-03-26T06:41:41.885 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.077500493, 2024-03-26T06:41:41.886 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000224 2024-03-26T06:41:41.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.886 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T06:41:41.886 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T06:41:41.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.887 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:41.887 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 54012, 2024-03-26T06:41:41.887 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 54558, 2024-03-26T06:41:41.888 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 17628877, 2024-03-26T06:41:41.888 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 162497979, 2024-03-26T06:41:41.888 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 9, 2024-03-26T06:41:41.888 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 7, 2024-03-26T06:41:41.888 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 3.668445968, 2024-03-26T06:41:41.888 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 1.410091896, 2024-03-26T06:41:41.889 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 1.761007239, 2024-03-26T06:41:41.889 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.657784660, 2024-03-26T06:41:41.889 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:41.889 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 54557, 2024-03-26T06:41:41.889 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 2.339797977, 2024-03-26T06:41:41.890 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000042887 2024-03-26T06:41:41.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.890 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:41.890 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 65749, 2024-03-26T06:41:41.890 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.012181981, 2024-03-26T06:41:41.890 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000185 2024-03-26T06:41:41.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.891 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1948112, 2024-03-26T06:41:41.891 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 213856 2024-03-26T06:41:41.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.891 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:41.891 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 152968, 2024-03-26T06:41:41.891 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 130775, 2024-03-26T06:41:41.891 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 47990514, 2024-03-26T06:41:41.891 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 124565405, 2024-03-26T06:41:41.892 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 11, 2024-03-26T06:41:41.892 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-03-26T06:41:41.892 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 8.062252646, 2024-03-26T06:41:41.892 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 3.299477921, 2024-03-26T06:41:41.892 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 5.510072285, 2024-03-26T06:41:41.892 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.438758888, 2024-03-26T06:41:41.892 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:41.892 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 130775, 2024-03-26T06:41:41.892 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 3.989338997, 2024-03-26T06:41:41.893 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000030505 2024-03-26T06:41:41.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.893 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:41.893 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 251923, 2024-03-26T06:41:41.893 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.038601265, 2024-03-26T06:41:41.893 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000153 2024-03-26T06:41:41.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.893 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-26T06:41:41.893 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 615648 2024-03-26T06:41:41.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.894 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-26T06:41:41.894 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-26T06:41:41.894 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-26T06:41:41.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.894 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-26T06:41:41.894 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-26T06:41:41.894 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-26T06:41:41.894 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 58323, 2024-03-26T06:41:41.895 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 64.742858862, 2024-03-26T06:41:41.895 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001110074 2024-03-26T06:41:41.895 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:41.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.895 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-26T06:41:41.895 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-26T06:41:41.895 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-26T06:41:41.895 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 37714, 2024-03-26T06:41:41.895 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 5.116158277, 2024-03-26T06:41:41.896 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000135656 2024-03-26T06:41:41.896 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:41.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.896 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-26T06:41:41.896 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 134681, 2024-03-26T06:41:41.896 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 125379, 2024-03-26T06:41:41.896 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-26T06:41:41.897 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 125379, 2024-03-26T06:41:41.897 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 415.801964271, 2024-03-26T06:41:41.897 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003316360 2024-03-26T06:41:41.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.897 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-26T06:41:41.897 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 9302, 2024-03-26T06:41:41.897 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-26T06:41:41.897 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-26T06:41:41.897 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 60, 2024-03-26T06:41:41.898 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 5, 2024-03-26T06:41:41.898 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 8, 2024-03-26T06:41:41.898 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 1254, 2024-03-26T06:41:41.898 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 454, 2024-03-26T06:41:41.898 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 716, 2024-03-26T06:41:41.898 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 84, 2024-03-26T06:41:41.898 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 604, 2024-03-26T06:41:41.899 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 344, 2024-03-26T06:41:41.899 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 118, 2024-03-26T06:41:41.899 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 118, 2024-03-26T06:41:41.899 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 20, 2024-03-26T06:41:41.899 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 212243, 2024-03-26T06:41:41.899 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 122036, 2024-03-26T06:41:41.899 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 9292, 2024-03-26T06:41:41.899 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 67, 2024-03-26T06:41:41.899 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-26T06:41:41.900 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-26T06:41:41.900 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 2914, 2024-03-26T06:41:41.900 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 6861, 2024-03-26T06:41:41.900 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-26T06:41:41.900 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 5, 2024-03-26T06:41:41.900 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 232, 2024-03-26T06:41:41.900 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 6, 2024-03-26T06:41:41.900 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 284, 2024-03-26T06:41:41.900 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-26T06:41:41.901 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-03-26T06:41:41.901 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-26T06:41:41.901 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 705, 2024-03-26T06:41:41.901 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 221354645, 2024-03-26T06:41:41.901 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 6, 2024-03-26T06:41:41.901 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T06:41:41.901 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-26T06:41:41.901 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T06:41:41.901 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-26T06:41:41.902 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-03-26T06:41:41.902 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T06:41:41.902 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T06:41:41.902 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-03-26T06:41:41.902 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 4, 2024-03-26T06:41:41.902 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 28509, 2024-03-26T06:41:41.902 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 83194, 2024-03-26T06:41:41.902 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T06:41:41.902 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T06:41:41.903 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T06:41:41.903 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 57041, 2024-03-26T06:41:41.903 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 14861, 2024-03-26T06:41:41.903 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 7007, 2024-03-26T06:41:41.903 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 87680 2024-03-26T06:41:41.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.903 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-26T06:41:41.903 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 17, 2024-03-26T06:41:41.904 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 123, 2024-03-26T06:41:41.904 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 85, 2024-03-26T06:41:41.904 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 134, 2024-03-26T06:41:41.904 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 143, 2024-03-26T06:41:41.904 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-03-26T06:41:41.904 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-26T06:41:41.904 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-26T06:41:41.904 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 19793, 2024-03-26T06:41:41.904 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 19793, 2024-03-26T06:41:41.905 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-26T06:41:41.905 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-26T06:41:41.905 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-26T06:41:41.905 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-26T06:41:41.905 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-26T06:41:41.905 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-26T06:41:41.905 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-26T06:41:41.905 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T06:41:41.905 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 6, 2024-03-26T06:41:41.905 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-26T06:41:41.906 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 13, 2024-03-26T06:41:41.906 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-26T06:41:41.906 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-26T06:41:41.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.906 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-26T06:41:41.906 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-26T06:41:41.906 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 100616, 2024-03-26T06:41:41.906 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 13491, 2024-03-26T06:41:41.906 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 12291, 2024-03-26T06:41:41.907 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 88325, 2024-03-26T06:41:41.907 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-26T06:41:41.907 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 1200, 2024-03-26T06:41:41.907 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 145, 2024-03-26T06:41:41.907 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 17, 2024-03-26T06:41:41.907 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 15, 2024-03-26T06:41:41.907 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 130, 2024-03-26T06:41:41.907 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 8, 2024-03-26T06:41:41.908 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-26T06:41:41.908 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 2, 2024-03-26T06:41:41.908 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 23069759, 2024-03-26T06:41:41.908 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 389985310, 2024-03-26T06:41:41.908 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-03-26T06:41:41.908 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-26T06:41:41.908 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 31377, 2024-03-26T06:41:41.908 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 194.755493385, 2024-03-26T06:41:41.909 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.006206950 2024-03-26T06:41:41.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.909 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-03-26T06:41:41.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.909 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-26T06:41:41.909 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 588, 2024-03-26T06:41:41.909 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 20629, 2024-03-26T06:41:41.909 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 20041, 2024-03-26T06:41:41.910 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 152, 2024-03-26T06:41:41.910 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 283, 2024-03-26T06:41:41.910 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 131, 2024-03-26T06:41:41.910 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 1254, 2024-03-26T06:41:41.910 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 38190, 2024-03-26T06:41:41.910 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 36936, 2024-03-26T06:41:41.910 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 118, 2024-03-26T06:41:41.910 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 21966, 2024-03-26T06:41:41.911 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 21848, 2024-03-26T06:41:41.911 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 74824, 2024-03-26T06:41:41.911 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 207228 2024-03-26T06:41:41.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.911 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-26T06:41:41.911 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 184963, 2024-03-26T06:41:41.911 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 1602, 2024-03-26T06:41:41.911 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 134681, 2024-03-26T06:41:41.911 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 3033, 2024-03-26T06:41:41.912 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 8527, 2024-03-26T06:41:41.912 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-26T06:41:41.912 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 19795, 2024-03-26T06:41:41.912 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 50.755700256, 2024-03-26T06:41:41.912 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002564066 2024-03-26T06:41:41.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.912 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-26T06:41:41.912 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1638, 2024-03-26T06:41:41.912 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 33.157352478, 2024-03-26T06:41:41.912 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.020242583 2024-03-26T06:41:41.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.913 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-26T06:41:41.913 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.913 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.913 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.913 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-26T06:41:41.913 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.913 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.914 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.914 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-26T06:41:41.914 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 41694, 2024-03-26T06:41:41.914 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 64.162880563, 2024-03-26T06:41:41.914 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001538899 2024-03-26T06:41:41.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.915 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-26T06:41:41.915 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.915 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.915 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.915 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-26T06:41:41.915 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.915 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.915 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.916 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-26T06:41:41.916 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.916 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.916 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.916 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-26T06:41:41.916 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.916 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.917 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.917 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-26T06:41:41.917 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.917 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.917 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.918 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-26T06:41:41.918 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.918 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.918 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.919 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-26T06:41:41.919 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.919 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.919 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.919 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-26T06:41:41.919 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.919 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.919 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.920 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-26T06:41:41.920 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.920 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.920 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.920 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-26T06:41:41.920 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.921 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.921 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.921 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-26T06:41:41.921 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 36204, 2024-03-26T06:41:41.921 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 209.213878266, 2024-03-26T06:41:41.921 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.005778750 2024-03-26T06:41:41.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.921 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-26T06:41:41.922 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 5135, 2024-03-26T06:41:41.922 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 13.732384419, 2024-03-26T06:41:41.922 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002674271 2024-03-26T06:41:41.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.922 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-26T06:41:41.922 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.922 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.923 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.923 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-26T06:41:41.923 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.923 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.923 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.923 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-26T06:41:41.923 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.924 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.924 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.924 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-26T06:41:41.924 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.924 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.924 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.925 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-26T06:41:41.925 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.925 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.925 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.925 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-26T06:41:41.925 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.925 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.926 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.926 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-26T06:41:41.926 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.926 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.926 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.927 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-26T06:41:41.927 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1120, 2024-03-26T06:41:41.927 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.984124987, 2024-03-26T06:41:41.927 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000878683 2024-03-26T06:41:41.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.927 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-26T06:41:41.927 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.927 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.927 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.928 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-26T06:41:41.928 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.928 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.928 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.928 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-26T06:41:41.928 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.928 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.929 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.929 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-26T06:41:41.929 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 19793, 2024-03-26T06:41:41.929 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 43.795643302, 2024-03-26T06:41:41.929 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002212683 2024-03-26T06:41:41.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.929 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-26T06:41:41.929 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-26T06:41:41.930 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-26T06:41:41.930 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.930 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.930 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.930 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:41.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.930 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-26T06:41:41.930 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-03-26T06:41:41.930 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 1, 2024-03-26T06:41:41.931 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 0, 2024-03-26T06:41:41.931 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-03-26T06:41:41.931 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-26T06:41:41.931 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 27315, 2024-03-26T06:41:41.931 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 27315, 2024-03-26T06:41:41.931 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 1031, 2024-03-26T06:41:41.931 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-26T06:41:41.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.931 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-26T06:41:41.932 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-26T06:41:41.932 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-26T06:41:41.932 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:41.932 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:41.932 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:41.932 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:41.932 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:41.932 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:41.932 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:41.933 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:41.933 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:41.933 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:41.933 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:41.933 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:41.933 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:41.933 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:41.933 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:41.933 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:41.934 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:41.934 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:41.934 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:41.934 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:41.934 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:41.934 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:41.934 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:41.934 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:41.934 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:41.935 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:41.935 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:41.935 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:41.935 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:41.935 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-26T06:41:41.935 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:41.935 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:41.935 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:41.935 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:41.936 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 321650, 2024-03-26T06:41:41.936 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 398, 2024-03-26T06:41:41.936 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 264, 2024-03-26T06:41:41.936 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 3, 2024-03-26T06:41:41.936 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-26T06:41:41.936 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-26T06:41:41.936 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:41.936 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:41.936 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:41.936 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:41.937 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8032, 2024-03-26T06:41:41.937 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 167, 2024-03-26T06:41:41.937 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:41.937 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:41.937 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:41.937 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-26T06:41:41.937 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 2692659, 2024-03-26T06:41:41.937 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 24038, 2024-03-26T06:41:41.937 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:41.938 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-26T06:41:41.938 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:41.938 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-26T06:41:41.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.938 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-26T06:41:41.938 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-26T06:41:41.938 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-26T06:41:41.938 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 54413, 2024-03-26T06:41:41.939 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 390292074, 2024-03-26T06:41:41.939 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 28, 2024-03-26T06:41:41.939 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 54413, 2024-03-26T06:41:41.939 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-26T06:41:41.939 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 54413, 2024-03-26T06:41:41.939 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 378.751335103, 2024-03-26T06:41:41.939 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.006960677 2024-03-26T06:41:41.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.940 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-26T06:41:41.940 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-26T06:41:41.940 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 54413, 2024-03-26T06:41:41.940 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 55652 2024-03-26T06:41:41.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.940 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 54413, 2024-03-26T06:41:41.940 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 891, 2024-03-26T06:41:41.940 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 53522, 2024-03-26T06:41:41.940 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-26T06:41:41.941 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-26T06:41:41.941 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-03-26T06:41:41.941 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 21, 2024-03-26T06:41:41.941 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 891, 2024-03-26T06:41:41.941 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 32268, 2024-03-26T06:41:41.941 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 507, 2024-03-26T06:41:41.941 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-26T06:41:41.941 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-26T06:41:41.941 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-26T06:41:41.942 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 5, 2024-03-26T06:41:41.942 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 19936, 2024-03-26T06:41:41.942 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-26T06:41:41.942 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-26T06:41:41.942 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-26T06:41:41.942 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 0, 2024-03-26T06:41:41.942 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 42, 2024-03-26T06:41:41.942 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-26T06:41:41.942 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-26T06:41:41.942 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-26T06:41:41.943 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-26T06:41:41.943 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-26T06:41:41.943 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-26T06:41:41.943 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T06:41:41.943 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-26T06:41:41.943 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-26T06:41:41.943 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-26T06:41:41.943 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-26T06:41:41.943 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-26T06:41:41.944 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-26T06:41:41.944 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-26T06:41:41.944 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-26T06:41:41.944 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-26T06:41:41.944 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-26T06:41:41.944 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-26T06:41:41.944 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-26T06:41:41.944 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-26T06:41:41.945 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-26T06:41:41.945 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-26T06:41:41.945 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-26T06:41:41.945 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-26T06:41:41.945 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-26T06:41:41.945 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-26T06:41:41.945 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 116, 2024-03-26T06:41:41.945 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-26T06:41:41.945 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 31, 2024-03-26T06:41:41.946 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2024-03-26T06:41:41.946 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2024-03-26T06:41:41.946 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-26T06:41:41.946 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-26T06:41:41.946 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 1473, 2024-03-26T06:41:41.946 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-03-26T06:41:41.946 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 509 2024-03-26T06:41:41.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.947 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-26T06:41:41.947 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-03-26T06:41:41.947 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-26T06:41:41.947 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-26T06:41:41.947 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-26T06:41:41.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.947 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-26T06:41:41.947 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-26T06:41:41.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.948 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-26T06:41:41.948 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-26T06:41:41.948 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 6, 2024-03-26T06:41:41.948 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-26T06:41:41.948 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 6, 2024-03-26T06:41:41.948 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 19793, 2024-03-26T06:41:41.948 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 19793, 2024-03-26T06:41:41.948 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-26T06:41:41.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.948 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:41.949 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:41.949 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-26T06:41:41.949 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:41.949 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 460417, 2024-03-26T06:41:41.949 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 102491594, 2024-03-26T06:41:41.949 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:41.949 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 460417, 2024-03-26T06:41:41.949 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:41.950 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:41.950 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 460417, 2024-03-26T06:41:41.950 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 102491594, 2024-03-26T06:41:41.950 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:41.950 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.950 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.950 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.951 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:41.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.951 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-26T06:41:41.951 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:41.951 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-26T06:41:41.951 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:41.951 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:41.951 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:41.951 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:41.952 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:41.952 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 54413, 2024-03-26T06:41:41.952 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 392292631, 2024-03-26T06:41:41.952 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 54413, 2024-03-26T06:41:41.952 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 392292631, 2024-03-26T06:41:41.952 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:41.952 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.952 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.952 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.953 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:41.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.953 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-26T06:41:41.953 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:41.953 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-26T06:41:41.953 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:41.953 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:41.953 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:41.953 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:41.954 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:41.954 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 54413, 2024-03-26T06:41:41.954 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 54413, 2024-03-26T06:41:41.954 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 54413, 2024-03-26T06:41:41.954 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 54413, 2024-03-26T06:41:41.954 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:41.954 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.954 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.954 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.955 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:41.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.955 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-26T06:41:41.955 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:41.955 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-26T06:41:41.955 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:41.955 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 19793, 2024-03-26T06:41:41.955 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 2000557, 2024-03-26T06:41:41.956 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:41.956 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 19793, 2024-03-26T06:41:41.956 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:41.956 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:41.956 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 891, 2024-03-26T06:41:41.956 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 2000557, 2024-03-26T06:41:41.956 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:41.956 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.956 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.956 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.957 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:41.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:41.957 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x560863eea6e0": { 2024-03-26T06:41:41.957 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:41.957 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-26T06:41:41.957 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:41.957 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 100616, 2024-03-26T06:41:41.957 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 385791006, 2024-03-26T06:41:41.958 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:41.958 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 100616, 2024-03-26T06:41:41.958 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:41.958 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:41.958 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 31377, 2024-03-26T06:41:41.958 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 385791006, 2024-03-26T06:41:41.958 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:41.958 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:41.959 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:41.959 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:41.959 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:41.959 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:41.959 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-26T06:41:42.661 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:42 smithi149 ceph-mon[36681]: pgmap v592: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 253 KiB/s wr, 26 op/s 2024-03-26T06:41:42.758 INFO:tasks.check_counter:Daemon mds.i mds.exported=5 2024-03-26T06:41:42.758 INFO:tasks.check_counter:Daemon mds.i mds.imported=6 2024-03-26T06:41:42.759 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-26T06:41:42.759 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-26T06:41:42.759 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T06:41:42.759 DEBUG:tasks.check_counter:Getting stats from l 2024-03-26T06:41:42.759 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f shell --fsid a86fb218-eb37-11ee-95d0-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-26T06:41:42.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:42 smithi066 ceph-mon[36504]: pgmap v592: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 253 KiB/s wr, 26 op/s 2024-03-26T06:41:42.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:42 smithi037 ceph-mon[31974]: pgmap v592: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 253 KiB/s wr, 26 op/s 2024-03-26T06:41:42.998 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T06:41:44.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:44 smithi149 ceph-mon[36681]: pgmap v593: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 138 KiB/s wr, 19 op/s 2024-03-26T06:41:44.740 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-26T06:41:44.740 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T06:41:44.740 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 2477, 2024-03-26T06:41:44.740 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2479, 2024-03-26T06:41:44.740 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 139145194, 2024-03-26T06:41:44.740 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 787687, 2024-03-26T06:41:44.741 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 8, 2024-03-26T06:41:44.741 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-26T06:41:44.741 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.289115105, 2024-03-26T06:41:44.741 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.071900288, 2024-03-26T06:41:44.741 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.124896221, 2024-03-26T06:41:44.741 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.072035555, 2024-03-26T06:41:44.741 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:44.742 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2477, 2024-03-26T06:41:44.742 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.093792286, 2024-03-26T06:41:44.742 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000037865 2024-03-26T06:41:44.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.742 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:44.742 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.743 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.743 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.743 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T06:41:44.743 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T06:41:44.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.743 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T06:41:44.743 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1366, 2024-03-26T06:41:44.744 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1623, 2024-03-26T06:41:44.744 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 134567254, 2024-03-26T06:41:44.744 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 1047300, 2024-03-26T06:41:44.744 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 9, 2024-03-26T06:41:44.744 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 7, 2024-03-26T06:41:44.744 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.175385723, 2024-03-26T06:41:44.744 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.045261221, 2024-03-26T06:41:44.745 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.084408329, 2024-03-26T06:41:44.745 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.034698103, 2024-03-26T06:41:44.745 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:44.745 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1621, 2024-03-26T06:41:44.745 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.060899330, 2024-03-26T06:41:44.745 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000037568 2024-03-26T06:41:44.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.746 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:44.746 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.746 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.746 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.746 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-26T06:41:44.746 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 614752 2024-03-26T06:41:44.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.747 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T06:41:44.747 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1120, 2024-03-26T06:41:44.747 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1062, 2024-03-26T06:41:44.747 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 115282737, 2024-03-26T06:41:44.747 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 429930, 2024-03-26T06:41:44.747 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2024-03-26T06:41:44.748 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-26T06:41:44.748 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.191515133, 2024-03-26T06:41:44.748 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.034159316, 2024-03-26T06:41:44.748 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.139124491, 2024-03-26T06:41:44.748 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.024184284, 2024-03-26T06:41:44.748 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T06:41:44.748 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1061, 2024-03-26T06:41:44.749 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.036161270, 2024-03-26T06:41:44.749 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000034082 2024-03-26T06:41:44.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.749 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-26T06:41:44.749 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.749 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.750 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.750 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1942720, 2024-03-26T06:41:44.750 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 212736 2024-03-26T06:41:44.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.750 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-26T06:41:44.750 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-26T06:41:44.751 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-26T06:41:44.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.751 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-26T06:41:44.751 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-26T06:41:44.751 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-26T06:41:44.751 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7580, 2024-03-26T06:41:44.752 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.185359972, 2024-03-26T06:41:44.752 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000024453 2024-03-26T06:41:44.752 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:44.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.752 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-26T06:41:44.752 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-26T06:41:44.752 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-26T06:41:44.753 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.753 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.753 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.753 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:44.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.753 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-26T06:41:44.753 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-03-26T06:41:44.754 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-03-26T06:41:44.754 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-26T06:41:44.754 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.754 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.754 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.754 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-26T06:41:44.755 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-03-26T06:41:44.755 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-26T06:41:44.755 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-26T06:41:44.755 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2024-03-26T06:41:44.755 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-26T06:41:44.755 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-26T06:41:44.755 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 20453, 2024-03-26T06:41:44.755 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 1257, 2024-03-26T06:41:44.756 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 19196, 2024-03-26T06:41:44.756 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-03-26T06:41:44.756 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 18658, 2024-03-26T06:41:44.756 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-03-26T06:41:44.756 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-03-26T06:41:44.756 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-03-26T06:41:44.756 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 12, 2024-03-26T06:41:44.757 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2024-03-26T06:41:44.757 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2024-03-26T06:41:44.757 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-26T06:41:44.757 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-03-26T06:41:44.757 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-26T06:41:44.757 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-26T06:41:44.757 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-26T06:41:44.758 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-26T06:41:44.758 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-26T06:41:44.758 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-03-26T06:41:44.758 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-03-26T06:41:44.758 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-03-26T06:41:44.758 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-03-26T06:41:44.759 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-26T06:41:44.759 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-03-26T06:41:44.759 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-26T06:41:44.759 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 0, 2024-03-26T06:41:44.759 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 0, 2024-03-26T06:41:44.759 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-03-26T06:41:44.759 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T06:41:44.759 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-26T06:41:44.760 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T06:41:44.760 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-26T06:41:44.760 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-03-26T06:41:44.760 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T06:41:44.760 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T06:41:44.760 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-03-26T06:41:44.761 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-26T06:41:44.761 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T06:41:44.761 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-03-26T06:41:44.761 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T06:41:44.761 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T06:41:44.761 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T06:41:44.761 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-03-26T06:41:44.761 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-03-26T06:41:44.762 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-03-26T06:41:44.762 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-03-26T06:41:44.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.762 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-26T06:41:44.762 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-26T06:41:44.762 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-26T06:41:44.762 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-26T06:41:44.763 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-03-26T06:41:44.763 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-03-26T06:41:44.763 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-03-26T06:41:44.763 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-26T06:41:44.763 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-26T06:41:44.763 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-03-26T06:41:44.763 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-03-26T06:41:44.764 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-26T06:41:44.764 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-26T06:41:44.764 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-26T06:41:44.764 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-26T06:41:44.764 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-26T06:41:44.764 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-26T06:41:44.764 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-26T06:41:44.764 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T06:41:44.765 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-03-26T06:41:44.765 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-26T06:41:44.765 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-26T06:41:44.765 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-26T06:41:44.765 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-26T06:41:44.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.765 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-26T06:41:44.765 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-26T06:41:44.766 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2024-03-26T06:41:44.766 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-26T06:41:44.766 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-26T06:41:44.766 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 100616, 2024-03-26T06:41:44.766 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-26T06:41:44.766 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-26T06:41:44.766 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2024-03-26T06:41:44.767 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-26T06:41:44.767 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-26T06:41:44.767 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 130, 2024-03-26T06:41:44.767 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 10, 2024-03-26T06:41:44.767 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-26T06:41:44.767 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-26T06:41:44.767 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 23069759, 2024-03-26T06:41:44.768 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 389985310, 2024-03-26T06:41:44.768 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 389979886, 2024-03-26T06:41:44.768 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-26T06:41:44.768 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.768 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.768 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.768 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 100616 2024-03-26T06:41:44.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.769 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-26T06:41:44.769 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 544, 2024-03-26T06:41:44.769 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 20512, 2024-03-26T06:41:44.769 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 19968, 2024-03-26T06:41:44.769 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 55, 2024-03-26T06:41:44.769 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 103, 2024-03-26T06:41:44.769 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 48, 2024-03-26T06:41:44.770 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 20453, 2024-03-26T06:41:44.770 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 20675, 2024-03-26T06:41:44.770 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 222, 2024-03-26T06:41:44.770 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-03-26T06:41:44.770 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-03-26T06:41:44.770 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-03-26T06:41:44.770 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 72176, 2024-03-26T06:41:44.771 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215420 2024-03-26T06:41:44.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.771 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-26T06:41:44.771 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-03-26T06:41:44.771 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-03-26T06:41:44.771 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-03-26T06:41:44.771 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-03-26T06:41:44.772 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-03-26T06:41:44.772 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-26T06:41:44.772 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.772 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.772 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.772 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-26T06:41:44.773 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.773 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.773 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.773 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-26T06:41:44.773 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.773 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.773 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.774 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-26T06:41:44.774 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.774 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.774 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.774 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-26T06:41:44.774 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.775 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.775 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-26T06:41:44.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.775 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.776 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-26T06:41:44.776 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.776 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.776 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.776 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-26T06:41:44.776 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.776 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.777 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.777 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-26T06:41:44.777 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.777 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.777 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.778 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-26T06:41:44.778 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.778 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.778 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.778 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-26T06:41:44.778 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.778 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.779 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.779 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-26T06:41:44.779 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.780 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:44 smithi037 ceph-mon[31974]: pgmap v593: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 138 KiB/s wr, 19 op/s 2024-03-26T06:41:44.780 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.780 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.780 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-26T06:41:44.780 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.781 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.781 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.781 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-26T06:41:44.781 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.781 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.781 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.782 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-26T06:41:44.782 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.782 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.782 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.782 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-26T06:41:44.782 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.783 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.783 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.783 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-26T06:41:44.783 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.783 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.783 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.784 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-26T06:41:44.784 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.784 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.784 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.784 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-26T06:41:44.784 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.785 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.785 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.785 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-26T06:41:44.785 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.785 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.786 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.786 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-26T06:41:44.786 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.786 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.786 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.787 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-26T06:41:44.787 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.787 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.787 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.787 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-26T06:41:44.787 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.788 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.788 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.788 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-26T06:41:44.788 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.788 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.788 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.789 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-26T06:41:44.789 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.789 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.789 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.789 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-26T06:41:44.789 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.790 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.790 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.790 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-26T06:41:44.790 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.790 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.790 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.791 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-26T06:41:44.791 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.791 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.791 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.791 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-26T06:41:44.791 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.792 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.792 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.792 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-26T06:41:44.792 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-26T06:41:44.792 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-26T06:41:44.792 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.792 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.793 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.793 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:44.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.793 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-26T06:41:44.793 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-03-26T06:41:44.793 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 1, 2024-03-26T06:41:44.793 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 0, 2024-03-26T06:41:44.793 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-03-26T06:41:44.794 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-26T06:41:44.794 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-03-26T06:41:44.794 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-03-26T06:41:44.794 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 2577, 2024-03-26T06:41:44.794 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-26T06:41:44.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.794 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-26T06:41:44.794 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-26T06:41:44.795 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-26T06:41:44.795 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T06:41:44.795 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-26T06:41:44.795 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T06:41:44.795 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-26T06:41:44.795 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T06:41:44.795 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T06:41:44.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T06:41:44.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T06:41:44.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T06:41:44.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-26T06:41:44.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T06:41:44.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T06:41:44.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-26T06:41:44.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-26T06:41:44.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-26T06:41:44.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-26T06:41:44.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T06:41:44.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T06:41:44.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T06:41:44.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T06:41:44.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T06:41:44.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-26T06:41:44.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-26T06:41:44.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-26T06:41:44.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T06:41:44.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T06:41:44.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-26T06:41:44.799 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-26T06:41:44.799 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-26T06:41:44.799 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-26T06:41:44.799 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T06:41:44.799 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-26T06:41:44.799 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T06:41:44.799 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-26T06:41:44.800 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 264011, 2024-03-26T06:41:44.800 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 725, 2024-03-26T06:41:44.800 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-26T06:41:44.800 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-26T06:41:44.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-26T06:41:44.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-26T06:41:44.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-26T06:41:44.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-26T06:41:44.801 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-26T06:41:44.801 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-26T06:41:44.801 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8032, 2024-03-26T06:41:44.801 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 167, 2024-03-26T06:41:44.801 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T06:41:44.801 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-26T06:41:44.801 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-26T06:41:44.801 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-26T06:41:44.802 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 15902240, 2024-03-26T06:41:44.802 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 96745, 2024-03-26T06:41:44.802 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-26T06:41:44.802 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-26T06:41:44.802 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-26T06:41:44.802 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-26T06:41:44.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.802 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-26T06:41:44.803 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-26T06:41:44.803 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-26T06:41:44.803 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 4520, 2024-03-26T06:41:44.803 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 21, 2024-03-26T06:41:44.803 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 3, 2024-03-26T06:41:44.803 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 4520, 2024-03-26T06:41:44.803 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-26T06:41:44.803 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4520, 2024-03-26T06:41:44.804 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 11.069775999, 2024-03-26T06:41:44.804 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002449065 2024-03-26T06:41:44.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.804 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-26T06:41:44.804 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-26T06:41:44.804 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4520, 2024-03-26T06:41:44.804 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 4521 2024-03-26T06:41:44.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.805 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 4520, 2024-03-26T06:41:44.805 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 4520, 2024-03-26T06:41:44.805 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2024-03-26T06:41:44.805 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-26T06:41:44.805 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-26T06:41:44.805 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 2531, 2024-03-26T06:41:44.805 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2024-03-26T06:41:44.806 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 1987, 2024-03-26T06:41:44.806 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2024-03-26T06:41:44.806 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2024-03-26T06:41:44.806 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-26T06:41:44.806 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-26T06:41:44.806 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-26T06:41:44.806 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-26T06:41:44.807 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2024-03-26T06:41:44.807 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-26T06:41:44.807 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-26T06:41:44.807 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-26T06:41:44.807 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 1, 2024-03-26T06:41:44.807 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2024-03-26T06:41:44.807 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-26T06:41:44.808 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-26T06:41:44.808 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-26T06:41:44.808 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-26T06:41:44.808 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-26T06:41:44.808 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-26T06:41:44.808 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T06:41:44.808 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-26T06:41:44.809 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-26T06:41:44.809 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-26T06:41:44.809 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-26T06:41:44.809 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-26T06:41:44.809 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-26T06:41:44.809 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-26T06:41:44.809 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-26T06:41:44.809 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-26T06:41:44.810 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-26T06:41:44.810 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-26T06:41:44.810 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-26T06:41:44.810 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-26T06:41:44.810 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-26T06:41:44.810 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-26T06:41:44.810 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-26T06:41:44.810 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-26T06:41:44.811 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-26T06:41:44.811 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-26T06:41:44.811 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 116, 2024-03-26T06:41:44.811 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-26T06:41:44.811 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 31, 2024-03-26T06:41:44.811 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 11, 2024-03-26T06:41:44.811 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 11, 2024-03-26T06:41:44.811 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-26T06:41:44.812 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-26T06:41:44.812 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2024-03-26T06:41:44.812 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2024-03-26T06:41:44.812 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2024-03-26T06:41:44.812 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.812 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-26T06:41:44.812 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2024-03-26T06:41:44.813 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-26T06:41:44.813 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-26T06:41:44.813 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-26T06:41:44.813 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.813 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-26T06:41:44.813 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-26T06:41:44.813 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.814 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-26T06:41:44.814 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-26T06:41:44.814 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T06:41:44.814 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-26T06:41:44.814 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-03-26T06:41:44.814 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-03-26T06:41:44.814 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-03-26T06:41:44.815 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-26T06:41:44.815 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.815 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T06:41:44.815 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:44.815 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-26T06:41:44.815 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:44.815 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 4959, 2024-03-26T06:41:44.815 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 388523912, 2024-03-26T06:41:44.816 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:44.816 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 4959, 2024-03-26T06:41:44.816 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:44.816 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:44.816 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4959, 2024-03-26T06:41:44.816 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 388523912, 2024-03-26T06:41:44.816 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:44.817 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.817 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.817 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.817 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:44.817 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.817 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-26T06:41:44.817 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:44.817 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-26T06:41:44.818 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:44.818 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:44.818 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:44.818 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:44.818 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:44.818 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 4520, 2024-03-26T06:41:44.818 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 385797653, 2024-03-26T06:41:44.819 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4520, 2024-03-26T06:41:44.819 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 385797653, 2024-03-26T06:41:44.819 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:44.819 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.819 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.819 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.819 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:44.820 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.820 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-26T06:41:44.820 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:44.820 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-26T06:41:44.820 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:44.820 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:44.820 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:44.820 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:44.821 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:44.821 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 4520, 2024-03-26T06:41:44.821 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 4520, 2024-03-26T06:41:44.821 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4520, 2024-03-26T06:41:44.821 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 4520, 2024-03-26T06:41:44.821 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:44.821 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.821 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.822 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.822 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:44.822 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.822 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-26T06:41:44.822 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:44.822 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-26T06:41:44.822 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:44.822 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:44.823 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:44.823 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:44.823 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:44.823 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:44.823 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:44.823 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-26T06:41:44.823 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-26T06:41:44.823 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:44.824 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.824 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.824 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.824 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:44.824 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-26T06:41:44.824 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55ead3c0e960": { 2024-03-26T06:41:44.824 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-26T06:41:44.824 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-26T06:41:44.825 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-26T06:41:44.825 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-26T06:41:44.825 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-26T06:41:44.825 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-26T06:41:44.825 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-26T06:41:44.825 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-26T06:41:44.825 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-26T06:41:44.825 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-26T06:41:44.826 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-26T06:41:44.826 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-26T06:41:44.826 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-26T06:41:44.826 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-26T06:41:44.826 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-26T06:41:44.826 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:44.826 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-26T06:41:44.827 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-26T06:41:44.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:44 smithi066 ceph-mon[36504]: pgmap v593: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 138 KiB/s wr, 19 op/s 2024-03-26T06:41:45.540 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-26T06:41:45.540 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-26T06:41:45.540 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-26T06:41:45.540 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-26T06:41:45.540 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T06:41:45.541 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-26T06:41:45.554 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-26T06:41:45.554 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:41:45.554 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T06:41:45.574 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-26T06:41:45.574 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T06:41:45.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:41:45.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.776+0000 7fb62ea4d640 1 -- 172.21.15.37:0/4132100509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb628105530 msgr2=0x7fb62810d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:45.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.776+0000 7fb62ea4d640 1 --2- 172.21.15.37:0/4132100509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb628105530 0x7fb62810d100 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb620009ca0 tx=0x7fb62002c0f0 comp rx=0 tx=0).stop 2024-03-26T06:41:45.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 -- 172.21.15.37:0/4132100509 shutdown_connections 2024-03-26T06:41:45.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 --2- 172.21.15.37:0/4132100509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb628105530 0x7fb62810d100 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:45.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 --2- 172.21.15.37:0/4132100509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb628100e10 0x7fb628104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:45.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 --2- 172.21.15.37:0/4132100509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb628100460 0x7fb628100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:45.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 -- 172.21.15.37:0/4132100509 >> 172.21.15.37:0/4132100509 conn(0x7fb6280772f0 msgr2=0x7fb628077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:45.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 -- 172.21.15.37:0/4132100509 shutdown_connections 2024-03-26T06:41:45.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 -- 172.21.15.37:0/4132100509 wait complete. 2024-03-26T06:41:45.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 Processor -- start 2024-03-26T06:41:45.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 -- start start 2024-03-26T06:41:45.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb628100460 0x7fb628198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:45.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb628105530 0x7fb628198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:45.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6281994e0 0x7fb6281a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.777+0000 7fb62ea4d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6281105d0 con 0x7fb628105530 2024-03-26T06:41:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb62ea4d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628110450 con 0x7fb628100460 2024-03-26T06:41:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb62ea4d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628110750 con 0x7fb6281994e0 2024-03-26T06:41:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb628105530 0x7fb628198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb628105530 0x7fb628198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33074/0 (socket says 172.21.15.37:33074) 2024-03-26T06:41:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6277fe640 1 -- 172.21.15.37:0/2482423547 learned_addr learned my addr 172.21.15.37:0/2482423547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb62cfea640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6281994e0 0x7fb6281a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb627fff640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb628100460 0x7fb628198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6277fe640 1 -- 172.21.15.37:0/2482423547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6281994e0 msgr2=0x7fb6281a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:45.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6277fe640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6281994e0 0x7fb6281a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:45.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6277fe640 1 -- 172.21.15.37:0/2482423547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb628100460 msgr2=0x7fb628198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:45.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6277fe640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb628100460 0x7fb628198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:45.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6277fe640 1 -- 172.21.15.37:0/2482423547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6281a4290 con 0x7fb628105530 2024-03-26T06:41:45.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb627fff640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb628100460 0x7fb628198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:41:45.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6277fe640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb628105530 0x7fb628198f50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005fd0 tx=0x7fb61c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:45.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6257fa640 1 -- 172.21.15.37:0/2482423547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c017070 con 0x7fb628105530 2024-03-26T06:41:45.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb62ea4d640 1 -- 172.21.15.37:0/2482423547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6281a4580 con 0x7fb628105530 2024-03-26T06:41:45.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6257fa640 1 -- 172.21.15.37:0/2482423547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb61c009c30 con 0x7fb628105530 2024-03-26T06:41:45.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb6257fa640 1 -- 172.21.15.37:0/2482423547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c010460 con 0x7fb628105530 2024-03-26T06:41:45.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.778+0000 7fb62ea4d640 1 -- 172.21.15.37:0/2482423547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6281a4ad0 con 0x7fb628105530 2024-03-26T06:41:45.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.780+0000 7fb62ea4d640 1 -- 172.21.15.37:0/2482423547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6280680e0 con 0x7fb628105530 2024-03-26T06:41:45.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.781+0000 7fb6257fa640 1 -- 172.21.15.37:0/2482423547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fb61c003e70 con 0x7fb628105530 2024-03-26T06:41:45.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.781+0000 7fb6257fa640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb6100a2970 0x7fb6100a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:45.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.782+0000 7fb627fff640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb6100a2970 0x7fb6100a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:45.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.782+0000 7fb6257fa640 1 -- 172.21.15.37:0/2482423547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb61c0cf270 con 0x7fb628105530 2024-03-26T06:41:45.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.782+0000 7fb627fff640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb6100a2970 0x7fb6100a4e30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb618009c20 tx=0x7fb618005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:45.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.785+0000 7fb6257fa640 1 -- 172.21.15.37:0/2482423547 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb61c020030 con 0x7fb628105530 2024-03-26T06:41:45.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.971+0000 7fb62ea4d640 1 -- 172.21.15.37:0/2482423547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6281087c0 con 0x7fb628105530 2024-03-26T06:41:45.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.973+0000 7fb6257fa640 1 -- 172.21.15.37:0/2482423547 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+23048 (secure 0 0 0) 0x7fb61c09bfd0 con 0x7fb628105530 2024-03-26T06:41:45.976 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:41:45.976 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":116,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:41:14.837860+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/641545212":"2024-03-27T06:22:58.745093+0000","172.21.15.37:6800/641545212":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/2221744469":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/656518427":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/374271722":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/3212150316":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:41:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 -- 172.21.15.37:0/2482423547 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb6100a2970 msgr2=0x7fb6100a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb6100a2970 0x7fb6100a4e30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb618009c20 tx=0x7fb618005f90 comp rx=0 tx=0).stop 2024-03-26T06:41:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 -- 172.21.15.37:0/2482423547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb628105530 msgr2=0x7fb628198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb628105530 0x7fb628198f50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005fd0 tx=0x7fb61c00dde0 comp rx=0 tx=0).stop 2024-03-26T06:41:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 -- 172.21.15.37:0/2482423547 shutdown_connections 2024-03-26T06:41:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fb6100a2970 0x7fb6100a4e30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6281994e0 0x7fb6281a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb628105530 0x7fb628198f50 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 --2- 172.21.15.37:0/2482423547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb628100460 0x7fb628198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 -- 172.21.15.37:0/2482423547 >> 172.21.15.37:0/2482423547 conn(0x7fb6280772f0 msgr2=0x7fb6280fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 -- 172.21.15.37:0/2482423547 shutdown_connections 2024-03-26T06:41:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:45.975+0000 7fb62ea4d640 1 -- 172.21.15.37:0/2482423547 wait complete. 2024-03-26T06:41:45.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T06:16:17.005303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T06:20:51.878592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T06:20:52.836733+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 115, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T06:41:45.992 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T06:41:46.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.097+0000 7f133ac0c640 1 -- 172.21.15.37:0/3209431542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334100d80 msgr2=0x7f1334104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.097+0000 7f133ac0c640 1 --2- 172.21.15.37:0/3209431542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334100d80 0x7f1334104dc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1320009900 tx=0x7f132002bc30 comp rx=0 tx=0).stop 2024-03-26T06:41:46.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.097+0000 7f133ac0c640 1 -- 172.21.15.37:0/3209431542 shutdown_connections 2024-03-26T06:41:46.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.097+0000 7f133ac0c640 1 --2- 172.21.15.37:0/3209431542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13341054a0 0x7f133410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.097+0000 7f133ac0c640 1 --2- 172.21.15.37:0/3209431542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334100d80 0x7f1334104dc0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.097+0000 7f133ac0c640 1 --2- 172.21.15.37:0/3209431542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13341003d0 0x7f13341007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.097+0000 7f133ac0c640 1 -- 172.21.15.37:0/3209431542 >> 172.21.15.37:0/3209431542 conn(0x7f13340772f0 msgr2=0x7f1334077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:46.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.097+0000 7f133ac0c640 1 -- 172.21.15.37:0/3209431542 shutdown_connections 2024-03-26T06:41:46.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.097+0000 7f133ac0c640 1 -- 172.21.15.37:0/3209431542 wait complete. 2024-03-26T06:41:46.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.097+0000 7f133ac0c640 1 Processor -- start 2024-03-26T06:41:46.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.097+0000 7f133ac0c640 1 -- start start 2024-03-26T06:41:46.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f133ac0c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13341003d0 0x7f1334101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f133ac0c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13341054a0 0x7f13341023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f133ac0c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13341028e0 0x7f13341a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f133ac0c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1334110530 con 0x7f13341028e0 2024-03-26T06:41:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f133ac0c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13341103b0 con 0x7f13341054a0 2024-03-26T06:41:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f133ac0c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13341106b0 con 0x7f13341003d0 2024-03-26T06:41:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f13391a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13341028e0 0x7f13341a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f13391a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13341028e0 0x7f13341a5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33084/0 (socket says 172.21.15.37:33084) 2024-03-26T06:41:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f13391a9640 1 -- 172.21.15.37:0/2242303154 learned_addr learned my addr 172.21.15.37:0/2242303154 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f13389a8640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13341003d0 0x7f1334101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f1333fff640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13341054a0 0x7f13341023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f13389a8640 1 -- 172.21.15.37:0/2242303154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13341054a0 msgr2=0x7f13341023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f13389a8640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13341054a0 0x7f13341023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f13389a8640 1 -- 172.21.15.37:0/2242303154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13341028e0 msgr2=0x7f13341a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f13389a8640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13341028e0 0x7f13341a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f13389a8640 1 -- 172.21.15.37:0/2242303154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13341a6450 con 0x7f13341003d0 2024-03-26T06:41:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f13391a9640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13341028e0 0x7f13341a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:41:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.098+0000 7f13389a8640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13341003d0 0x7f1334101e60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f132800e9f0 tx=0x7f132800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.099+0000 7f1331ffb640 1 -- 172.21.15.37:0/2242303154 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132800ccc0 con 0x7f13341003d0 2024-03-26T06:41:46.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.099+0000 7f133ac0c640 1 -- 172.21.15.37:0/2242303154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13341a6740 con 0x7f13341003d0 2024-03-26T06:41:46.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.099+0000 7f133ac0c640 1 -- 172.21.15.37:0/2242303154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13341a6cb0 con 0x7f13341003d0 2024-03-26T06:41:46.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.099+0000 7f1331ffb640 1 -- 172.21.15.37:0/2242303154 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1328010910 con 0x7f13341003d0 2024-03-26T06:41:46.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.099+0000 7f1331ffb640 1 -- 172.21.15.37:0/2242303154 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328018880 con 0x7f13341003d0 2024-03-26T06:41:46.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.100+0000 7f133ac0c640 1 -- 172.21.15.37:0/2242303154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13340680e0 con 0x7f13341003d0 2024-03-26T06:41:46.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.101+0000 7f1331ffb640 1 -- 172.21.15.37:0/2242303154 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f1328010ac0 con 0x7f13341003d0 2024-03-26T06:41:46.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.102+0000 7f1331ffb640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f13240a29a0 0x7f13240a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.102+0000 7f1333fff640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f13240a29a0 0x7f13240a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.102+0000 7f1331ffb640 1 -- 172.21.15.37:0/2242303154 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f13280cf3e0 con 0x7f13341003d0 2024-03-26T06:41:46.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.102+0000 7f1333fff640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f13240a29a0 0x7f13240a4e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1320009230 tx=0x7f1320005fd0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:46.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.105+0000 7f1331ffb640 1 -- 172.21.15.37:0/2242303154 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f132809ba90 con 0x7f13341003d0 2024-03-26T06:41:46.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.289+0000 7f133ac0c640 1 -- 172.21.15.37:0/2242303154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1334109700 con 0x7f13341003d0 2024-03-26T06:41:46.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.290+0000 7f1331ffb640 1 -- 172.21.15.37:0/2242303154 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f132809fb30 con 0x7f13341003d0 2024-03-26T06:41:46.293 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-26T06:41:46.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 -- 172.21.15.37:0/2242303154 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f13240a29a0 msgr2=0x7f13240a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f13240a29a0 0x7f13240a4e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1320009230 tx=0x7f1320005fd0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 -- 172.21.15.37:0/2242303154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13341003d0 msgr2=0x7f1334101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13341003d0 0x7f1334101e60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f132800e9f0 tx=0x7f132800eeb0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 -- 172.21.15.37:0/2242303154 shutdown_connections 2024-03-26T06:41:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f13240a29a0 0x7f13240a4e60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13341028e0 0x7f13341a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f13341054a0 0x7f13341023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 --2- 172.21.15.37:0/2242303154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13341003d0 0x7f1334101e60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 -- 172.21.15.37:0/2242303154 >> 172.21.15.37:0/2242303154 conn(0x7f13340772f0 msgr2=0x7f13340ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:46.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 -- 172.21.15.37:0/2242303154 shutdown_connections 2024-03-26T06:41:46.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.292+0000 7f133ac0c640 1 -- 172.21.15.37:0/2242303154 wait complete. 2024-03-26T06:41:46.306 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T06:41:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.413+0000 7f78f6a8e640 1 -- 172.21.15.37:0/2404329166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78f01003d0 msgr2=0x7f78f01007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.413+0000 7f78f6a8e640 1 --2- 172.21.15.37:0/2404329166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78f01003d0 0x7f78f01007b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009900 tx=0x7f78e402bc30 comp rx=0 tx=0).stop 2024-03-26T06:41:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.413+0000 7f78f6a8e640 1 -- 172.21.15.37:0/2404329166 shutdown_connections 2024-03-26T06:41:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.413+0000 7f78f6a8e640 1 --2- 172.21.15.37:0/2404329166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f01054a0 0x7f78f010d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.413+0000 7f78f6a8e640 1 --2- 172.21.15.37:0/2404329166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78f0100d80 0x7f78f0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.413+0000 7f78f6a8e640 1 --2- 172.21.15.37:0/2404329166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78f01003d0 0x7f78f01007b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.413+0000 7f78f6a8e640 1 -- 172.21.15.37:0/2404329166 >> 172.21.15.37:0/2404329166 conn(0x7f78f00772f0 msgr2=0x7f78f0077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.413+0000 7f78f6a8e640 1 -- 172.21.15.37:0/2404329166 shutdown_connections 2024-03-26T06:41:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.413+0000 7f78f6a8e640 1 -- 172.21.15.37:0/2404329166 wait complete. 2024-03-26T06:41:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f6a8e640 1 Processor -- start 2024-03-26T06:41:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f6a8e640 1 -- start start 2024-03-26T06:41:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f6a8e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78f01003d0 0x7f78f0101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f6a8e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f01054a0 0x7f78f01023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f6a8e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78f01028e0 0x7f78f01a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f6a8e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f0110530 con 0x7f78f01003d0 2024-03-26T06:41:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f6a8e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f01103b0 con 0x7f78f01028e0 2024-03-26T06:41:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f6a8e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f01106b0 con 0x7f78f01054a0 2024-03-26T06:41:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f482a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78f01003d0 0x7f78f0101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f482a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78f01003d0 0x7f78f0101e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33110/0 (socket says 172.21.15.37:33110) 2024-03-26T06:41:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f482a640 1 -- 172.21.15.37:0/1871595489 learned_addr learned my addr 172.21.15.37:0/1871595489 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78effff640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f01054a0 0x7f78f01023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.414+0000 7f78f502b640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78f01028e0 0x7f78f01a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78f482a640 1 -- 172.21.15.37:0/1871595489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f01054a0 msgr2=0x7f78f01023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78f482a640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f01054a0 0x7f78f01023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78f482a640 1 -- 172.21.15.37:0/1871595489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78f01028e0 msgr2=0x7f78f01a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78f482a640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78f01028e0 0x7f78f01a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78f482a640 1 -- 172.21.15.37:0/1871595489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f01a64f0 con 0x7f78f01003d0 2024-03-26T06:41:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78f502b640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78f01028e0 0x7f78f01a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:41:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78effff640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f01054a0 0x7f78f01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:41:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78f482a640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78f01003d0 0x7f78f0101e60 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f78e4006010 tx=0x7f78e402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78edffb640 1 -- 172.21.15.37:0/1871595489 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4041070 con 0x7f78f01003d0 2024-03-26T06:41:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78edffb640 1 -- 172.21.15.37:0/1871595489 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78e4004450 con 0x7f78f01003d0 2024-03-26T06:41:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78f6a8e640 1 -- 172.21.15.37:0/1871595489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f01a6780 con 0x7f78f01003d0 2024-03-26T06:41:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78edffb640 1 -- 172.21.15.37:0/1871595489 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4034440 con 0x7f78f01003d0 2024-03-26T06:41:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.415+0000 7f78f6a8e640 1 -- 172.21.15.37:0/1871595489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f01a6c70 con 0x7f78f01003d0 2024-03-26T06:41:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.416+0000 7f78f6a8e640 1 -- 172.21.15.37:0/1871595489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f00680e0 con 0x7f78f01003d0 2024-03-26T06:41:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.417+0000 7f78edffb640 1 -- 172.21.15.37:0/1871595489 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f78e4044070 con 0x7f78f01003d0 2024-03-26T06:41:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.418+0000 7f78edffb640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f78e00a29a0 0x7f78e00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.418+0000 7f78effff640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f78e00a29a0 0x7f78e00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.418+0000 7f78edffb640 1 -- 172.21.15.37:0/1871595489 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f78e40f0a30 con 0x7f78f01003d0 2024-03-26T06:41:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.418+0000 7f78effff640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f78e00a29a0 0x7f78e00a4e60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0058c0 tx=0x7f78dc005850 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:46.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.422+0000 7f78edffb640 1 -- 172.21.15.37:0/1871595489 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f78e40bd8a0 con 0x7f78f01003d0 2024-03-26T06:41:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.625+0000 7f78f6a8e640 1 -- 172.21.15.37:0/1871595489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78f0100d80 con 0x7f78f01003d0 2024-03-26T06:41:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.625+0000 7f78edffb640 1 -- 172.21.15.37:0/1871595489 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78e40bd1a0 con 0x7f78f01003d0 2024-03-26T06:41:46.628 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-26T06:41:46.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 -- 172.21.15.37:0/1871595489 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f78e00a29a0 msgr2=0x7f78e00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f78e00a29a0 0x7f78e00a4e60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0058c0 tx=0x7f78dc005850 comp rx=0 tx=0).stop 2024-03-26T06:41:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 -- 172.21.15.37:0/1871595489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78f01003d0 msgr2=0x7f78f0101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78f01003d0 0x7f78f0101e60 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f78e4006010 tx=0x7f78e402cc70 comp rx=0 tx=0).stop 2024-03-26T06:41:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 -- 172.21.15.37:0/1871595489 shutdown_connections 2024-03-26T06:41:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f78e00a29a0 0x7f78e00a4e60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78f01028e0 0x7f78f01a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f01054a0 0x7f78f01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 --2- 172.21.15.37:0/1871595489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78f01003d0 0x7f78f0101e60 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 -- 172.21.15.37:0/1871595489 >> 172.21.15.37:0/1871595489 conn(0x7f78f00772f0 msgr2=0x7f78f0104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 -- 172.21.15.37:0/1871595489 shutdown_connections 2024-03-26T06:41:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.627+0000 7f78f6a8e640 1 -- 172.21.15.37:0/1871595489 wait complete. 2024-03-26T06:41:46.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:46 smithi037 ceph-mon[31974]: pgmap v594: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 145 KiB/s wr, 20 op/s 2024-03-26T06:41:46.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:46 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2482423547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:41:46.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:46 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2242303154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:41:46.641 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T06:41:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.744+0000 7f4b7122b640 1 -- 172.21.15.37:0/2301127218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b6c1065c0 msgr2=0x7f4b6c10e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.744+0000 7f4b7122b640 1 --2- 172.21.15.37:0/2301127218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b6c1065c0 0x7f4b6c10e190 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009ca0 tx=0x7f4b6402c0f0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.744+0000 7f4b7122b640 1 -- 172.21.15.37:0/2301127218 shutdown_connections 2024-03-26T06:41:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.744+0000 7f4b7122b640 1 --2- 172.21.15.37:0/2301127218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b6c1065c0 0x7f4b6c10e190 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.744+0000 7f4b7122b640 1 --2- 172.21.15.37:0/2301127218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b6c101ea0 0x7f4b6c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.744+0000 7f4b7122b640 1 --2- 172.21.15.37:0/2301127218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c1014f0 0x7f4b6c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.744+0000 7f4b7122b640 1 -- 172.21.15.37:0/2301127218 >> 172.21.15.37:0/2301127218 conn(0x7f4b6c0fcd90 msgr2=0x7f4b6c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.744+0000 7f4b7122b640 1 -- 172.21.15.37:0/2301127218 shutdown_connections 2024-03-26T06:41:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.744+0000 7f4b7122b640 1 -- 172.21.15.37:0/2301127218 wait complete. 2024-03-26T06:41:46.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.745+0000 7f4b7122b640 1 Processor -- start 2024-03-26T06:41:46.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.745+0000 7f4b7122b640 1 -- start start 2024-03-26T06:41:46.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.745+0000 7f4b7122b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c1014f0 0x7f4b6c19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.745+0000 7f4b7122b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b6c1065c0 0x7f4b6c19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.745+0000 7f4b7122b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b6c197a00 0x7f4b6c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.745+0000 7f4b7122b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c111680 con 0x7f4b6c1065c0 2024-03-26T06:41:46.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.745+0000 7f4b7122b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c111500 con 0x7f4b6c1014f0 2024-03-26T06:41:46.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.745+0000 7f4b7122b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c111800 con 0x7f4b6c197a00 2024-03-26T06:41:46.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b6c1065c0 0x7f4b6c19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b6c1065c0 0x7f4b6c19dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33140/0 (socket says 172.21.15.37:33140) 2024-03-26T06:41:46.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b6a59c640 1 -- 172.21.15.37:0/1881844039 learned_addr learned my addr 172.21.15.37:0/1881844039 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:46.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b6b59e640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b6c197a00 0x7f4b6c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b6ad9d640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c1014f0 0x7f4b6c19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b6a59c640 1 -- 172.21.15.37:0/1881844039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b6c197a00 msgr2=0x7f4b6c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b6a59c640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b6c197a00 0x7f4b6c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b6a59c640 1 -- 172.21.15.37:0/1881844039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c1014f0 msgr2=0x7f4b6c19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b6a59c640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c1014f0 0x7f4b6c19d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b6a59c640 1 -- 172.21.15.37:0/1881844039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6c198520 con 0x7f4b6c1065c0 2024-03-26T06:41:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b6a59c640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b6c1065c0 0x7f4b6c19dd40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00e9f0 tx=0x7f4b5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b53fff640 1 -- 172.21.15.37:0/1881844039 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00ccc0 con 0x7f4b6c1065c0 2024-03-26T06:41:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b53fff640 1 -- 172.21.15.37:0/1881844039 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b5c005500 con 0x7f4b6c1065c0 2024-03-26T06:41:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b53fff640 1 -- 172.21.15.37:0/1881844039 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c010690 con 0x7f4b6c1065c0 2024-03-26T06:41:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b7122b640 1 -- 172.21.15.37:0/1881844039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c198810 con 0x7f4b6c1065c0 2024-03-26T06:41:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.746+0000 7f4b7122b640 1 -- 172.21.15.37:0/1881844039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c1a49c0 con 0x7f4b6c1065c0 2024-03-26T06:41:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.747+0000 7f4b7122b640 1 -- 172.21.15.37:0/1881844039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6c0680e0 con 0x7f4b6c1065c0 2024-03-26T06:41:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.749+0000 7f4b53fff640 1 -- 172.21.15.37:0/1881844039 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f4b5c0046c0 con 0x7f4b6c1065c0 2024-03-26T06:41:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.750+0000 7f4b53fff640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b540a2970 0x7f4b540a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.750+0000 7f4b53fff640 1 -- 172.21.15.37:0/1881844039 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4b5c0cf140 con 0x7f4b6c1065c0 2024-03-26T06:41:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.750+0000 7f4b6ad9d640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b540a2970 0x7f4b540a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.750+0000 7f4b6ad9d640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b540a2970 0x7f4b540a4e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0fd230 tx=0x7f4b60005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.754+0000 7f4b53fff640 1 -- 172.21.15.37:0/1881844039 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4b5c09bea0 con 0x7f4b6c1065c0 2024-03-26T06:41:46.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:46 smithi066 ceph-mon[36504]: pgmap v594: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 145 KiB/s wr, 20 op/s 2024-03-26T06:41:46.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:46 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2482423547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:41:46.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:46 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2242303154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:41:46.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:46 smithi149 ceph-mon[36681]: pgmap v594: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 145 KiB/s wr, 20 op/s 2024-03-26T06:41:46.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:46 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2482423547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:41:46.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:46 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2242303154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T06:41:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.938+0000 7f4b7122b640 1 -- 172.21.15.37:0/1881844039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b6c101ea0 con 0x7f4b6c1065c0 2024-03-26T06:41:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.938+0000 7f4b53fff640 1 -- 172.21.15.37:0/1881844039 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b5c09b7a0 con 0x7f4b6c1065c0 2024-03-26T06:41:46.941 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-03-26T06:41:46.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 -- 172.21.15.37:0/1881844039 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b540a2970 msgr2=0x7f4b540a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b540a2970 0x7f4b540a4e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0fd230 tx=0x7f4b60005f90 comp rx=0 tx=0).stop 2024-03-26T06:41:46.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 -- 172.21.15.37:0/1881844039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b6c1065c0 msgr2=0x7f4b6c19dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:46.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b6c1065c0 0x7f4b6c19dd40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00e9f0 tx=0x7f4b5c00eeb0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 -- 172.21.15.37:0/1881844039 shutdown_connections 2024-03-26T06:41:46.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f4b540a2970 0x7f4b540a4e30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b6c197a00 0x7f4b6c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b6c1065c0 0x7f4b6c19dd40 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 --2- 172.21.15.37:0/1881844039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b6c1014f0 0x7f4b6c19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:46.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 -- 172.21.15.37:0/1881844039 >> 172.21.15.37:0/1881844039 conn(0x7f4b6c0fcd90 msgr2=0x7f4b6c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:46.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 -- 172.21.15.37:0/1881844039 shutdown_connections 2024-03-26T06:41:46.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:46.940+0000 7f4b7122b640 1 -- 172.21.15.37:0/1881844039 wait complete. 2024-03-26T06:41:46.954 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T06:41:47.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.056+0000 7f3d918d1640 1 -- 172.21.15.37:0/4209666803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d8c100e30 msgr2=0x7f3d8c104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:47.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.056+0000 7f3d918d1640 1 --2- 172.21.15.37:0/4209666803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d8c100e30 0x7f3d8c104e70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009900 tx=0x7f3d8002bc30 comp rx=0 tx=0).stop 2024-03-26T06:41:47.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.056+0000 7f3d918d1640 1 -- 172.21.15.37:0/4209666803 shutdown_connections 2024-03-26T06:41:47.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.056+0000 7f3d918d1640 1 --2- 172.21.15.37:0/4209666803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d8c105550 0x7f3d8c10d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:47.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.056+0000 7f3d918d1640 1 --2- 172.21.15.37:0/4209666803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d8c100e30 0x7f3d8c104e70 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:47.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.056+0000 7f3d918d1640 1 --2- 172.21.15.37:0/4209666803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d8c100480 0x7f3d8c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:47.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.056+0000 7f3d918d1640 1 -- 172.21.15.37:0/4209666803 >> 172.21.15.37:0/4209666803 conn(0x7f3d8c077350 msgr2=0x7f3d8c077760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:47.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.056+0000 7f3d918d1640 1 -- 172.21.15.37:0/4209666803 shutdown_connections 2024-03-26T06:41:47.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.056+0000 7f3d918d1640 1 -- 172.21.15.37:0/4209666803 wait complete. 2024-03-26T06:41:47.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.056+0000 7f3d918d1640 1 Processor -- start 2024-03-26T06:41:47.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.056+0000 7f3d918d1640 1 -- start start 2024-03-26T06:41:47.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d918d1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d8c100480 0x7f3d8c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:47.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d918d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d8c105550 0x7f3d8c198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:47.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d918d1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d8c199510 0x7f3d8c1a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:47.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d918d1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c1105e0 con 0x7f3d8c105550 2024-03-26T06:41:47.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d918d1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c110460 con 0x7f3d8c100480 2024-03-26T06:41:47.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d918d1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c110760 con 0x7f3d8c199510 2024-03-26T06:41:47.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d8c105550 0x7f3d8c198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:47.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d8c105550 0x7f3d8c198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33150/0 (socket says 172.21.15.37:33150) 2024-03-26T06:41:47.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8a7fc640 1 -- 172.21.15.37:0/4229915492 learned_addr learned my addr 172.21.15.37:0/4229915492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:47.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8b7fe640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d8c199510 0x7f3d8c1a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:47.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8affd640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d8c100480 0x7f3d8c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:47.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8a7fc640 1 -- 172.21.15.37:0/4229915492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d8c199510 msgr2=0x7f3d8c1a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:47.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8a7fc640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d8c199510 0x7f3d8c1a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:47.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8a7fc640 1 -- 172.21.15.37:0/4229915492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d8c100480 msgr2=0x7f3d8c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:47.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8a7fc640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d8c100480 0x7f3d8c198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:47.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8a7fc640 1 -- 172.21.15.37:0/4229915492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c1a4290 con 0x7f3d8c105550 2024-03-26T06:41:47.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8b7fe640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d8c199510 0x7f3d8c1a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:41:47.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d8a7fc640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d8c105550 0x7f3d8c198f80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3d80036040 tx=0x7f3d8002cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:47.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d908cf640 1 -- 172.21.15.37:0/4229915492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80041070 con 0x7f3d8c105550 2024-03-26T06:41:47.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d918d1640 1 -- 172.21.15.37:0/4229915492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8c1a4520 con 0x7f3d8c105550 2024-03-26T06:41:47.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d908cf640 1 -- 172.21.15.37:0/4229915492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d800054e0 con 0x7f3d8c105550 2024-03-26T06:41:47.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d908cf640 1 -- 172.21.15.37:0/4229915492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80034440 con 0x7f3d8c105550 2024-03-26T06:41:47.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.057+0000 7f3d918d1640 1 -- 172.21.15.37:0/4229915492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8c1a4a10 con 0x7f3d8c105550 2024-03-26T06:41:47.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.058+0000 7f3d918d1640 1 -- 172.21.15.37:0/4229915492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8c068530 con 0x7f3d8c105550 2024-03-26T06:41:47.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.060+0000 7f3d908cf640 1 -- 172.21.15.37:0/4229915492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3d80044070 con 0x7f3d8c105550 2024-03-26T06:41:47.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.061+0000 7f3d908cf640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3d740a2970 0x7f3d740a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:47.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.061+0000 7f3d8affd640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3d740a2970 0x7f3d740a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:47.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.061+0000 7f3d908cf640 1 -- 172.21.15.37:0/4229915492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3d800f0fa0 con 0x7f3d8c105550 2024-03-26T06:41:47.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.061+0000 7f3d8affd640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3d740a2970 0x7f3d740a4e30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3d84005fd0 tx=0x7f3d84005850 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:47.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.064+0000 7f3d908cf640 1 -- 172.21.15.37:0/4229915492 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3d800bde10 con 0x7f3d8c105550 2024-03-26T06:41:47.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.259+0000 7f3d918d1640 1 -- 172.21.15.37:0/4229915492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3d8c108780 con 0x7f3d8c105550 2024-03-26T06:41:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.259+0000 7f3d908cf640 1 -- 172.21.15.37:0/4229915492 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v116) v1 ==== 81+0+938 (secure 0 0 0) 0x7f3d800bd710 con 0x7f3d8c105550 2024-03-26T06:41:47.263 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/641545212 2024-03-27T06:22:58.745093+0000 2024-03-26T06:41:47.263 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/641545212 2024-03-27T06:22:58.745093+0000 2024-03-26T06:41:47.263 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2221744469 2024-03-27T06:22:58.745093+0000 2024-03-26T06:41:47.263 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/656518427 2024-03-27T06:22:58.745093+0000 2024-03-26T06:41:47.263 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2549717127 2024-03-27T06:13:46.665797+0000 2024-03-26T06:41:47.264 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2065883347 2024-03-27T06:13:46.665797+0000 2024-03-26T06:41:47.264 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/401419890 2024-03-27T06:13:46.665797+0000 2024-03-26T06:41:47.264 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1548093856 2024-03-27T06:13:18.570869+0000 2024-03-26T06:41:47.264 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/401419890 2024-03-27T06:13:46.665797+0000 2024-03-26T06:41:47.264 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/718373544 2024-03-27T06:13:18.570869+0000 2024-03-26T06:41:47.264 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/374271722 2024-03-27T06:22:58.745093+0000 2024-03-26T06:41:47.264 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3212150316 2024-03-27T06:22:58.745093+0000 2024-03-26T06:41:47.264 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3042978467 2024-03-27T06:13:18.570869+0000 2024-03-26T06:41:47.264 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/718373544 2024-03-27T06:13:18.570869+0000 2024-03-26T06:41:47.265 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3391028472 2024-03-27T06:13:18.570869+0000 2024-03-26T06:41:47.265 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1382640415 2024-03-27T06:13:46.665797+0000 2024-03-26T06:41:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:listed 16 entries 2024-03-26T06:41:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.261+0000 7f3d918d1640 1 -- 172.21.15.37:0/4229915492 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3d740a2970 msgr2=0x7f3d740a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.261+0000 7f3d918d1640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3d740a2970 0x7f3d740a4e30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3d84005fd0 tx=0x7f3d84005850 comp rx=0 tx=0).stop 2024-03-26T06:41:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.261+0000 7f3d918d1640 1 -- 172.21.15.37:0/4229915492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d8c105550 msgr2=0x7f3d8c198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.261+0000 7f3d918d1640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d8c105550 0x7f3d8c198f80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3d80036040 tx=0x7f3d8002cbf0 comp rx=0 tx=0).stop 2024-03-26T06:41:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.261+0000 7f3d918d1640 1 -- 172.21.15.37:0/4229915492 shutdown_connections 2024-03-26T06:41:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.261+0000 7f3d918d1640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f3d740a2970 0x7f3d740a4e30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.261+0000 7f3d918d1640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d8c199510 0x7f3d8c1a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.261+0000 7f3d918d1640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d8c105550 0x7f3d8c198f80 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.261+0000 7f3d918d1640 1 --2- 172.21.15.37:0/4229915492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d8c100480 0x7f3d8c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.261+0000 7f3d918d1640 1 -- 172.21.15.37:0/4229915492 >> 172.21.15.37:0/4229915492 conn(0x7f3d8c077350 msgr2=0x7f3d8c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.262+0000 7f3d918d1640 1 -- 172.21.15.37:0/4229915492 shutdown_connections 2024-03-26T06:41:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:47.262+0000 7f3d918d1640 1 -- 172.21.15.37:0/4229915492 wait complete. 2024-03-26T06:41:47.275 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-26T06:41:47.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:41:47.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-26T06:41:47.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:47 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1871595489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:41:47.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:47 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1881844039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:41:47.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:47 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/4229915492' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T06:41:47.716 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-03-26T06:41:47.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:41:47.717 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-26T06:41:47.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:41:47.765 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-03-26T06:41:47.812 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-26T06:41:47.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:41:47.813 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T06:41:47.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:47 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1871595489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:41:47.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:47 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1881844039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:41:47.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:47 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/4229915492' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T06:41:47.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:47 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1871595489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T06:41:47.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:47 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1881844039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T06:41:47.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:47 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/4229915492' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T06:41:47.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T06:41:47.913 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-03-26T06:41:48.003 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-26T06:41:48.014 INFO:tasks.cephadm:Teardown begin 2024-03-26T06:41:48.014 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-26T06:41:48.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:41:48.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.153+0000 7f3633f76640 1 -- 172.21.15.37:0/2478370243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c102e80 msgr2=0x7f362c10b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.153+0000 7f3633f76640 1 --2- 172.21.15.37:0/2478370243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c102e80 0x7f362c10b360 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f361c009900 tx=0x7f361c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:41:48.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.153+0000 7f3633f76640 1 -- 172.21.15.37:0/2478370243 shutdown_connections 2024-03-26T06:41:48.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.153+0000 7f3633f76640 1 --2- 172.21.15.37:0/2478370243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c10b930 0x7f362c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.153+0000 7f3633f76640 1 --2- 172.21.15.37:0/2478370243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c102e80 0x7f362c10b360 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.153+0000 7f3633f76640 1 --2- 172.21.15.37:0/2478370243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c102560 0x7f362c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.153+0000 7f3633f76640 1 -- 172.21.15.37:0/2478370243 >> 172.21.15.37:0/2478370243 conn(0x7f362c0772f0 msgr2=0x7f362c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:48.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.153+0000 7f3633f76640 1 -- 172.21.15.37:0/2478370243 shutdown_connections 2024-03-26T06:41:48.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.153+0000 7f3633f76640 1 -- 172.21.15.37:0/2478370243 wait complete. 2024-03-26T06:41:48.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3633f76640 1 Processor -- start 2024-03-26T06:41:48.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3633f76640 1 -- start start 2024-03-26T06:41:48.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3633f76640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c102560 0x7f362c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3633f76640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c102e80 0x7f362c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3633f76640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c10b930 0x7f362c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3633f76640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1116f0 con 0x7f362c102560 2024-03-26T06:41:48.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3633f76640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c111570 con 0x7f362c102e80 2024-03-26T06:41:48.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3633f76640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c111870 con 0x7f362c10b930 2024-03-26T06:41:48.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3631d12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c102560 0x7f362c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3631d12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c102560 0x7f362c199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58936/0 (socket says 172.21.15.37:58936) 2024-03-26T06:41:48.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3631d12640 1 -- 172.21.15.37:0/3535870405 learned_addr learned my addr 172.21.15.37:0/3535870405 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:48.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3632513640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c10b930 0x7f362c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3631511640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c102e80 0x7f362c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3631d12640 1 -- 172.21.15.37:0/3535870405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c10b930 msgr2=0x7f362c19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3631d12640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c10b930 0x7f362c19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3631d12640 1 -- 172.21.15.37:0/3535870405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c102e80 msgr2=0x7f362c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3631d12640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c102e80 0x7f362c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3631d12640 1 -- 172.21.15.37:0/3535870405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c19ee20 con 0x7f362c102560 2024-03-26T06:41:48.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.154+0000 7f3631d12640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c102560 0x7f362c199e30 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f362800e9f0 tx=0x7f362800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:48.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.155+0000 7f3622ffd640 1 -- 172.21.15.37:0/3535870405 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362800cd80 con 0x7f362c102560 2024-03-26T06:41:48.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.155+0000 7f3622ffd640 1 -- 172.21.15.37:0/3535870405 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3628010910 con 0x7f362c102560 2024-03-26T06:41:48.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.155+0000 7f3633f76640 1 -- 172.21.15.37:0/3535870405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c19f110 con 0x7f362c102560 2024-03-26T06:41:48.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.155+0000 7f3622ffd640 1 -- 172.21.15.37:0/3535870405 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36280188f0 con 0x7f362c102560 2024-03-26T06:41:48.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.155+0000 7f3633f76640 1 -- 172.21.15.37:0/3535870405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c1a69a0 con 0x7f362c102560 2024-03-26T06:41:48.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.156+0000 7f3633f76640 1 -- 172.21.15.37:0/3535870405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c0680e0 con 0x7f362c102560 2024-03-26T06:41:48.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.157+0000 7f3622ffd640 1 -- 172.21.15.37:0/3535870405 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f3628010ac0 con 0x7f362c102560 2024-03-26T06:41:48.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.158+0000 7f3622ffd640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f36180a2970 0x7f36180a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.158+0000 7f3631511640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f36180a2970 0x7f36180a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.158+0000 7f3622ffd640 1 -- 172.21.15.37:0/3535870405 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f36280cf690 con 0x7f362c102560 2024-03-26T06:41:48.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.158+0000 7f3631511640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f36180a2970 0x7f36180a4e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f362c10b260 tx=0x7f361c000bc0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:48.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.162+0000 7f3622ffd640 1 -- 172.21.15.37:0/3535870405 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f362809c3f0 con 0x7f362c102560 2024-03-26T06:41:48.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.389+0000 7f3633f76640 1 -- 172.21.15.37:0/3535870405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f362c1a6b70 con 0x7f362c102560 2024-03-26T06:41:48.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.389+0000 7f3622ffd640 1 -- 172.21.15.37:0/3535870405 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+11391 (secure 0 0 0) 0x7f362809bcf0 con 0x7f362c102560 2024-03-26T06:41:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:41:48.393 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T06:20:53.631953+0000","modified":"2024-03-26T06:40:44.201367+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24494,"mds_2":24470,"mds_3":24469,"mds_4":14682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14670":{"gid":14670,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/1944002831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1944002831},{"type":"v1","addr":"172.21.15.37:6839","nonce":1944002831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14682":{"gid":14682,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6841/933565564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":933565564},{"type":"v1","addr":"172.21.15.37:6841","nonce":933565564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24394":{"gid":24394,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/1420321367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1420321367},{"type":"v1","addr":"172.21.15.66:6839","nonce":1420321367}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26155":{"gid":26155,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2880689237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2880689237},{"type":"v1","addr":"172.21.15.149:6837","nonce":2880689237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-26T06:41:48.393 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 70 2024-03-26T06:41:48.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.391+0000 7f3633f76640 1 -- 172.21.15.37:0/3535870405 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f36180a2970 msgr2=0x7f36180a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.391+0000 7f3633f76640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f36180a2970 0x7f36180a4e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f362c10b260 tx=0x7f361c000bc0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.391+0000 7f3633f76640 1 -- 172.21.15.37:0/3535870405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c102560 msgr2=0x7f362c199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.391+0000 7f3633f76640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c102560 0x7f362c199e30 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f362800e9f0 tx=0x7f362800eeb0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.392+0000 7f3633f76640 1 -- 172.21.15.37:0/3535870405 shutdown_connections 2024-03-26T06:41:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.392+0000 7f3633f76640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f36180a2970 0x7f36180a4e30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.392+0000 7f3633f76640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f362c10b930 0x7f362c19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.392+0000 7f3633f76640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f362c102e80 0x7f362c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.392+0000 7f3633f76640 1 --2- 172.21.15.37:0/3535870405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f362c102560 0x7f362c199e30 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.392+0000 7f3633f76640 1 -- 172.21.15.37:0/3535870405 >> 172.21.15.37:0/3535870405 conn(0x7f362c0772f0 msgr2=0x7f362c1077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.392+0000 7f3633f76640 1 -- 172.21.15.37:0/3535870405 shutdown_connections 2024-03-26T06:41:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.392+0000 7f3633f76640 1 -- 172.21.15.37:0/3535870405 wait complete. 2024-03-26T06:41:48.406 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T06:41:48.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.505+0000 7f5af5110640 1 -- 172.21.15.37:0/4042238906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af0106610 msgr2=0x7f5af010e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.505+0000 7f5af5110640 1 --2- 172.21.15.37:0/4042238906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af0106610 0x7f5af010e1e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009ca0 tx=0x7f5ae802c0f0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.505+0000 7f5af5110640 1 -- 172.21.15.37:0/4042238906 shutdown_connections 2024-03-26T06:41:48.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.505+0000 7f5af5110640 1 --2- 172.21.15.37:0/4042238906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af0106610 0x7f5af010e1e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.505+0000 7f5af5110640 1 --2- 172.21.15.37:0/4042238906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af006a5c0 0x7f5af0105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.505+0000 7f5af5110640 1 --2- 172.21.15.37:0/4042238906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0069c10 0x7f5af0069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.505+0000 7f5af5110640 1 -- 172.21.15.37:0/4042238906 >> 172.21.15.37:0/4042238906 conn(0x7f5af0077290 msgr2=0x7f5af00776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:48.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.505+0000 7f5af5110640 1 -- 172.21.15.37:0/4042238906 shutdown_connections 2024-03-26T06:41:48.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.505+0000 7f5af5110640 1 -- 172.21.15.37:0/4042238906 wait complete. 2024-03-26T06:41:48.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.506+0000 7f5af5110640 1 Processor -- start 2024-03-26T06:41:48.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.506+0000 7f5af5110640 1 -- start start 2024-03-26T06:41:48.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.506+0000 7f5af5110640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af0069c10 0x7f5af019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.506+0000 7f5af5110640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af0106610 0x7f5af019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.506+0000 7f5af5110640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0197980 0x7f5af0197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.506+0000 7f5af5110640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01116c0 con 0x7f5af0106610 2024-03-26T06:41:48.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.506+0000 7f5af5110640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af0111540 con 0x7f5af0069c10 2024-03-26T06:41:48.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.506+0000 7f5af5110640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af0111840 con 0x7f5af0197980 2024-03-26T06:41:48.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.506+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af0106610 0x7f5af019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.506+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af0106610 0x7f5af019dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58950/0 (socket says 172.21.15.37:58950) 2024-03-26T06:41:48.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.506+0000 7f5ae7fff640 1 -- 172.21.15.37:0/1483079413 learned_addr learned my addr 172.21.15.37:0/1483079413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:48.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5aeed9d640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af0069c10 0x7f5af019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5aef59e640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0197980 0x7f5af0197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5ae7fff640 1 -- 172.21.15.37:0/1483079413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0197980 msgr2=0x7f5af0197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5ae7fff640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0197980 0x7f5af0197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5ae7fff640 1 -- 172.21.15.37:0/1483079413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af0069c10 msgr2=0x7f5af019d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5ae7fff640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af0069c10 0x7f5af019d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5ae7fff640 1 -- 172.21.15.37:0/1483079413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af01984a0 con 0x7f5af0106610 2024-03-26T06:41:48.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5ae7fff640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af0106610 0x7f5af019dcc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800e9f0 tx=0x7f5ad800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:48.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5aecd99640 1 -- 172.21.15.37:0/1483079413 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800ccc0 con 0x7f5af0106610 2024-03-26T06:41:48.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5af5110640 1 -- 172.21.15.37:0/1483079413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af0198790 con 0x7f5af0106610 2024-03-26T06:41:48.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5aecd99640 1 -- 172.21.15.37:0/1483079413 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ad8010910 con 0x7f5af0106610 2024-03-26T06:41:48.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5aecd99640 1 -- 172.21.15.37:0/1483079413 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80188c0 con 0x7f5af0106610 2024-03-26T06:41:48.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.507+0000 7f5af5110640 1 -- 172.21.15.37:0/1483079413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af01a4990 con 0x7f5af0106610 2024-03-26T06:41:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.508+0000 7f5af5110640 1 -- 172.21.15.37:0/1483079413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af0102a10 con 0x7f5af0106610 2024-03-26T06:41:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.509+0000 7f5aecd99640 1 -- 172.21.15.37:0/1483079413 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f5ad8010ac0 con 0x7f5af0106610 2024-03-26T06:41:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.510+0000 7f5aecd99640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f5adc0a2970 0x7f5adc0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.510+0000 7f5aeed9d640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f5adc0a2970 0x7f5adc0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.510+0000 7f5aecd99640 1 -- 172.21.15.37:0/1483079413 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5ad80cf5c0 con 0x7f5af0106610 2024-03-26T06:41:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.510+0000 7f5aeed9d640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f5adc0a2970 0x7f5adc0a4e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5af0106280 tx=0x7f5ae000b410 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:48.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.513+0000 7f5aecd99640 1 -- 172.21.15.37:0/1483079413 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5ad809c2c0 con 0x7f5af0106610 2024-03-26T06:41:48.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:48 smithi037 ceph-mon[31974]: pgmap v595: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 144 KiB/s wr, 22 op/s 2024-03-26T06:41:48.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:48 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/3535870405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:41:48.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.741+0000 7f5af5110640 1 -- 172.21.15.37:0/1483079413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5af0068530 con 0x7f5af0106610 2024-03-26T06:41:48.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.742+0000 7f5aecd99640 1 -- 172.21.15.37:0/1483079413 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+11391 (secure 0 0 0) 0x7f5ad809bbc0 con 0x7f5af0106610 2024-03-26T06:41:48.745 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:41:48.746 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14586,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1987413446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1987413446},{"type":"v1","addr":"172.21.15.37:6835","nonce":1987413446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3479806072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3479806072},{"type":"v1","addr":"172.21.15.37:6837","nonce":3479806072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T06:20:53.631953+0000","modified":"2024-03-26T06:40:44.201367+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24545,"mds_1":24494,"mds_2":24470,"mds_3":24469,"mds_4":14682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14670":{"gid":14670,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/1944002831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1944002831},{"type":"v1","addr":"172.21.15.37:6839","nonce":1944002831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14682":{"gid":14682,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6841/933565564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":933565564},{"type":"v1","addr":"172.21.15.37:6841","nonce":933565564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24394":{"gid":24394,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2169208399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2169208399},{"type":"v1","addr":"172.21.15.149:6833","nonce":2169208399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/2811553067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":2811553067},{"type":"v1","addr":"172.21.15.66:6833","nonce":2811553067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1494796094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1494796094},{"type":"v1","addr":"172.21.15.149:6835","nonce":1494796094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.149:6839/1810727373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1810727373},{"type":"v1","addr":"172.21.15.149:6839","nonce":1810727373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.66:6835/4186424754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":4186424754},{"type":"v1","addr":"172.21.15.66:6835","nonce":4186424754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6837/3870845253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":3870845253},{"type":"v1","addr":"172.21.15.66:6837","nonce":3870845253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6839/1420321367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1420321367},{"type":"v1","addr":"172.21.15.66:6839","nonce":1420321367}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26155":{"gid":26155,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2880689237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2880689237},{"type":"v1","addr":"172.21.15.149:6837","nonce":2880689237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-26T06:41:48.746 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 70 2024-03-26T06:41:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 -- 172.21.15.37:0/1483079413 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f5adc0a2970 msgr2=0x7f5adc0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f5adc0a2970 0x7f5adc0a4e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5af0106280 tx=0x7f5ae000b410 comp rx=0 tx=0).stop 2024-03-26T06:41:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 -- 172.21.15.37:0/1483079413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af0106610 msgr2=0x7f5af019dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af0106610 0x7f5af019dcc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800e9f0 tx=0x7f5ad800eeb0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 -- 172.21.15.37:0/1483079413 shutdown_connections 2024-03-26T06:41:48.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f5adc0a2970 0x7f5adc0a4e30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0197980 0x7f5af0197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af0106610 0x7f5af019dcc0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 --2- 172.21.15.37:0/1483079413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af0069c10 0x7f5af019d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 -- 172.21.15.37:0/1483079413 >> 172.21.15.37:0/1483079413 conn(0x7f5af0077290 msgr2=0x7f5af00fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:48.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 -- 172.21.15.37:0/1483079413 shutdown_connections 2024-03-26T06:41:48.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.744+0000 7f5af5110640 1 -- 172.21.15.37:0/1483079413 wait complete. 2024-03-26T06:41:48.758 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T06:41:48.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:48 smithi066 ceph-mon[36504]: pgmap v595: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 144 KiB/s wr, 22 op/s 2024-03-26T06:41:48.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:48 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/3535870405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:41:48.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 -- 172.21.15.37:0/678272625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5800100dd0 msgr2=0x7f5800104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 --2- 172.21.15.37:0/678272625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5800100dd0 0x7f5800104e10 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009900 tx=0x7f57e802bc30 comp rx=0 tx=0).stop 2024-03-26T06:41:48.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 -- 172.21.15.37:0/678272625 shutdown_connections 2024-03-26T06:41:48.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 --2- 172.21.15.37:0/678272625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58001054f0 0x7f580010d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 --2- 172.21.15.37:0/678272625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5800100dd0 0x7f5800104e10 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 --2- 172.21.15.37:0/678272625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5800100420 0x7f5800100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 -- 172.21.15.37:0/678272625 >> 172.21.15.37:0/678272625 conn(0x7f5800077290 msgr2=0x7f58000776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:48.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 -- 172.21.15.37:0/678272625 shutdown_connections 2024-03-26T06:41:48.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 -- 172.21.15.37:0/678272625 wait complete. 2024-03-26T06:41:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 Processor -- start 2024-03-26T06:41:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 -- start start 2024-03-26T06:41:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5800100420 0x7f580019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.859+0000 7f58059f1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58001054f0 0x7f580019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f58059f1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5800197a40 0x7f5800197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f58059f1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5800111700 con 0x7f58001054f0 2024-03-26T06:41:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f58059f1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5800111580 con 0x7f5800100420 2024-03-26T06:41:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f58059f1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5800111880 con 0x7f5800197a40 2024-03-26T06:41:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f57f67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58001054f0 0x7f580019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f57f67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58001054f0 0x7f580019ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58962/0 (socket says 172.21.15.37:58962) 2024-03-26T06:41:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f57f67fc640 1 -- 172.21.15.37:0/1703261051 learned_addr learned my addr 172.21.15.37:0/1703261051 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f57feffd640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5800100420 0x7f580019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f57ff7fe640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5800197a40 0x7f5800197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f57f67fc640 1 -- 172.21.15.37:0/1703261051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5800197a40 msgr2=0x7f5800197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f57f67fc640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5800197a40 0x7f5800197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f57f67fc640 1 -- 172.21.15.37:0/1703261051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5800100420 msgr2=0x7f580019d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f57f67fc640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5800100420 0x7f580019d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f57f67fc640 1 -- 172.21.15.37:0/1703261051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5800198560 con 0x7f58001054f0 2024-03-26T06:41:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f57f67fc640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58001054f0 0x7f580019ddd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f57e8000c00 tx=0x7f57e802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f58049ef640 1 -- 172.21.15.37:0/1703261051 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8041070 con 0x7f58001054f0 2024-03-26T06:41:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f58049ef640 1 -- 172.21.15.37:0/1703261051 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57e8004450 con 0x7f58001054f0 2024-03-26T06:41:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f58049ef640 1 -- 172.21.15.37:0/1703261051 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8034440 con 0x7f58001054f0 2024-03-26T06:41:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f58059f1640 1 -- 172.21.15.37:0/1703261051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58001987f0 con 0x7f58001054f0 2024-03-26T06:41:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.860+0000 7f58059f1640 1 -- 172.21.15.37:0/1703261051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58001a4990 con 0x7f58001054f0 2024-03-26T06:41:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.861+0000 7f58059f1640 1 -- 172.21.15.37:0/1703261051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5800068530 con 0x7f58001054f0 2024-03-26T06:41:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.862+0000 7f58049ef640 1 -- 172.21.15.37:0/1703261051 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f57e8044070 con 0x7f58001054f0 2024-03-26T06:41:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.863+0000 7f58049ef640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f57ec0a2970 0x7f57ec0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.863+0000 7f57feffd640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f57ec0a2970 0x7f57ec0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.863+0000 7f58049ef640 1 -- 172.21.15.37:0/1703261051 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57e80f0b60 con 0x7f58001054f0 2024-03-26T06:41:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.863+0000 7f57feffd640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f57ec0a2970 0x7f57ec0a4e30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5800198b80 tx=0x7f57f0008040 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:48.867+0000 7f58049ef640 1 -- 172.21.15.37:0/1703261051 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f57e80bd970 con 0x7f58001054f0 2024-03-26T06:41:48.911 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:48 smithi149 ceph-mon[36681]: pgmap v595: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 144 KiB/s wr, 22 op/s 2024-03-26T06:41:48.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:48 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/3535870405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:41:49.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.052+0000 7f58059f1640 1 -- 172.21.15.37:0/1703261051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5800100dd0 con 0x7f58001054f0 2024-03-26T06:41:49.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.053+0000 7f58049ef640 1 -- 172.21.15.37:0/1703261051 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+23048 (secure 0 0 0) 0x7f57e80bd270 con 0x7f58001054f0 2024-03-26T06:41:49.057 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-26T06:41:49.057 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":116,"fsid":"a86fb218-eb37-11ee-95d0-87774f69a715","created":"2024-03-26T06:12:42.888145+0000","modified":"2024-03-26T06:41:14.837860+0000","last_up_change":"2024-03-26T06:20:03.048107+0000","last_in_change":"2024-03-26T06:19:43.687585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T06:16:17.005303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T06:20:51.878592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T06:20:52.836733+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"16004b33-cf5b-43d7-8c6d-04223f061c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6803","nonce":362526230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6805","nonce":362526230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6809","nonce":362526230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":362526230},{"type":"v1","addr":"172.21.15.37:6807","nonce":362526230}]},"public_addr":"172.21.15.37:6803/362526230","cluster_addr":"172.21.15.37:6805/362526230","heartbeat_back_addr":"172.21.15.37:6809/362526230","heartbeat_front_addr":"172.21.15.37:6807/362526230","state":["exists","up"]},{"osd":1,"uuid":"802bebb3-7af9-45af-bd5e-b5b7e1156403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6801","nonce":331563667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6803","nonce":331563667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6807","nonce":331563667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":331563667},{"type":"v1","addr":"172.21.15.66:6805","nonce":331563667}]},"public_addr":"172.21.15.66:6801/331563667","cluster_addr":"172.21.15.66:6803/331563667","heartbeat_back_addr":"172.21.15.66:6807/331563667","heartbeat_front_addr":"172.21.15.66:6805/331563667","state":["exists","up"]},{"osd":2,"uuid":"47b4afc5-f7fb-4e4c-91a2-c059b3d63cc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6801","nonce":611823377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6803","nonce":611823377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6807","nonce":611823377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":611823377},{"type":"v1","addr":"172.21.15.149:6805","nonce":611823377}]},"public_addr":"172.21.15.149:6801/611823377","cluster_addr":"172.21.15.149:6803/611823377","heartbeat_back_addr":"172.21.15.149:6807/611823377","heartbeat_front_addr":"172.21.15.149:6805/611823377","state":["exists","up"]},{"osd":3,"uuid":"68dcbfb5-853c-4843-b19d-5018e9c6e268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6811","nonce":2227972060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6813","nonce":2227972060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6817","nonce":2227972060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2227972060},{"type":"v1","addr":"172.21.15.37:6815","nonce":2227972060}]},"public_addr":"172.21.15.37:6811/2227972060","cluster_addr":"172.21.15.37:6813/2227972060","heartbeat_back_addr":"172.21.15.37:6817/2227972060","heartbeat_front_addr":"172.21.15.37:6815/2227972060","state":["exists","up"]},{"osd":4,"uuid":"4ce571a6-aa1d-40e6-a6d3-df9122391314","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6809","nonce":1994195514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6811","nonce":1994195514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6815","nonce":1994195514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1994195514},{"type":"v1","addr":"172.21.15.66:6813","nonce":1994195514}]},"public_addr":"172.21.15.66:6809/1994195514","cluster_addr":"172.21.15.66:6811/1994195514","heartbeat_back_addr":"172.21.15.66:6815/1994195514","heartbeat_front_addr":"172.21.15.66:6813/1994195514","state":["exists","up"]},{"osd":5,"uuid":"c2ade669-ee83-41ad-957b-ea5b88768477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6809","nonce":3463585638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6811","nonce":3463585638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6815","nonce":3463585638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3463585638},{"type":"v1","addr":"172.21.15.149:6813","nonce":3463585638}]},"public_addr":"172.21.15.149:6809/3463585638","cluster_addr":"172.21.15.149:6811/3463585638","heartbeat_back_addr":"172.21.15.149:6815/3463585638","heartbeat_front_addr":"172.21.15.149:6813/3463585638","state":["exists","up"]},{"osd":6,"uuid":"5dd601fe-399f-4ba5-87ab-d602a0930403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6819","nonce":1971653917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6821","nonce":1971653917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6825","nonce":1971653917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1971653917},{"type":"v1","addr":"172.21.15.37:6823","nonce":1971653917}]},"public_addr":"172.21.15.37:6819/1971653917","cluster_addr":"172.21.15.37:6821/1971653917","heartbeat_back_addr":"172.21.15.37:6825/1971653917","heartbeat_front_addr":"172.21.15.37:6823/1971653917","state":["exists","up"]},{"osd":7,"uuid":"f84e7511-f0e6-4cd0-b78f-cee1696df1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6817","nonce":1024919875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6819","nonce":1024919875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6823","nonce":1024919875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1024919875},{"type":"v1","addr":"172.21.15.66:6821","nonce":1024919875}]},"public_addr":"172.21.15.66:6817/1024919875","cluster_addr":"172.21.15.66:6819/1024919875","heartbeat_back_addr":"172.21.15.66:6823/1024919875","heartbeat_front_addr":"172.21.15.66:6821/1024919875","state":["exists","up"]},{"osd":8,"uuid":"388ebc46-fd6a-48e1-a245-a8fe4c8d937e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6817","nonce":192122283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6819","nonce":192122283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6823","nonce":192122283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":192122283},{"type":"v1","addr":"172.21.15.149:6821","nonce":192122283}]},"public_addr":"172.21.15.149:6817/192122283","cluster_addr":"172.21.15.149:6819/192122283","heartbeat_back_addr":"172.21.15.149:6823/192122283","heartbeat_front_addr":"172.21.15.149:6821/192122283","state":["exists","up"]},{"osd":9,"uuid":"c030ee66-7e52-4cf8-a8c9-3987bd5e7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6827","nonce":2324500917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6829","nonce":2324500917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6833","nonce":2324500917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2324500917},{"type":"v1","addr":"172.21.15.37:6831","nonce":2324500917}]},"public_addr":"172.21.15.37:6827/2324500917","cluster_addr":"172.21.15.37:6829/2324500917","heartbeat_back_addr":"172.21.15.37:6833/2324500917","heartbeat_front_addr":"172.21.15.37:6831/2324500917","state":["exists","up"]},{"osd":10,"uuid":"e7ea7170-f2ed-4d72-afee-7b2828f3a774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6825","nonce":364342155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6827","nonce":364342155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6831","nonce":364342155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":364342155},{"type":"v1","addr":"172.21.15.66:6829","nonce":364342155}]},"public_addr":"172.21.15.66:6825/364342155","cluster_addr":"172.21.15.66:6827/364342155","heartbeat_back_addr":"172.21.15.66:6831/364342155","heartbeat_front_addr":"172.21.15.66:6829/364342155","state":["exists","up"]},{"osd":11,"uuid":"780b3500-9d7a-48c1-8e39-ea0abb98dd5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6825","nonce":2983393449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6827","nonce":2983393449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6831","nonce":2983393449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2983393449},{"type":"v1","addr":"172.21.15.149:6829","nonce":2983393449}]},"public_addr":"172.21.15.149:6825/2983393449","cluster_addr":"172.21.15.149:6827/2983393449","heartbeat_back_addr":"172.21.15.149:6831/2983393449","heartbeat_front_addr":"172.21.15.149:6829/2983393449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:26.432564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:15:50.729270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:14.134236+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:16:39.448961+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:04.310668+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:28.800975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:17:54.635964+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:19.756183+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:18:44.032865+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:10.474845+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:19:36.290082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T06:20:01.526860+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/641545212":"2024-03-27T06:22:58.745093+0000","172.21.15.37:6800/641545212":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/2221744469":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/656518427":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/2549717127":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/2065883347":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:0/1548093856":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/401419890":"2024-03-27T06:13:46.665797+0000","172.21.15.37:6800/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/374271722":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/3212150316":"2024-03-27T06:22:58.745093+0000","172.21.15.37:0/3042978467":"2024-03-27T06:13:18.570869+0000","172.21.15.37:6801/718373544":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/3391028472":"2024-03-27T06:13:18.570869+0000","172.21.15.37:0/1382640415":"2024-03-27T06:13:46.665797+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T06:41:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 -- 172.21.15.37:0/1703261051 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f57ec0a2970 msgr2=0x7f57ec0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f57ec0a2970 0x7f57ec0a4e30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5800198b80 tx=0x7f57f0008040 comp rx=0 tx=0).stop 2024-03-26T06:41:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 -- 172.21.15.37:0/1703261051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58001054f0 msgr2=0x7f580019ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58001054f0 0x7f580019ddd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f57e8000c00 tx=0x7f57e802cc70 comp rx=0 tx=0).stop 2024-03-26T06:41:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 -- 172.21.15.37:0/1703261051 shutdown_connections 2024-03-26T06:41:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f57ec0a2970 0x7f57ec0a4e30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5800197a40 0x7f5800197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58001054f0 0x7f580019ddd0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 --2- 172.21.15.37:0/1703261051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5800100420 0x7f580019d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 -- 172.21.15.37:0/1703261051 >> 172.21.15.37:0/1703261051 conn(0x7f5800077290 msgr2=0x7f58000ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 -- 172.21.15.37:0/1703261051 shutdown_connections 2024-03-26T06:41:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.055+0000 7f58059f1640 1 -- 172.21.15.37:0/1703261051 wait complete. 2024-03-26T06:41:49.070 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-26T06:41:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.171+0000 7f37bfa8e640 1 -- 172.21.15.37:0/568051798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b8100d80 msgr2=0x7f37b8104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.171+0000 7f37bfa8e640 1 --2- 172.21.15.37:0/568051798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b8100d80 0x7f37b8104dc0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009900 tx=0x7f37a802bc30 comp rx=0 tx=0).stop 2024-03-26T06:41:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.171+0000 7f37bfa8e640 1 -- 172.21.15.37:0/568051798 shutdown_connections 2024-03-26T06:41:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.171+0000 7f37bfa8e640 1 --2- 172.21.15.37:0/568051798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37b81054a0 0x7f37b810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.171+0000 7f37bfa8e640 1 --2- 172.21.15.37:0/568051798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b8100d80 0x7f37b8104dc0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.171+0000 7f37bfa8e640 1 --2- 172.21.15.37:0/568051798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37b81003d0 0x7f37b81007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.171+0000 7f37bfa8e640 1 -- 172.21.15.37:0/568051798 >> 172.21.15.37:0/568051798 conn(0x7f37b80772f0 msgr2=0x7f37b8077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:49.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.171+0000 7f37bfa8e640 1 -- 172.21.15.37:0/568051798 shutdown_connections 2024-03-26T06:41:49.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.171+0000 7f37bfa8e640 1 -- 172.21.15.37:0/568051798 wait complete. 2024-03-26T06:41:49.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bfa8e640 1 Processor -- start 2024-03-26T06:41:49.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bfa8e640 1 -- start start 2024-03-26T06:41:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bfa8e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37b81003d0 0x7f37b8101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bfa8e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b81054a0 0x7f37b81023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bfa8e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37b81028e0 0x7f37b81a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bfa8e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b8110530 con 0x7f37b81054a0 2024-03-26T06:41:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bfa8e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b81103b0 con 0x7f37b81003d0 2024-03-26T06:41:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bfa8e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b81106b0 con 0x7f37b81028e0 2024-03-26T06:41:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bd029640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b81054a0 0x7f37b81023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bd029640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b81054a0 0x7f37b81023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58982/0 (socket says 172.21.15.37:58982) 2024-03-26T06:41:49.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bd029640 1 -- 172.21.15.37:0/682096642 learned_addr learned my addr 172.21.15.37:0/682096642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:49.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37be02b640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37b81028e0 0x7f37b81a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:49.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bd82a640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37b81003d0 0x7f37b8101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:49.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bd029640 1 -- 172.21.15.37:0/682096642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37b81028e0 msgr2=0x7f37b81a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:49.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bd029640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37b81028e0 0x7f37b81a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bd029640 1 -- 172.21.15.37:0/682096642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37b81003d0 msgr2=0x7f37b8101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:49.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bd029640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37b81003d0 0x7f37b8101e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bd029640 1 -- 172.21.15.37:0/682096642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b81a6470 con 0x7f37b81054a0 2024-03-26T06:41:49.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.172+0000 7f37bd82a640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37b81003d0 0x7f37b8101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:41:49.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.173+0000 7f37bd029640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b81054a0 0x7f37b81023a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f37a8036040 tx=0x7f37a802cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:49.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.173+0000 7f37aeffd640 1 -- 172.21.15.37:0/682096642 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8041070 con 0x7f37b81054a0 2024-03-26T06:41:49.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.173+0000 7f37bfa8e640 1 -- 172.21.15.37:0/682096642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b81a6700 con 0x7f37b81054a0 2024-03-26T06:41:49.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.173+0000 7f37aeffd640 1 -- 172.21.15.37:0/682096642 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37a80054e0 con 0x7f37b81054a0 2024-03-26T06:41:49.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.173+0000 7f37aeffd640 1 -- 172.21.15.37:0/682096642 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8034440 con 0x7f37b81054a0 2024-03-26T06:41:49.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.173+0000 7f37bfa8e640 1 -- 172.21.15.37:0/682096642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b81a6c40 con 0x7f37b81054a0 2024-03-26T06:41:49.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.174+0000 7f37bfa8e640 1 -- 172.21.15.37:0/682096642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b80680e0 con 0x7f37b81054a0 2024-03-26T06:41:49.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.175+0000 7f37aeffd640 1 -- 172.21.15.37:0/682096642 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f37a8044070 con 0x7f37b81054a0 2024-03-26T06:41:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.175+0000 7f37aeffd640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f37a40a29a0 0x7f37a40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.175+0000 7f37bd82a640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f37a40a29a0 0x7f37a40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.175+0000 7f37aeffd640 1 -- 172.21.15.37:0/682096642 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f37a80f0ae0 con 0x7f37b81054a0 2024-03-26T06:41:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.176+0000 7f37bd82a640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f37a40a29a0 0x7f37a40a4e60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f37b40058c0 tx=0x7f37b4005d30 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.179+0000 7f37aeffd640 1 -- 172.21.15.37:0/682096642 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f37a80bd950 con 0x7f37b81054a0 2024-03-26T06:41:49.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.404+0000 7f37bfa8e640 1 -- 172.21.15.37:0/682096642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f37b8109700 con 0x7f37b81054a0 2024-03-26T06:41:49.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.407+0000 7f37aeffd640 1 -- 172.21.15.37:0/682096642 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) v1 ==== 161+0+0 (secure 0 0 0) 0x7f37a80bd250 con 0x7f37b81054a0 2024-03-26T06:41:49.410 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-26T06:41:49.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.410+0000 7f37bfa8e640 1 -- 172.21.15.37:0/682096642 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f37a40a29a0 msgr2=0x7f37a40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:49.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.410+0000 7f37bfa8e640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f37a40a29a0 0x7f37a40a4e60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f37b40058c0 tx=0x7f37b4005d30 comp rx=0 tx=0).stop 2024-03-26T06:41:49.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.410+0000 7f37bfa8e640 1 -- 172.21.15.37:0/682096642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b81054a0 msgr2=0x7f37b81023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:49.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.410+0000 7f37bfa8e640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b81054a0 0x7f37b81023a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f37a8036040 tx=0x7f37a802cbf0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.411+0000 7f37bfa8e640 1 -- 172.21.15.37:0/682096642 shutdown_connections 2024-03-26T06:41:49.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.411+0000 7f37bfa8e640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f37a40a29a0 0x7f37a40a4e60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.411+0000 7f37bfa8e640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37b81028e0 0x7f37b81a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.411+0000 7f37bfa8e640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b81054a0 0x7f37b81023a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.411+0000 7f37bfa8e640 1 --2- 172.21.15.37:0/682096642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37b81003d0 0x7f37b8101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.411+0000 7f37bfa8e640 1 -- 172.21.15.37:0/682096642 >> 172.21.15.37:0/682096642 conn(0x7f37b80772f0 msgr2=0x7f37b80687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:49.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.411+0000 7f37bfa8e640 1 -- 172.21.15.37:0/682096642 shutdown_connections 2024-03-26T06:41:49.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.411+0000 7f37bfa8e640 1 -- 172.21.15.37:0/682096642 wait complete. 2024-03-26T06:41:49.426 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-26T06:41:49.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.533+0000 7f5179de6640 1 -- 172.21.15.37:0/2832861408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51741016c0 msgr2=0x7f517410d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:49.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.533+0000 7f5179de6640 1 --2- 172.21.15.37:0/2832861408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51741016c0 0x7f517410d070 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f516c009cd0 tx=0x7f516c02c0f0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.533+0000 7f5179de6640 1 -- 172.21.15.37:0/2832861408 shutdown_connections 2024-03-26T06:41:49.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.533+0000 7f5179de6640 1 --2- 172.21.15.37:0/2832861408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51741016c0 0x7f517410d070 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.533+0000 7f5179de6640 1 --2- 172.21.15.37:0/2832861408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5174100d00 0x7f5174101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.533+0000 7f5179de6640 1 --2- 172.21.15.37:0/2832861408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5174105190 0x7f5174105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.533+0000 7f5179de6640 1 -- 172.21.15.37:0/2832861408 >> 172.21.15.37:0/2832861408 conn(0x7f5174075450 msgr2=0x7f5174075860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:49.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.533+0000 7f5179de6640 1 -- 172.21.15.37:0/2832861408 shutdown_connections 2024-03-26T06:41:49.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.533+0000 7f5179de6640 1 -- 172.21.15.37:0/2832861408 wait complete. 2024-03-26T06:41:49.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5179de6640 1 Processor -- start 2024-03-26T06:41:49.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5179de6640 1 -- start start 2024-03-26T06:41:49.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5179de6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5174100d00 0x7f517419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5179de6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51741016c0 0x7f517419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5179de6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5174105190 0x7f51741979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5179de6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51741105c0 con 0x7f5174105190 2024-03-26T06:41:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5179de6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5174110440 con 0x7f5174100d00 2024-03-26T06:41:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5179de6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5174110740 con 0x7f51741016c0 2024-03-26T06:41:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5173fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5174105190 0x7f51741979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5173fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5174105190 0x7f51741979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59030/0 (socket says 172.21.15.37:59030) 2024-03-26T06:41:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5173fff640 1 -- 172.21.15.37:0/2989476656 learned_addr learned my addr 172.21.15.37:0/2989476656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:49.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f51737fe640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5174100d00 0x7f517419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:49.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5172ffd640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51741016c0 0x7f517419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:49.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5173fff640 1 -- 172.21.15.37:0/2989476656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51741016c0 msgr2=0x7f517419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:49.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5173fff640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51741016c0 0x7f517419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5173fff640 1 -- 172.21.15.37:0/2989476656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5174100d00 msgr2=0x7f517419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:49.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5173fff640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5174100d00 0x7f517419d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:49.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5173fff640 1 -- 172.21.15.37:0/2989476656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174198120 con 0x7f5174105190 2024-03-26T06:41:49.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f5172ffd640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51741016c0 0x7f517419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:41:49.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.534+0000 7f51737fe640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5174100d00 0x7f517419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:41:49.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.535+0000 7f5173fff640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5174105190 0x7f51741979a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f516c009ca0 tx=0x7f516c032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:49.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.535+0000 7f5170ff9640 1 -- 172.21.15.37:0/2989476656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c042020 con 0x7f5174105190 2024-03-26T06:41:49.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.535+0000 7f5170ff9640 1 -- 172.21.15.37:0/2989476656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f516c034640 con 0x7f5174105190 2024-03-26T06:41:49.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.535+0000 7f5179de6640 1 -- 172.21.15.37:0/2989476656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51741983b0 con 0x7f5174105190 2024-03-26T06:41:49.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.535+0000 7f5170ff9640 1 -- 172.21.15.37:0/2989476656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c039040 con 0x7f5174105190 2024-03-26T06:41:49.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.535+0000 7f5179de6640 1 -- 172.21.15.37:0/2989476656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51741a46b0 con 0x7f5174105190 2024-03-26T06:41:49.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.536+0000 7f5179de6640 1 -- 172.21.15.37:0/2989476656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5174068530 con 0x7f5174105190 2024-03-26T06:41:49.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.537+0000 7f5170ff9640 1 -- 172.21.15.37:0/2989476656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f516c045020 con 0x7f5174105190 2024-03-26T06:41:49.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.537+0000 7f5170ff9640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f515c0a2970 0x7f515c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:49.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.538+0000 7f51737fe640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f515c0a2970 0x7f515c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:49.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.538+0000 7f5170ff9640 1 -- 172.21.15.37:0/2989476656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8899+0+0 (secure 0 0 0) 0x7f516c0f0f40 con 0x7f5174105190 2024-03-26T06:41:49.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.538+0000 7f51737fe640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f515c0a2970 0x7f515c0a4e30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5174198ae0 tx=0x7f5164005f90 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:49.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.542+0000 7f5170ff9640 1 -- 172.21.15.37:0/2989476656 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f516c04c020 con 0x7f5174105190 2024-03-26T06:41:49.700 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a[74242]: ignoring --setuser ceph since I am not root 2024-03-26T06:41:49.701 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a[74242]: ignoring --setgroup ceph since I am not root 2024-03-26T06:41:49.701 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-mds[74246]: -- 172.21.15.37:0/3527030005 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f9d6589a40 con 0x55f9d655d000 2024-03-26T06:41:49.701 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a[74242]: starting mds.a at 2024-03-26T06:41:49.701 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g[73877]: ignoring --setuser ceph since I am not root 2024-03-26T06:41:49.702 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g[73877]: ignoring --setgroup ceph since I am not root 2024-03-26T06:41:49.702 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-mds[73881]: -- 172.21.15.37:0/1479821121 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556d4c0c3a40 con 0x556d4c096800 2024-03-26T06:41:49.702 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g[73877]: starting mds.g at 2024-03-26T06:41:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a[31970]: 2024-03-26T06:41:49.405+0000 7f0b1b43a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T06:41:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1483079413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:41:49.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1703261051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:41:49.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/682096642' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T06:41:49.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-mon[31974]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T06:41:49.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-mon[31974]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T06:41:49.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/682096642' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T06:41:49.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-mon[31974]: osdmap e117: 12 total, 12 up, 12 in 2024-03-26T06:41:49.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:49 smithi037 ceph-mon[31974]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T06:41:49.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:49.766+0000 7f5179de6640 1 -- 172.21.15.37:0/2989476656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f517402ce70 con 0x7f5174105190 2024-03-26T06:41:49.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1483079413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:41:49.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1703261051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:41:49.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/682096642' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T06:41:49.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mon[36504]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T06:41:49.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mon[36504]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T06:41:49.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/682096642' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T06:41:49.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mon[36504]: osdmap e117: 12 total, 12 up, 12 in 2024-03-26T06:41:49.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mon[36504]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T06:41:49.851 INFO:journalctl@ceph.mds.e.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-e[66664]: ignoring --setuser ceph since I am not root 2024-03-26T06:41:49.851 INFO:journalctl@ceph.mds.e.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-e[66664]: ignoring --setgroup ceph since I am not root 2024-03-26T06:41:49.851 INFO:journalctl@ceph.mds.e.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mds[66668]: -- 172.21.15.66:0/1081050298 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bbdeb91a40 con 0x55bbdeb64800 2024-03-26T06:41:49.851 INFO:journalctl@ceph.mds.e.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-e[66664]: starting mds.e at 2024-03-26T06:41:49.852 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h[66971]: ignoring --setuser ceph since I am not root 2024-03-26T06:41:49.852 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h[66971]: ignoring --setgroup ceph since I am not root 2024-03-26T06:41:49.852 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mds[66975]: -- 172.21.15.66:0/3268248054 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55be0c88da40 con 0x55be0c860800 2024-03-26T06:41:49.852 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h[66971]: starting mds.h at 2024-03-26T06:41:49.852 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b[67275]: ignoring --setuser ceph since I am not root 2024-03-26T06:41:49.852 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b[67275]: ignoring --setgroup ceph since I am not root 2024-03-26T06:41:49.853 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mds[67279]: -- 172.21.15.66:0/3778591542 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55827e5d7a40 con 0x55827e5aa800 2024-03-26T06:41:49.853 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b[67275]: starting mds.b at 2024-03-26T06:41:49.853 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k[66341]: ignoring --setuser ceph since I am not root 2024-03-26T06:41:49.853 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k[66341]: ignoring --setgroup ceph since I am not root 2024-03-26T06:41:49.853 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-mds[66345]: -- 172.21.15.66:0/2970229967 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a29df53a40 con 0x55a29df26800 2024-03-26T06:41:49.853 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:41:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k[66341]: starting mds.k at 2024-03-26T06:41:49.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1483079413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T06:41:49.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1703261051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T06:41:49.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/682096642' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T06:41:49.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mon[36681]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T06:41:49.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mon[36681]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T06:41:49.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/682096642' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T06:41:49.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mon[36681]: osdmap e117: 12 total, 12 up, 12 in 2024-03-26T06:41:49.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mon[36681]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T06:41:49.913 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f[67449]: ignoring --setuser ceph since I am not root 2024-03-26T06:41:49.913 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f[67449]: ignoring --setgroup ceph since I am not root 2024-03-26T06:41:49.913 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f[67449]: starting mds.f at 2024-03-26T06:41:49.913 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: ignoring --setuser ceph since I am not root 2024-03-26T06:41:49.914 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: ignoring --setgroup ceph since I am not root 2024-03-26T06:41:49.914 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mds[67761]: -- 172.21.15.149:0/2312271972 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558a221c3a40 con 0x558a22196800 2024-03-26T06:41:49.914 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: starting mds.c at 2024-03-26T06:41:49.914 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l[67135]: ignoring --setuser ceph since I am not root 2024-03-26T06:41:49.914 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l[67135]: ignoring --setgroup ceph since I am not root 2024-03-26T06:41:49.914 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mds[67139]: -- 172.21.15.149:0/2880308432 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a08a1404e0 con 0x55a08a116c00 2024-03-26T06:41:49.915 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mds[67139]: -- 172.21.15.149:0/2880308432 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a08a143a40 con 0x55a08a116c00 2024-03-26T06:41:49.915 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l[67135]: starting mds.l at 2024-03-26T06:41:49.915 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i[68066]: ignoring --setuser ceph since I am not root 2024-03-26T06:41:49.915 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i[68066]: ignoring --setgroup ceph since I am not root 2024-03-26T06:41:49.915 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mds[68070]: -- 172.21.15.149:0/486115434 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5557945f04e0 con 0x5557945c6c00 2024-03-26T06:41:49.915 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-mds[68070]: -- 172.21.15.149:0/486115434 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5557945f3a40 con 0x5557945c6c00 2024-03-26T06:41:49.915 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:41:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i[68066]: starting mds.i at 2024-03-26T06:41:50.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.408+0000 7f5170ff9640 1 -- 172.21.15.37:0/2989476656 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 104+0+0 (secure 0 0 0) 0x7f516c0bdc30 con 0x7f5174105190 2024-03-26T06:41:50.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 -- 172.21.15.37:0/2989476656 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f515c0a2970 msgr2=0x7f515c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:50.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f515c0a2970 0x7f515c0a4e30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5174198ae0 tx=0x7f5164005f90 comp rx=0 tx=0).stop 2024-03-26T06:41:50.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 -- 172.21.15.37:0/2989476656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5174105190 msgr2=0x7f51741979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:50.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5174105190 0x7f51741979a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f516c009ca0 tx=0x7f516c032ec0 comp rx=0 tx=0).stop 2024-03-26T06:41:50.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 -- 172.21.15.37:0/2989476656 shutdown_connections 2024-03-26T06:41:50.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f515c0a2970 0x7f515c0a4e30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:50.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5174105190 0x7f51741979a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:50.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51741016c0 0x7f517419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:50.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 --2- 172.21.15.37:0/2989476656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5174100d00 0x7f517419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:50.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 -- 172.21.15.37:0/2989476656 >> 172.21.15.37:0/2989476656 conn(0x7f5174075450 msgr2=0x7f51740fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:50.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 -- 172.21.15.37:0/2989476656 shutdown_connections 2024-03-26T06:41:50.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.411+0000 7f5179de6640 1 -- 172.21.15.37:0/2989476656 wait complete. 2024-03-26T06:41:50.431 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-26T06:41:50.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.541+0000 7f22ae273640 1 -- 172.21.15.37:0/1682839747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8101e80 msgr2=0x7f22a8105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:50.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.541+0000 7f22ae273640 1 --2- 172.21.15.37:0/1682839747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8101e80 0x7f22a8105ec0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f229c009900 tx=0x7f229c02bc30 comp rx=0 tx=0).stop 2024-03-26T06:41:50.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.541+0000 7f22ae273640 1 -- 172.21.15.37:0/1682839747 shutdown_connections 2024-03-26T06:41:50.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.541+0000 7f22ae273640 1 --2- 172.21.15.37:0/1682839747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a81065a0 0x7f22a810e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:50.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.541+0000 7f22ae273640 1 --2- 172.21.15.37:0/1682839747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8101e80 0x7f22a8105ec0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:50.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.541+0000 7f22ae273640 1 --2- 172.21.15.37:0/1682839747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22a81014d0 0x7f22a81018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:50.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.541+0000 7f22ae273640 1 -- 172.21.15.37:0/1682839747 >> 172.21.15.37:0/1682839747 conn(0x7f22a80fcd50 msgr2=0x7f22a80ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:50.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.541+0000 7f22ae273640 1 -- 172.21.15.37:0/1682839747 shutdown_connections 2024-03-26T06:41:50.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.541+0000 7f22ae273640 1 -- 172.21.15.37:0/1682839747 wait complete. 2024-03-26T06:41:50.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.541+0000 7f22ae273640 1 Processor -- start 2024-03-26T06:41:50.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ae273640 1 -- start start 2024-03-26T06:41:50.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ae273640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22a81014d0 0x7f22a819d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:50.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ae273640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a81065a0 0x7f22a819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ae273640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8197ac0 0x7f22a8197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ae273640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a8111690 con 0x7f22a8197ac0 2024-03-26T06:41:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ae273640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a8111510 con 0x7f22a81014d0 2024-03-26T06:41:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ae273640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a8111810 con 0x7f22a81065a0 2024-03-26T06:41:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ac810640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8197ac0 0x7f22a8197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ac810640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8197ac0 0x7f22a8197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59044/0 (socket says 172.21.15.37:59044) 2024-03-26T06:41:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ac810640 1 -- 172.21.15.37:0/1479425998 learned_addr learned my addr 172.21.15.37:0/1479425998 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:50.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22a77fe640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a81065a0 0x7f22a819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:50.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22a7fff640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22a81014d0 0x7f22a819d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:50.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ac810640 1 -- 172.21.15.37:0/1479425998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a81065a0 msgr2=0x7f22a819de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:50.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ac810640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a81065a0 0x7f22a819de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:50.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ac810640 1 -- 172.21.15.37:0/1479425998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22a81014d0 msgr2=0x7f22a819d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:50.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ac810640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22a81014d0 0x7f22a819d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:50.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22ac810640 1 -- 172.21.15.37:0/1479425998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a81985e0 con 0x7f22a8197ac0 2024-03-26T06:41:50.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22a77fe640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a81065a0 0x7f22a819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:41:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.542+0000 7f22a7fff640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22a81014d0 0x7f22a819d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T06:41:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.543+0000 7f22ac810640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8197ac0 0x7f22a8197f70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f22a000b970 tx=0x7f22a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.543+0000 7f22a57fa640 1 -- 172.21.15.37:0/1479425998 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0019070 con 0x7f22a8197ac0 2024-03-26T06:41:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.543+0000 7f22a57fa640 1 -- 172.21.15.37:0/1479425998 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22a000cb40 con 0x7f22a8197ac0 2024-03-26T06:41:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.543+0000 7f22ae273640 1 -- 172.21.15.37:0/1479425998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a81988d0 con 0x7f22a8197ac0 2024-03-26T06:41:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.543+0000 7f22a57fa640 1 -- 172.21.15.37:0/1479425998 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0007650 con 0x7f22a8197ac0 2024-03-26T06:41:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.543+0000 7f22ae273640 1 -- 172.21.15.37:0/1479425998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a81a4a80 con 0x7f22a8197ac0 2024-03-26T06:41:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.544+0000 7f22ae273640 1 -- 172.21.15.37:0/1479425998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a8068530 con 0x7f22a8197ac0 2024-03-26T06:41:50.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.545+0000 7f22a57fa640 1 -- 172.21.15.37:0/1479425998 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7f22a001c070 con 0x7f22a8197ac0 2024-03-26T06:41:50.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.545+0000 7f22a57fa640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f22900a2970 0x7f22900a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:50.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.546+0000 7f22a7fff640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f22900a2970 0x7f22900a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:50.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.546+0000 7f22a57fa640 1 -- 172.21.15.37:0/1479425998 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8899+0+0 (secure 0 0 0) 0x7f22a00ce560 con 0x7f22a8197ac0 2024-03-26T06:41:50.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.546+0000 7f22a7fff640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f22900a2970 0x7f22900a4e30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2298005d00 tx=0x7f2298005ed0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:50.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.550+0000 7f22a57fa640 1 -- 172.21.15.37:0/1479425998 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f22a009b250 con 0x7f22a8197ac0 2024-03-26T06:41:50.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: pgmap v596: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-03-26T06:41:50.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2989476656' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T06:41:50.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T06:41:50.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T06:41:50.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: Cluster is now healthy 2024-03-26T06:41:50.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/2989476656' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T06:41:50.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.37:6839/2212632726,v1:172.21.15.37:6841/2212632726] up:boot 2024-03-26T06:41:50.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.37:6838/3034415038,v1:172.21.15.37:6840/3034415038] up:boot 2024-03-26T06:41:50.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.66:6834/1693712819,v1:172.21.15.66:6835/1693712819] up:boot 2024-03-26T06:41:50.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.66:6836/3202960805,v1:172.21.15.66:6837/3202960805] up:boot 2024-03-26T06:41:50.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.66:6832/1449005953,v1:172.21.15.66:6833/1449005953] up:boot 2024-03-26T06:41:50.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.66:6838/2567734936,v1:172.21.15.66:6839/2567734936] up:boot 2024-03-26T06:41:50.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6834/497075454,v1:172.21.15.149:6837/497075454] up:boot 2024-03-26T06:41:50.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6835/962826712,v1:172.21.15.149:6839/962826712] up:boot 2024-03-26T06:41:50.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6832/1824398236,v1:172.21.15.149:6836/1824398236] up:boot 2024-03-26T06:41:50.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: mds.? [v2:172.21.15.149:6833/3479648865,v1:172.21.15.149:6838/3479648865] up:boot 2024-03-26T06:41:50.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: fsmap 12 up:standby 2024-03-26T06:41:50.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T06:41:50.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T06:41:50.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:41:50.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T06:41:50.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T06:41:50.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T06:41:50.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T06:41:50.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T06:41:50.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T06:41:50.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:50 smithi037 ceph-mon[31974]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T06:41:50.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:50.734+0000 7f22ae273640 1 -- 172.21.15.37:0/1479425998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f22a8198fa0 con 0x7f22a8197ac0 2024-03-26T06:41:50.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: pgmap v596: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-03-26T06:41:50.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2989476656' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T06:41:50.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T06:41:50.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T06:41:50.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: Cluster is now healthy 2024-03-26T06:41:50.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/2989476656' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T06:41:50.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.37:6839/2212632726,v1:172.21.15.37:6841/2212632726] up:boot 2024-03-26T06:41:50.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.37:6838/3034415038,v1:172.21.15.37:6840/3034415038] up:boot 2024-03-26T06:41:50.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.66:6834/1693712819,v1:172.21.15.66:6835/1693712819] up:boot 2024-03-26T06:41:50.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.66:6836/3202960805,v1:172.21.15.66:6837/3202960805] up:boot 2024-03-26T06:41:50.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.66:6832/1449005953,v1:172.21.15.66:6833/1449005953] up:boot 2024-03-26T06:41:50.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.66:6838/2567734936,v1:172.21.15.66:6839/2567734936] up:boot 2024-03-26T06:41:50.851 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6834/497075454,v1:172.21.15.149:6837/497075454] up:boot 2024-03-26T06:41:50.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6835/962826712,v1:172.21.15.149:6839/962826712] up:boot 2024-03-26T06:41:50.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6832/1824398236,v1:172.21.15.149:6836/1824398236] up:boot 2024-03-26T06:41:50.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: mds.? [v2:172.21.15.149:6833/3479648865,v1:172.21.15.149:6838/3479648865] up:boot 2024-03-26T06:41:50.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: fsmap 12 up:standby 2024-03-26T06:41:50.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T06:41:50.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T06:41:50.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:41:50.852 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T06:41:50.853 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T06:41:50.853 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T06:41:50.853 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T06:41:50.853 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T06:41:50.853 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T06:41:50.853 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:50 smithi066 ceph-mon[36504]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T06:41:50.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: pgmap v596: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-03-26T06:41:50.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2989476656' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T06:41:50.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T06:41:50.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T06:41:50.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: Cluster is now healthy 2024-03-26T06:41:50.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/2989476656' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T06:41:50.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.37:6839/2212632726,v1:172.21.15.37:6841/2212632726] up:boot 2024-03-26T06:41:50.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.37:6838/3034415038,v1:172.21.15.37:6840/3034415038] up:boot 2024-03-26T06:41:50.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.66:6834/1693712819,v1:172.21.15.66:6835/1693712819] up:boot 2024-03-26T06:41:50.913 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.66:6836/3202960805,v1:172.21.15.66:6837/3202960805] up:boot 2024-03-26T06:41:50.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.66:6832/1449005953,v1:172.21.15.66:6833/1449005953] up:boot 2024-03-26T06:41:50.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.66:6838/2567734936,v1:172.21.15.66:6839/2567734936] up:boot 2024-03-26T06:41:50.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6834/497075454,v1:172.21.15.149:6837/497075454] up:boot 2024-03-26T06:41:50.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6835/962826712,v1:172.21.15.149:6839/962826712] up:boot 2024-03-26T06:41:50.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6832/1824398236,v1:172.21.15.149:6836/1824398236] up:boot 2024-03-26T06:41:50.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: mds.? [v2:172.21.15.149:6833/3479648865,v1:172.21.15.149:6838/3479648865] up:boot 2024-03-26T06:41:50.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: fsmap 12 up:standby 2024-03-26T06:41:50.915 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T06:41:50.915 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T06:41:50.915 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T06:41:50.915 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T06:41:50.915 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T06:41:50.915 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T06:41:50.915 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T06:41:50.915 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T06:41:50.916 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T06:41:50.916 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:50 smithi149 ceph-mon[36681]: from='mgr.15411 172.21.15.37:0/4157069250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T06:41:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.528+0000 7f22a57fa640 1 -- 172.21.15.37:0/1479425998 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v118) v1 ==== 181+0+0 (secure 0 0 0) 0x7f22a8198fa0 con 0x7f22a8197ac0 2024-03-26T06:41:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-03-26T06:41:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.531+0000 7f22ae273640 1 -- 172.21.15.37:0/1479425998 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f22900a2970 msgr2=0x7f22900a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.531+0000 7f22ae273640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f22900a2970 0x7f22900a4e30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2298005d00 tx=0x7f2298005ed0 comp rx=0 tx=0).stop 2024-03-26T06:41:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.531+0000 7f22ae273640 1 -- 172.21.15.37:0/1479425998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8197ac0 msgr2=0x7f22a8197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.531+0000 7f22ae273640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8197ac0 0x7f22a8197f70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f22a000b970 tx=0x7f22a000be30 comp rx=0 tx=0).stop 2024-03-26T06:41:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.531+0000 7f22ae273640 1 -- 172.21.15.37:0/1479425998 shutdown_connections 2024-03-26T06:41:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.531+0000 7f22ae273640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7f22900a2970 0x7f22900a4e30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:51.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.531+0000 7f22ae273640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8197ac0 0x7f22a8197f70 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:51.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.531+0000 7f22ae273640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22a81065a0 0x7f22a819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:51.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.531+0000 7f22ae273640 1 --2- 172.21.15.37:0/1479425998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22a81014d0 0x7f22a819d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:51.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.531+0000 7f22ae273640 1 -- 172.21.15.37:0/1479425998 >> 172.21.15.37:0/1479425998 conn(0x7f22a80fcd50 msgr2=0x7f22a80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:51.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.532+0000 7f22ae273640 1 -- 172.21.15.37:0/1479425998 shutdown_connections 2024-03-26T06:41:51.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.532+0000 7f22ae273640 1 -- 172.21.15.37:0/1479425998 wait complete. 2024-03-26T06:41:51.554 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-26T06:41:51.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.708+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1838450832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0b8100d90 msgr2=0x7fc0b8104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:51.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.708+0000 7fc0be0b2640 1 --2- 172.21.15.37:0/1838450832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0b8100d90 0x7fc0b8104dd0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009900 tx=0x7fc0ac02bc30 comp rx=0 tx=0).stop 2024-03-26T06:41:51.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.708+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1838450832 shutdown_connections 2024-03-26T06:41:51.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.708+0000 7fc0be0b2640 1 --2- 172.21.15.37:0/1838450832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0b81054b0 0x7fc0b810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:51.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.708+0000 7fc0be0b2640 1 --2- 172.21.15.37:0/1838450832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0b8100d90 0x7fc0b8104dd0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:51.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.708+0000 7fc0be0b2640 1 --2- 172.21.15.37:0/1838450832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b81003e0 0x7fc0b81007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:51.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.708+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1838450832 >> 172.21.15.37:0/1838450832 conn(0x7fc0b80773a0 msgr2=0x7fc0b80777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.708+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1838450832 shutdown_connections 2024-03-26T06:41:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.708+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1838450832 wait complete. 2024-03-26T06:41:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0be0b2640 1 Processor -- start 2024-03-26T06:41:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0be0b2640 1 -- start start 2024-03-26T06:41:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0be0b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b81003e0 0x7fc0b81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0be0b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0b81054b0 0x7fc0b8198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0be0b2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0b8199490 0x7fc0b81a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0be0b2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b8110520 con 0x7fc0b81054b0 2024-03-26T06:41:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0be0b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b81103a0 con 0x7fc0b81003e0 2024-03-26T06:41:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0be0b2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b81106a0 con 0x7fc0b8199490 2024-03-26T06:41:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0b6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0b81054b0 0x7fc0b8198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0b6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0b81054b0 0x7fc0b8198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59058/0 (socket says 172.21.15.37:59058) 2024-03-26T06:41:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0b6ffd640 1 -- 172.21.15.37:0/1448719082 learned_addr learned my addr 172.21.15.37:0/1448719082 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-26T06:41:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0b77fe640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b81003e0 0x7fc0b81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.709+0000 7fc0b7fff640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0b8199490 0x7fc0b81a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0b6ffd640 1 -- 172.21.15.37:0/1448719082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0b8199490 msgr2=0x7fc0b81a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:51.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0b6ffd640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0b8199490 0x7fc0b81a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:51.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0b6ffd640 1 -- 172.21.15.37:0/1448719082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b81003e0 msgr2=0x7fc0b81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:51.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0b6ffd640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b81003e0 0x7fc0b81989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:51.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0b6ffd640 1 -- 172.21.15.37:0/1448719082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b81a4210 con 0x7fc0b81054b0 2024-03-26T06:41:51.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0b7fff640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0b8199490 0x7fc0b81a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T06:41:51.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0b77fe640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b81003e0 0x7fc0b81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T06:41:51.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0b6ffd640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0b81054b0 0x7fc0b8198f00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac036040 tx=0x7fc0ac02ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:51.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0b4ff9640 1 -- 172.21.15.37:0/1448719082 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac041070 con 0x7fc0b81054b0 2024-03-26T06:41:51.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0b4ff9640 1 -- 172.21.15.37:0/1448719082 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac0054e0 con 0x7fc0b81054b0 2024-03-26T06:41:51.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1448719082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b81a44a0 con 0x7fc0b81054b0 2024-03-26T06:41:51.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0b4ff9640 1 -- 172.21.15.37:0/1448719082 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac034440 con 0x7fc0b81054b0 2024-03-26T06:41:51.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.710+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1448719082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b81a4990 con 0x7fc0b81054b0 2024-03-26T06:41:51.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.711+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1448719082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b80680e0 con 0x7fc0b81054b0 2024-03-26T06:41:51.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.712+0000 7fc0b4ff9640 1 -- 172.21.15.37:0/1448719082 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) v1 ==== 137421+0+0 (secure 0 0 0) 0x7fc0ac044070 con 0x7fc0b81054b0 2024-03-26T06:41:51.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.713+0000 7fc0b4ff9640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc0a00a2970 0x7fc0a00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T06:41:51.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.713+0000 7fc0b77fe640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc0a00a2970 0x7fc0a00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T06:41:51.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.713+0000 7fc0b4ff9640 1 -- 172.21.15.37:0/1448719082 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8374+0+0 (secure 0 0 0) 0x7fc0ac0f0b50 con 0x7fc0b81054b0 2024-03-26T06:41:51.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.713+0000 7fc0b77fe640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc0a00a2970 0x7fc0a00a4e30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80058c0 tx=0x7fc0a800a2d0 comp rx=0 tx=0).ready entity=mgr.15411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T06:41:51.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.716+0000 7fc0b4ff9640 1 -- 172.21.15.37:0/1448719082 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc0ac0bdae0 con 0x7fc0b81054b0 2024-03-26T06:41:51.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:51 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1479425998' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T06:41:51.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:51 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1479425998' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T06:41:51.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:51 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1479425998' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T06:41:51.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:51.977+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1448719082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0b81086e0 con 0x7fc0b81054b0 2024-03-26T06:41:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.530+0000 7fc0b4ff9640 1 -- 172.21.15.37:0/1448719082 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc0ac004c50 con 0x7fc0b81054b0 2024-03-26T06:41:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-03-26T06:41:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1448719082 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc0a00a2970 msgr2=0x7fc0a00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc0a00a2970 0x7fc0a00a4e30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80058c0 tx=0x7fc0a800a2d0 comp rx=0 tx=0).stop 2024-03-26T06:41:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1448719082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0b81054b0 msgr2=0x7fc0b8198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T06:41:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0b81054b0 0x7fc0b8198f00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac036040 tx=0x7fc0ac02ce20 comp rx=0 tx=0).stop 2024-03-26T06:41:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1448719082 shutdown_connections 2024-03-26T06:41:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.37:6800/3894397572,v1:172.21.15.37:6801/3894397572] conn(0x7fc0a00a2970 0x7fc0a00a4e30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0b8199490 0x7fc0b81a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0b81054b0 0x7fc0b8198f00 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 --2- 172.21.15.37:0/1448719082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0b81003e0 0x7fc0b81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T06:41:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1448719082 >> 172.21.15.37:0/1448719082 conn(0x7fc0b80773a0 msgr2=0x7fc0b80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T06:41:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1448719082 shutdown_connections 2024-03-26T06:41:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-26T06:41:52.533+0000 7fc0be0b2640 1 -- 172.21.15.37:0/1448719082 wait complete. 2024-03-26T06:41:52.554 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T06:41:52.582 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T06:41:52.629 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T06:41:52.658 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-26T06:41:52.659 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T06:41:52.673 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T06:41:52.688 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T06:41:52.712 INFO:tasks.cephadm:Stopping all daemons... 2024-03-26T06:41:52.713 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-26T06:41:52.713 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.a 2024-03-26T06:41:52.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:52 smithi066 ceph-mon[36504]: pgmap v598: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 296 KiB/s wr, 17 op/s 2024-03-26T06:41:52.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:52 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1479425998' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T06:41:52.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:52 smithi066 ceph-mon[36504]: osdmap e118: 12 total, 12 up, 12 in 2024-03-26T06:41:52.850 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:52 smithi066 ceph-mon[36504]: from='client.? 172.21.15.37:0/1448719082' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T06:41:52.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:52 smithi149 ceph-mon[36681]: pgmap v598: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 296 KiB/s wr, 17 op/s 2024-03-26T06:41:52.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:52 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1479425998' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T06:41:52.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:52 smithi149 ceph-mon[36681]: osdmap e118: 12 total, 12 up, 12 in 2024-03-26T06:41:52.912 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:52 smithi149 ceph-mon[36681]: from='client.? 172.21.15.37:0/1448719082' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T06:41:52.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:52 smithi037 ceph-mon[31974]: pgmap v598: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 296 KiB/s wr, 17 op/s 2024-03-26T06:41:52.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:52 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1479425998' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T06:41:52.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:52 smithi037 ceph-mon[31974]: osdmap e118: 12 total, 12 up, 12 in 2024-03-26T06:41:52.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:52 smithi037 ceph-mon[31974]: from='client.? 172.21.15.37:0/1448719082' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T06:41:52.951 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:52 smithi037 systemd[1]: Stopping Ceph mon.a for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:41:53.451 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a[31970]: 2024-03-26T06:41:53.055+0000 7f0b20c45700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:41:53.451 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a[31970]: 2024-03-26T06:41:53.055+0000 7f0b20c45700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-26T06:41:53.451 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:53 smithi037 podman[96001]: 2024-03-26 06:41:53.096878798 +0000 UTC m=+0.174486337 container died 7e242629978a455360fddaf2f813bf3ffc4352b4f07663664f9fc9e9c728e0db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:41:53.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:53 smithi037 podman[96001]: 2024-03-26 06:41:53.547980272 +0000 UTC m=+0.625587812 container cleanup 7e242629978a455360fddaf2f813bf3ffc4352b4f07663664f9fc9e9c728e0db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-26T06:41:53.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Mar 26 06:41:53 smithi037 bash[96001]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-a 2024-03-26T06:41:54.023 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.a.service' 2024-03-26T06:41:54.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:41:54.073 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-26T06:41:54.074 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-26T06:41:54.074 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.b 2024-03-26T06:41:54.449 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:54 smithi066 systemd[1]: Stopping Ceph mon.b for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:41:54.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:54 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-b[36500]: 2024-03-26T06:41:54.445+0000 7fe69b8b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:41:54.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:54 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-b[36500]: 2024-03-26T06:41:54.445+0000 7fe69b8b3700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-26T06:41:54.800 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:54 smithi066 podman[74442]: 2024-03-26 06:41:54.538227933 +0000 UTC m=+0.197756869 container died c5ad1d6a14325445ea927855991c47b62abb53b64a9188acf05af158472385dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:41:55.099 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:54 smithi066 podman[74442]: 2024-03-26 06:41:54.921179594 +0000 UTC m=+0.580708500 container cleanup c5ad1d6a14325445ea927855991c47b62abb53b64a9188acf05af158472385dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T06:41:55.100 INFO:journalctl@ceph.mon.b.smithi066.stdout:Mar 26 06:41:54 smithi066 bash[74442]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-b 2024-03-26T06:41:55.332 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.b.service' 2024-03-26T06:41:55.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:41:55.384 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-26T06:41:55.385 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-26T06:41:55.385 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.c 2024-03-26T06:41:55.661 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:55 smithi149 systemd[1]: Stopping Ceph mon.c for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:41:56.108 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-c[36672]: 2024-03-26T06:41:55.720+0000 7fabb2ffc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:41:56.109 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-c[36672]: 2024-03-26T06:41:55.720+0000 7fabb2ffc700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-26T06:41:56.109 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:55 smithi149 podman[76913]: 2024-03-26 06:41:55.802987443 +0000 UTC m=+0.191552983 container died db92c4504829203fa82cf49ed105495d5c25821bfd0cf1126775e0c66a29133c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS) 2024-03-26T06:41:56.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:56 smithi149 podman[76913]: 2024-03-26 06:41:56.213411166 +0000 UTC m=+0.601976706 container cleanup db92c4504829203fa82cf49ed105495d5c25821bfd0cf1126775e0c66a29133c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:41:56.412 INFO:journalctl@ceph.mon.c.smithi149.stdout:Mar 26 06:41:56 smithi149 bash[76913]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mon-c 2024-03-26T06:41:56.589 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mon.c.service' 2024-03-26T06:41:56.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:41:56.658 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-26T06:41:56.659 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-26T06:41:56.659 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.x 2024-03-26T06:41:56.950 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:41:56 smithi037 systemd[1]: Stopping Ceph mgr.x for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:41:57.450 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:41:57 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:41:57.102+0000 7fbc88bf7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:41:57.451 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:41:57 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x[32205]: 2024-03-26T06:41:57.102+0000 7fbc88bf7700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-26T06:41:57.451 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:41:57 smithi037 podman[96120]: 2024-03-26 06:41:57.134745045 +0000 UTC m=+0.285793568 container died e777f6a6a5132cb1c797fc1c9bc248c224ccc3591a4c5278e4a3e75066c95449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-26T06:41:57.781 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:41:57 smithi037 podman[96120]: 2024-03-26 06:41:57.544122659 +0000 UTC m=+0.695171176 container cleanup e777f6a6a5132cb1c797fc1c9bc248c224ccc3591a4c5278e4a3e75066c95449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7e98f5f6b535defc7fa715b456904ffc0500aa9f, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-26T06:41:57.781 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Mar 26 06:41:57 smithi037 bash[96120]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-x 2024-03-26T06:41:57.905 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.x.service' 2024-03-26T06:41:57.942 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:41:57.943 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-26T06:41:57.943 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-26T06:41:57.943 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.y 2024-03-26T06:41:58.298 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:41:58 smithi066 systemd[1]: Stopping Ceph mgr.y for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:41:58.600 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:41:58 smithi066 podman[74566]: 2024-03-26 06:41:58.296221963 +0000 UTC m=+0.145834731 container died 9354924ed7bbe9e74a5d342d136f37acd8c0f30138c100b0c6fdfdcc6180aa46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:41:59.021 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:41:58 smithi066 podman[74566]: 2024-03-26 06:41:58.792974799 +0000 UTC m=+0.642587539 container cleanup 9354924ed7bbe9e74a5d342d136f37acd8c0f30138c100b0c6fdfdcc6180aa46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:41:59.021 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Mar 26 06:41:58 smithi066 bash[74566]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-y 2024-03-26T06:41:59.146 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.y.service' 2024-03-26T06:41:59.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:41:59.187 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-26T06:41:59.188 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-26T06:41:59.188 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.z 2024-03-26T06:41:59.515 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:41:59 smithi149 systemd[1]: Stopping Ceph mgr.z for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:41:59.804 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:41:59 smithi149 podman[77034]: 2024-03-26 06:41:59.513436372 +0000 UTC m=+0.131296141 container died 4847a6846fae7177027dc828841082d0c23adbeb9d6c98c7eb182aeca8f20df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-26T06:42:00.162 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:41:59 smithi149 podman[77034]: 2024-03-26 06:41:59.884397886 +0000 UTC m=+0.502257648 container cleanup 4847a6846fae7177027dc828841082d0c23adbeb9d6c98c7eb182aeca8f20df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-26T06:42:00.162 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Mar 26 06:41:59 smithi149 bash[77034]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mgr-z 2024-03-26T06:42:00.305 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mgr.z.service' 2024-03-26T06:42:00.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:42:00.342 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-26T06:42:00.342 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-26T06:42:00.343 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.0 2024-03-26T06:42:00.700 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:00 smithi037 systemd[1]: Stopping Ceph osd.0 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:42:01.200 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0[46788]: 2024-03-26T06:42:00.756+0000 7ff88cfca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:42:01.200 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0[46788]: 2024-03-26T06:42:00.756+0000 7ff88cfca700 -1 osd.0 119 *** Got signal Terminated *** 2024-03-26T06:42:01.201 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0[46788]: 2024-03-26T06:42:00.756+0000 7ff88cfca700 -1 osd.0 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:42:06.494 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:06 smithi037 podman[96241]: 2024-03-26 06:42:06.23275123 +0000 UTC m=+5.697746152 container died bd091674e0d1506f21095afdd1c2a5b978353ac2656582b7714614a30dd1ac26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T06:42:06.877 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:06 smithi037 podman[96241]: 2024-03-26 06:42:06.590706201 +0000 UTC m=+6.055701174 container cleanup bd091674e0d1506f21095afdd1c2a5b978353ac2656582b7714614a30dd1ac26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:42:06.877 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:06 smithi037 bash[96241]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0 2024-03-26T06:42:07.200 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:06 smithi037 podman[96259]: 2024-03-26 06:42:06.875195824 +0000 UTC m=+0.635296130 container remove bd091674e0d1506f21095afdd1c2a5b978353ac2656582b7714614a30dd1ac26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux ) 2024-03-26T06:42:07.477 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:07 smithi037 podman[96349]: 2024-03-26 06:42:07.109381944 +0000 UTC m=+0.017751622 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:42:07.477 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:07 smithi037 podman[96349]: 2024-03-26 06:42:07.351249103 +0000 UTC m=+0.259618757 container create f4f7e4bdc18e7e9533ac1d28076196fd7b19b3c8dabcad7a44aefff923e11d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-26T06:42:07.903 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:07 smithi037 podman[96349]: 2024-03-26 06:42:07.595499512 +0000 UTC m=+0.503869176 container init f4f7e4bdc18e7e9533ac1d28076196fd7b19b3c8dabcad7a44aefff923e11d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-26T06:42:07.904 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:07 smithi037 podman[96349]: 2024-03-26 06:42:07.598477904 +0000 UTC m=+0.506847556 container start f4f7e4bdc18e7e9533ac1d28076196fd7b19b3c8dabcad7a44aefff923e11d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:42:07.904 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:07 smithi037 podman[96349]: 2024-03-26 06:42:07.678769748 +0000 UTC m=+0.587139400 container attach f4f7e4bdc18e7e9533ac1d28076196fd7b19b3c8dabcad7a44aefff923e11d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:42:08.200 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 26 06:42:07 smithi037 podman[96349]: 2024-03-26 06:42:07.902083121 +0000 UTC m=+0.810452790 container died f4f7e4bdc18e7e9533ac1d28076196fd7b19b3c8dabcad7a44aefff923e11d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-26T06:42:08.516 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.0.service' 2024-03-26T06:42:08.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:42:08.562 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-26T06:42:08.562 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-26T06:42:08.562 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.1 2024-03-26T06:42:08.850 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:08 smithi066 systemd[1]: Stopping Ceph osd.1 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:42:09.350 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:09 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1[41142]: 2024-03-26T06:42:09.004+0000 7ff2e84e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:42:09.350 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:09 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1[41142]: 2024-03-26T06:42:09.004+0000 7ff2e84e5700 -1 osd.1 119 *** Got signal Terminated *** 2024-03-26T06:42:09.350 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:09 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1[41142]: 2024-03-26T06:42:09.004+0000 7ff2e84e5700 -1 osd.1 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:42:14.826 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:14 smithi066 podman[74701]: 2024-03-26 06:42:14.40448502 +0000 UTC m=+5.639021608 container died bccd2b310f58e36c305e80fbd5ad6c8d5e317f1ca9a7564254e060dcbb9d8445 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux ) 2024-03-26T06:42:15.099 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:14 smithi066 podman[74701]: 2024-03-26 06:42:14.913614346 +0000 UTC m=+6.148150923 container cleanup bccd2b310f58e36c305e80fbd5ad6c8d5e317f1ca9a7564254e060dcbb9d8445 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:42:15.099 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:14 smithi066 bash[74701]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1 2024-03-26T06:42:15.514 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:15 smithi066 podman[74719]: 2024-03-26 06:42:15.173646065 +0000 UTC m=+0.763428630 container remove bccd2b310f58e36c305e80fbd5ad6c8d5e317f1ca9a7564254e060dcbb9d8445 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-26T06:42:15.846 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:15 smithi066 podman[74809]: 2024-03-26 06:42:15.415608228 +0000 UTC m=+0.018426161 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:42:15.846 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:15 smithi066 podman[74809]: 2024-03-26 06:42:15.524426062 +0000 UTC m=+0.127243988 container create 483c86c8460a70e5d6d79965b5d0a1779e23270c45594054556d055ff4568d2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:42:15.846 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:15 smithi066 podman[74809]: 2024-03-26 06:42:15.74395098 +0000 UTC m=+0.346768927 container init 483c86c8460a70e5d6d79965b5d0a1779e23270c45594054556d055ff4568d2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T06:42:15.846 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:15 smithi066 podman[74809]: 2024-03-26 06:42:15.747421121 +0000 UTC m=+0.350239058 container start 483c86c8460a70e5d6d79965b5d0a1779e23270c45594054556d055ff4568d2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:42:15.847 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:15 smithi066 podman[74809]: 2024-03-26 06:42:15.843476937 +0000 UTC m=+0.446294887 container attach 483c86c8460a70e5d6d79965b5d0a1779e23270c45594054556d055ff4568d2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:42:16.498 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:16 smithi066 podman[74809]: 2024-03-26 06:42:16.166972879 +0000 UTC m=+0.769790819 container died 483c86c8460a70e5d6d79965b5d0a1779e23270c45594054556d055ff4568d2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-26T06:42:16.749 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:16 smithi066 podman[74835]: 2024-03-26 06:42:16.721855849 +0000 UTC m=+0.549022419 container remove 483c86c8460a70e5d6d79965b5d0a1779e23270c45594054556d055ff4568d2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:42:16.749 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:16 smithi066 systemd[1]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-26T06:42:16.749 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:16 smithi066 systemd[1]: Stopped Ceph osd.1 for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:42:16.750 INFO:journalctl@ceph.osd.1.smithi066.stdout:Mar 26 06:42:16 smithi066 systemd[1]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.1.service: Consumed 3min 29.576s CPU time. 2024-03-26T06:42:16.751 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.1.service' 2024-03-26T06:42:16.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:42:16.799 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-26T06:42:16.799 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-26T06:42:16.799 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.2 2024-03-26T06:42:17.107 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:16 smithi149 systemd[1]: Stopping Ceph osd.2 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:42:17.412 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:17 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2[41768]: 2024-03-26T06:42:17.104+0000 7f89ff2aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:42:17.412 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:17 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2[41768]: 2024-03-26T06:42:17.104+0000 7f89ff2aa700 -1 osd.2 119 *** Got signal Terminated *** 2024-03-26T06:42:17.412 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:17 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2[41768]: 2024-03-26T06:42:17.104+0000 7f89ff2aa700 -1 osd.2 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:42:22.998 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:22 smithi149 podman[77172]: 2024-03-26 06:42:22.674702032 +0000 UTC m=+5.680305653 container died 0cb87caeb09e82c1357249880d6448794c355d8a2cda0f071b61f9372d340665 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:42:23.412 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:22 smithi149 podman[77172]: 2024-03-26 06:42:22.995799788 +0000 UTC m=+6.001403397 container cleanup 0cb87caeb09e82c1357249880d6448794c355d8a2cda0f071b61f9372d340665 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-03-26T06:42:23.412 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:22 smithi149 bash[77172]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2 2024-03-26T06:42:23.412 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:23 smithi149 podman[77192]: 2024-03-26 06:42:23.197399307 +0000 UTC m=+0.515534096 container remove 0cb87caeb09e82c1357249880d6448794c355d8a2cda0f071b61f9372d340665 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-03-26T06:42:23.903 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:23 smithi149 podman[77283]: 2024-03-26 06:42:23.509620472 +0000 UTC m=+0.017939683 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:42:23.904 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:23 smithi149 podman[77283]: 2024-03-26 06:42:23.614766293 +0000 UTC m=+0.123085492 container create b79875f3d8da7de171eef709d5d2f204656f6821cb24347e24c7b0344c8d3da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-03-26T06:42:23.904 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:23 smithi149 podman[77283]: 2024-03-26 06:42:23.834478276 +0000 UTC m=+0.342797477 container init b79875f3d8da7de171eef709d5d2f204656f6821cb24347e24c7b0344c8d3da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2) 2024-03-26T06:42:23.904 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:23 smithi149 podman[77283]: 2024-03-26 06:42:23.837531391 +0000 UTC m=+0.345850587 container start b79875f3d8da7de171eef709d5d2f204656f6821cb24347e24c7b0344c8d3da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-26T06:42:23.904 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:23 smithi149 podman[77283]: 2024-03-26 06:42:23.901020747 +0000 UTC m=+0.409339958 container attach b79875f3d8da7de171eef709d5d2f204656f6821cb24347e24c7b0344c8d3da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:42:24.709 INFO:journalctl@ceph.osd.2.smithi149.stdout:Mar 26 06:42:24 smithi149 podman[77283]: 2024-03-26 06:42:24.442270634 +0000 UTC m=+0.950589838 container died b79875f3d8da7de171eef709d5d2f204656f6821cb24347e24c7b0344c8d3da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True) 2024-03-26T06:42:25.076 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.2.service' 2024-03-26T06:42:25.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:42:25.120 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-26T06:42:25.120 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-26T06:42:25.120 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.3 2024-03-26T06:42:25.437 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:25 smithi037 systemd[1]: Stopping Ceph osd.3 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:42:25.700 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:25 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3[53756]: 2024-03-26T06:42:25.434+0000 7f72be209700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:42:25.701 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:25 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3[53756]: 2024-03-26T06:42:25.434+0000 7f72be209700 -1 osd.3 119 *** Got signal Terminated *** 2024-03-26T06:42:25.701 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:25 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3[53756]: 2024-03-26T06:42:25.434+0000 7f72be209700 -1 osd.3 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:42:27.162 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:26 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:26.907+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:27.163 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:27 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:27.059+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:27.163 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:27 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:27.106+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:27.415 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:27 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3[53756]: 2024-03-26T06:42:27.059+0000 7f72b6ff6700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:00.974701+0000 front 2024-03-26T06:42:00.974720+0000 (oldest deadline 2024-03-26T06:42:26.274854+0000) 2024-03-26T06:42:28.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:27 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:27.941+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:28.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:28 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:28.073+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:28.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:28 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:28.099+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:28.450 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:28 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3[53756]: 2024-03-26T06:42:28.024+0000 7f72b6ff6700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:00.974701+0000 front 2024-03-26T06:42:00.974720+0000 (oldest deadline 2024-03-26T06:42:26.274854+0000) 2024-03-26T06:42:29.162 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:28 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:28.896+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:29.162 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:29.120+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:29.163 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:29.086+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:29.450 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:29 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3[53756]: 2024-03-26T06:42:29.014+0000 7f72b6ff6700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:00.974701+0000 front 2024-03-26T06:42:00.974720+0000 (oldest deadline 2024-03-26T06:42:26.274854+0000) 2024-03-26T06:42:29.599 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:29 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:29.099+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:30.162 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:29.880+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:30.162 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:30 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:30.077+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:30.163 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:30 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:30.086+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:30.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:30 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:30.091+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:30.450 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:30 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3[53756]: 2024-03-26T06:42:30.064+0000 7f72b6ff6700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:00.974701+0000 front 2024-03-26T06:42:00.974720+0000 (oldest deadline 2024-03-26T06:42:26.274854+0000) 2024-03-26T06:42:30.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:30 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:30.197+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:31.162 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:30 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:30.886+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:31.162 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:31 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:31.082+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:31.163 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:31 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:31.091+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:31.247 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:31 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:31.167+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:31.247 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:30 smithi037 podman[96459]: 2024-03-26 06:42:30.963072073 +0000 UTC m=+5.645861644 container died 589358d39148774b849268902d509306f1933fb15e308a18080507eda0e40c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:42:31.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:31 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:31.062+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:31.631 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:31 smithi037 podman[96459]: 2024-03-26 06:42:31.343999806 +0000 UTC m=+6.026789419 container cleanup 589358d39148774b849268902d509306f1933fb15e308a18080507eda0e40c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True) 2024-03-26T06:42:31.631 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:31 smithi037 bash[96459]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3 2024-03-26T06:42:31.951 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:31 smithi037 podman[96489]: 2024-03-26 06:42:31.628682494 +0000 UTC m=+0.663939094 container remove 589358d39148774b849268902d509306f1933fb15e308a18080507eda0e40c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T06:42:31.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:31 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:31.693+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:32.285 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:32 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:32.160+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:32.285 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:31 smithi037 podman[96578]: 2024-03-26 06:42:31.866946246 +0000 UTC m=+0.018015316 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:42:32.285 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:31 smithi037 podman[96578]: 2024-03-26 06:42:31.97959516 +0000 UTC m=+0.130664204 container create 03480ce605165e93612788db82c65f49eede7aa0b3ac9edb67496487dad0354c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3-deactivate, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-26T06:42:32.285 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:32 smithi037 podman[96578]: 2024-03-26 06:42:32.207170874 +0000 UTC m=+0.358239918 container init 03480ce605165e93612788db82c65f49eede7aa0b3ac9edb67496487dad0354c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3-deactivate, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-26T06:42:32.285 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:32 smithi037 podman[96578]: 2024-03-26 06:42:32.210287527 +0000 UTC m=+0.361356573 container start 03480ce605165e93612788db82c65f49eede7aa0b3ac9edb67496487dad0354c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-26T06:42:32.286 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:32 smithi037 podman[96578]: 2024-03-26 06:42:32.281982238 +0000 UTC m=+0.433051311 container attach 03480ce605165e93612788db82c65f49eede7aa0b3ac9edb67496487dad0354c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-26T06:42:32.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:32 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:32.080+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:32.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:31 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:31.914+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:32.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:32 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:32.068+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:32.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:32 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:32.085+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:32.700 INFO:journalctl@ceph.osd.3.smithi037.stdout:Mar 26 06:42:32 smithi037 podman[96578]: 2024-03-26 06:42:32.503807526 +0000 UTC m=+0.654876574 container died 03480ce605165e93612788db82c65f49eede7aa0b3ac9edb67496487dad0354c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-26T06:42:33.088 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:32 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:32.718+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:33.107 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.3.service' 2024-03-26T06:42:33.150 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:42:33.150 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-26T06:42:33.151 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-26T06:42:33.151 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.4 2024-03-26T06:42:33.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:32 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:32.956+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:33.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:33 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:33.026+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:33.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:33 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:33.069+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:33.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:33 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:33.210+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:33.462 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:33 smithi066 systemd[1]: Stopping Ceph osd.4 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:42:33.463 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:33 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:33.124+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:33.849 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:33 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4[48135]: 2024-03-26T06:42:33.458+0000 7f1689134700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:42:33.850 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:33 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4[48135]: 2024-03-26T06:42:33.458+0000 7f1689134700 -1 osd.4 119 *** Got signal Terminated *** 2024-03-26T06:42:33.850 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:33 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4[48135]: 2024-03-26T06:42:33.458+0000 7f1689134700 -1 osd.4 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:42:34.167 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:33 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:33.709+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:34.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:34 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:34.080+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:34.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:33 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:33.917+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:34.413 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:34 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:34.024+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:34.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:34 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:34.105+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:34.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:34 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:34.164+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:35.125 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:34 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:34.725+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:35.125 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:34 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:34.725+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:35.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:35 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:35.071+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:35.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:35 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:35.071+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:35.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:34 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:34.910+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:35.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:35 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:35.065+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:35.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:35 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:35.067+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:35.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:35 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:35.122+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:35.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:35 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:35.686+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:35.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:35 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:35.686+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:36.043 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:35 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:35.744+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:36.098 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:35 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:35.909+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:36.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:36 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:36.039+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:36.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:36 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:36.039+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:36.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:36 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:36.113+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:36.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:36 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:36.095+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:36.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:36 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:36.095+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:36.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:36 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:36.105+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:36.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:36 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:36.105+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:36.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:36 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:36.655+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:36.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:36 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:36.655+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:37.066 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:36 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:36.711+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:37.162 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:36 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:36.898+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:37.162 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:37 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:37.080+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:37.163 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:37 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:37.068+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:37.163 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:37 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:37.068+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:37.349 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:37 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4[48135]: 2024-03-26T06:42:37.063+0000 7f1685f29700 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.466978+0000 front 2024-03-26T06:42:11.466947+0000 (oldest deadline 2024-03-26T06:42:36.167084+0000) 2024-03-26T06:42:37.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:37 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:37.075+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:37.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:37 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:37.075+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:37.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:37 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:37.098+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:37.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:37 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:37.098+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:37.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:37 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:37.662+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:37.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:37 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:37.662+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:38.052 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:37 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:37.677+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:38.339 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:38 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4[48135]: 2024-03-26T06:42:38.048+0000 7f1685f29700 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.466978+0000 front 2024-03-26T06:42:11.466947+0000 (oldest deadline 2024-03-26T06:42:36.167084+0000) 2024-03-26T06:42:38.339 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:38 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:38.095+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:38.339 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:38 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:38.095+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:38.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:38 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:38.125+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:38.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:37 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:37.940+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:38.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:37 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:37.940+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:13.293260+0000 front 2024-03-26T06:42:13.293279+0000 (oldest deadline 2024-03-26T06:42:37.393269+0000) 2024-03-26T06:42:38.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:38 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:38.027+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:38.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:38 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:38.027+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:38.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:38 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:38.143+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:38.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:38 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:38.143+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:38.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:38 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:38.680+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:38.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:38 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:38.680+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:39.037 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:38 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:38.688+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:39.162 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:39 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:39.097+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:39.162 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:38 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:38.895+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:39.162 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:38 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:38.895+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:13.293260+0000 front 2024-03-26T06:42:13.293279+0000 (oldest deadline 2024-03-26T06:42:37.393269+0000) 2024-03-26T06:42:39.162 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:39 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:39.040+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:39.163 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:39 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:39.040+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:39.305 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:39 smithi066 podman[74914]: 2024-03-26 06:42:39.035156687 +0000 UTC m=+5.685419951 container died b55f12906546f450aa6d29f129c2a62290297a27d828f26c59f174b750c9ab9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-26T06:42:39.305 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:39 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:39.131+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:39.305 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:39 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:39.131+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:39.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:39 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:39.160+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:39.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:39 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:39.160+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:39.599 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:39 smithi066 podman[74914]: 2024-03-26 06:42:39.401616002 +0000 UTC m=+6.051879264 container cleanup b55f12906546f450aa6d29f129c2a62290297a27d828f26c59f174b750c9ab9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:42:39.600 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:39 smithi066 bash[74914]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4 2024-03-26T06:42:40.023 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:39 smithi066 podman[74943]: 2024-03-26 06:42:39.702587681 +0000 UTC m=+0.661583269 container remove b55f12906546f450aa6d29f129c2a62290297a27d828f26c59f174b750c9ab9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-26T06:42:40.023 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:39 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:39.664+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:40.123 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:39 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:39.711+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:40.123 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:39 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:39.711+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:40.275 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:40 smithi066 podman[75034]: 2024-03-26 06:42:39.923962272 +0000 UTC m=+0.018210421 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:42:40.275 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:40 smithi066 podman[75034]: 2024-03-26 06:42:40.070298962 +0000 UTC m=+0.164547086 container create 043306a96833e7be7b3e9f17647e88b202bde684f89c9c2030adbf9df3c7b7d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-26T06:42:40.275 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:40 smithi066 podman[75034]: 2024-03-26 06:42:40.272578515 +0000 UTC m=+0.366826646 container init 043306a96833e7be7b3e9f17647e88b202bde684f89c9c2030adbf9df3c7b7d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-26T06:42:40.275 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:40 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:40.098+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:40.276 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:40 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:40.098+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:40.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:40 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:40.106+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:40.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:40 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:40.045+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:40.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:40 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:40.045+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:40.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:39 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:39.927+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:40.413 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:39 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:39.927+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:13.293260+0000 front 2024-03-26T06:42:13.293279+0000 (oldest deadline 2024-03-26T06:42:37.393269+0000) 2024-03-26T06:42:40.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:40 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:40.120+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:40.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:40 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:40.120+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:40.578 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:40 smithi066 podman[75034]: 2024-03-26 06:42:40.275848209 +0000 UTC m=+0.370096342 container start 043306a96833e7be7b3e9f17647e88b202bde684f89c9c2030adbf9df3c7b7d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-26T06:42:40.579 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:40 smithi066 podman[75034]: 2024-03-26 06:42:40.356015344 +0000 UTC m=+0.450263487 container attach 043306a96833e7be7b3e9f17647e88b202bde684f89c9c2030adbf9df3c7b7d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T06:42:40.849 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:40 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:40.694+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:40.850 INFO:journalctl@ceph.osd.4.smithi066.stdout:Mar 26 06:42:40 smithi066 podman[75034]: 2024-03-26 06:42:40.576784883 +0000 UTC m=+0.671033023 container died 043306a96833e7be7b3e9f17647e88b202bde684f89c9c2030adbf9df3c7b7d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-26T06:42:41.142 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:40 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:40.744+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:41.142 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:40 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:40.744+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:41.162 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:41 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:41.067+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:41.162 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:41 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:41.068+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:41.162 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:41 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:41.068+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:41.162 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:40 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:40.908+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:41.163 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:40 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:40.908+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:13.293260+0000 front 2024-03-26T06:42:13.293279+0000 (oldest deadline 2024-03-26T06:42:37.393269+0000) 2024-03-26T06:42:41.186 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:41 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:41.081+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:41.186 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:41 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:41.081+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:41.205 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.4.service' 2024-03-26T06:42:41.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:42:41.253 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-26T06:42:41.253 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-26T06:42:41.254 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.5 2024-03-26T06:42:41.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:41.138+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:41.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:41.138+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:41.553 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:41 smithi149 systemd[1]: Stopping Ceph osd.5 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:42:41.912 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:41 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:41.550+0000 7fa5fd1bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:42:41.912 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:41 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:41.550+0000 7fa5fd1bd700 -1 osd.5 119 *** Got signal Terminated *** 2024-03-26T06:42:41.912 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:41 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:41.550+0000 7fa5fd1bd700 -1 osd.5 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:42:42.048 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:41 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:41.739+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:42.174 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:41.749+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:42.175 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:41 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:41.749+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:42.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:42 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:42.044+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:42.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:42 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:42.044+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:42.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:42 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:42.025+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:42.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:41 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:41.927+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:42.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:41 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:41.927+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:13.293260+0000 front 2024-03-26T06:42:13.293279+0000 (oldest deadline 2024-03-26T06:42:37.393269+0000) 2024-03-26T06:42:42.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:42 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:42.104+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:42.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:42 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:42.104+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:42.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:42 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:42.171+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:42.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:42 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:42.171+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:43.084 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:42 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:42.757+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:43.138 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:42 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:42.783+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:43.138 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:42 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:42.783+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:43.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:43 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:43.081+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:43.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:43 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:43.081+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:43.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:43 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:43.048+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:43.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:42 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:42.927+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:43.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:42 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:42.927+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:13.293260+0000 front 2024-03-26T06:42:13.293279+0000 (oldest deadline 2024-03-26T06:42:37.393269+0000) 2024-03-26T06:42:43.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:43 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:43.084+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:43.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:43 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:43.084+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:43.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:43 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:43.135+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:43.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:43 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:43.135+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:44.042 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:43 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:43.731+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:44.127 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:43 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:43.808+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:44.127 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:43 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:43.808+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:44.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:44 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:44.040+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:44.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:44 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:44.040+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:44.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:44 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:44.040+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:44.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:44 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:44.087+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:44.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:44 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:44.126+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:44.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:44 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:44.126+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:44.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:43 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:43.919+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:44.413 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:43 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:43.919+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:13.293260+0000 front 2024-03-26T06:42:13.293279+0000 (oldest deadline 2024-03-26T06:42:37.393269+0000) 2024-03-26T06:42:44.413 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:43 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:43.919+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:22.094144+0000 front 2024-03-26T06:42:22.094182+0000 (oldest deadline 2024-03-26T06:42:43.794304+0000) 2024-03-26T06:42:44.454 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:44 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:44.124+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:44.454 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:44 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:44.124+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:45.090 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:44 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:44.774+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:45.200 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:45 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:45.083+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:45.201 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:45 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:45.083+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:45.201 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:44 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:44.837+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:45.201 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:44 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:44.837+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:45.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:45 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:45.087+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:45.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:45 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:45.087+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:45.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:45 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:45.087+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:45.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:45 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:45.079+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:45.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:45 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:45.130+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:45.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:45 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:45.130+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:45.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:45 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:45.130+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:45.413 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:44 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:44.944+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:45.413 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:44 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:44.944+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:13.293260+0000 front 2024-03-26T06:42:13.293279+0000 (oldest deadline 2024-03-26T06:42:37.393269+0000) 2024-03-26T06:42:45.413 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:44 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:44.944+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:22.094144+0000 front 2024-03-26T06:42:22.094182+0000 (oldest deadline 2024-03-26T06:42:43.794304+0000) 2024-03-26T06:42:46.055 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:45 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:45.732+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:46.056 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:45 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:45.732+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:46.084 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:45 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:45.813+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:46.084 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:45 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:45.813+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:46.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:46 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:46.052+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:46.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:46 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:46.052+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:46.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:46 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:46.052+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:46.411 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:46 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:46.086+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:46.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:45 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:45.955+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:01.790177+0000 front 2024-03-26T06:42:01.790286+0000 (oldest deadline 2024-03-26T06:42:26.390845+0000) 2024-03-26T06:42:46.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:45 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:45.955+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:13.293260+0000 front 2024-03-26T06:42:13.293279+0000 (oldest deadline 2024-03-26T06:42:37.393269+0000) 2024-03-26T06:42:46.412 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:45 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5[48756]: 2024-03-26T06:42:45.955+0000 7fa5f9fb2700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:22.094144+0000 front 2024-03-26T06:42:22.094182+0000 (oldest deadline 2024-03-26T06:42:43.794304+0000) 2024-03-26T06:42:46.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:46 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:46.151+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:46.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:46 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:46.151+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:46.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:46 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:46.151+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:46.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:46 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:46.081+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:46.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:46 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:46.081+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:47.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:47 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:47.012+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:47.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:47 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:47.012+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:47.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:47 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:47.012+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:47.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:46 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:46.767+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:47.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:46 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:46.767+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:47.200 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:47 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:47.049+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:47.201 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:47 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:47.049+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:47.201 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:47 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:47.049+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.850063+0000 front 2024-03-26T06:42:21.849964+0000 (oldest deadline 2024-03-26T06:42:46.549985+0000) 2024-03-26T06:42:47.201 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:46 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:46.802+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:47.201 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:46 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:46.802+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:47.201 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:46 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:46.802+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:47.304 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:47 smithi149 podman[77412]: 2024-03-26 06:42:47.016500782 +0000 UTC m=+5.571527801 container died bec29ba7d6e42180317d6b6fb7fd9f8dc1b47465daaa530d960c76a2006e2ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-26T06:42:47.305 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:47 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:47.118+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:47.305 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:47 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:47.171+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:47.305 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:47 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:47.171+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:47.305 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:47 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:47.171+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:47.645 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:47 smithi149 podman[77412]: 2024-03-26 06:42:47.400873678 +0000 UTC m=+5.955900729 container cleanup bec29ba7d6e42180317d6b6fb7fd9f8dc1b47465daaa530d960c76a2006e2ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-26T06:42:47.646 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:47 smithi149 bash[77412]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5 2024-03-26T06:42:47.912 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:47 smithi149 podman[77429]: 2024-03-26 06:42:47.643451078 +0000 UTC m=+0.627722306 container remove bec29ba7d6e42180317d6b6fb7fd9f8dc1b47465daaa530d960c76a2006e2ff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-26T06:42:48.099 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:47 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:47.809+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:48.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:47 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:47.809+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:48.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:47 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:47.973+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:48.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:47 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:47.973+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:48.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:47 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:47.973+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:48.200 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:48 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:48.006+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:48.201 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:48 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:48.006+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:48.201 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:48 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:48.006+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.850063+0000 front 2024-03-26T06:42:21.849964+0000 (oldest deadline 2024-03-26T06:42:46.549985+0000) 2024-03-26T06:42:48.201 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:47 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:47.775+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:48.201 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:47 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:47.775+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:48.201 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:47 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:47.775+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:48.223 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:48 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:48.097+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:48.224 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:48 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:48.097+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:48.224 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:47 smithi149 podman[77519]: 2024-03-26 06:42:47.86848827 +0000 UTC m=+0.018180439 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:42:48.224 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:47 smithi149 podman[77519]: 2024-03-26 06:42:47.994392237 +0000 UTC m=+0.144084405 container create b02441ec02f41f63f97161ba33e3e308c285864bad0a45ab1868ca6c14a95014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-26T06:42:48.224 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:48 smithi149 podman[77519]: 2024-03-26 06:42:48.221678066 +0000 UTC m=+0.371370238 container init b02441ec02f41f63f97161ba33e3e308c285864bad0a45ab1868ca6c14a95014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-26T06:42:48.224 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:48 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:48.125+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:48.225 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:48 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:48.125+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:48.225 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:48 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:48.125+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:48.522 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:48 smithi149 podman[77519]: 2024-03-26 06:42:48.224935731 +0000 UTC m=+0.374627901 container start b02441ec02f41f63f97161ba33e3e308c285864bad0a45ab1868ca6c14a95014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:42:48.522 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:48 smithi149 podman[77519]: 2024-03-26 06:42:48.338301011 +0000 UTC m=+0.487993200 container attach b02441ec02f41f63f97161ba33e3e308c285864bad0a45ab1868ca6c14a95014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-26T06:42:48.802 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 26 06:42:48 smithi149 podman[77519]: 2024-03-26 06:42:48.519862858 +0000 UTC m=+0.669555039 container died b02441ec02f41f63f97161ba33e3e308c285864bad0a45ab1868ca6c14a95014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:42:49.038 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:48 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:48.739+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:49.039 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:48 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:48.739+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:49.039 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:48 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:48.739+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:49.071 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.5.service' 2024-03-26T06:42:49.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:48 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:48.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:49.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:48 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:48.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:49.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:48 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:48.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:49.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:48 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:48.776+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:49.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:48 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:48.776+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:49.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:42:49.112 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-26T06:42:49.112 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-26T06:42:49.113 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.6 2024-03-26T06:42:49.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:49.051+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:49.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:49.052+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:49.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:49.148+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:49.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:49.148+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:49.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:49 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:49.148+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:49.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:49.036+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:49.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:49.036+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:49.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:49.036+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.850063+0000 front 2024-03-26T06:42:21.849964+0000 (oldest deadline 2024-03-26T06:42:46.549985+0000) 2024-03-26T06:42:49.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:49 smithi037 systemd[1]: Stopping Ceph osd.6 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:42:49.951 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:49.546+0000 7f75263e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:42:49.951 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:49.546+0000 7f75263e5700 -1 osd.6 119 *** Got signal Terminated *** 2024-03-26T06:42:49.951 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:49.546+0000 7f75263e5700 -1 osd.6 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:42:49.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:49.688+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:49.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:49.688+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:49.952 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:49.688+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:50.099 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:49.742+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:50.099 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:49.742+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:50.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:49.927+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:50.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:49.927+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:50.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:49 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:49.927+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:50.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:50 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:50.050+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:50.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:50 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:50.050+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:50.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:50 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:50.141+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:50.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:50 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:50.141+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:50.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:50 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:50.141+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:50.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:50 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:50.001+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:50.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:50 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:50.001+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:50.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:50 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:50.001+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.850063+0000 front 2024-03-26T06:42:21.849964+0000 (oldest deadline 2024-03-26T06:42:46.549985+0000) 2024-03-26T06:42:50.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:50 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:50.675+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:50.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:50 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:50.675+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:50.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:50 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:50.675+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:51.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:50 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:50.776+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:51.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:50 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:50.776+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:51.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:50 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:50.936+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:51.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:50 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:50.936+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:51.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:50 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:50.936+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:51.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:51 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:51.059+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:51.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:51 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:51.059+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:51.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:51 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:51.092+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:51.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:51 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:51.092+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:51.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:51 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:51.092+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:51.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:51 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:51.001+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:51.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:51 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:51.001+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:51.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:51 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:51.001+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.850063+0000 front 2024-03-26T06:42:21.849964+0000 (oldest deadline 2024-03-26T06:42:46.549985+0000) 2024-03-26T06:42:51.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:51 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:51.636+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:51.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:51 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:51.636+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:51.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:51 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:51.636+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:52.099 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:51 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:51.751+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:52.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:51 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:51.751+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:52.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:51 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:51.751+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.756034+0000 front 2024-03-26T06:42:29.756123+0000 (oldest deadline 2024-03-26T06:42:51.456076+0000) 2024-03-26T06:42:52.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:51 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:51.891+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:52.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:51 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:51.891+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:52.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:51 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:51.891+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:52.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:51 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:51.891+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:42:52.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:52 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:52.091+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:52.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:52 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:52.091+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:52.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:52 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:52.091+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:52.413 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:52 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:52.090+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:52.413 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:52 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:52.090+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:52.413 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:52 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:52.090+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:42:52.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:52 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:52.033+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:52.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:52 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:52.033+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:52.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:52 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:52.033+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.850063+0000 front 2024-03-26T06:42:21.849964+0000 (oldest deadline 2024-03-26T06:42:46.549985+0000) 2024-03-26T06:42:52.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:52 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:52.033+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:26.550416+0000 front 2024-03-26T06:42:26.550347+0000 (oldest deadline 2024-03-26T06:42:51.850482+0000) 2024-03-26T06:42:52.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:52 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:52.656+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:52.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:52 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:52.656+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:52.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:52 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:52.656+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:53.099 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:52 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:52.766+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:53.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:52 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:52.766+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:53.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:52 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:52.766+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.756034+0000 front 2024-03-26T06:42:29.756123+0000 (oldest deadline 2024-03-26T06:42:51.456076+0000) 2024-03-26T06:42:53.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:52 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:52.854+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:53.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:52 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:52.854+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:53.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:52 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:52.854+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:53.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:52 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:52.854+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:42:53.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:53.118+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:53.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:53.118+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:53.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:53.118+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:53.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:53.042+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:53.413 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:53.042+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:53.413 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:53 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:53.042+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:42:53.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:53.044+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:53.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:53.044+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:53.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:53.044+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.850063+0000 front 2024-03-26T06:42:21.849964+0000 (oldest deadline 2024-03-26T06:42:46.549985+0000) 2024-03-26T06:42:53.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:53.044+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:26.550416+0000 front 2024-03-26T06:42:26.550347+0000 (oldest deadline 2024-03-26T06:42:51.850482+0000) 2024-03-26T06:42:53.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:53.657+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:53.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:53.657+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:53.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:53 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:53.657+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:54.099 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:53.814+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:54.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:53.814+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:54.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:53.814+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.756034+0000 front 2024-03-26T06:42:29.756123+0000 (oldest deadline 2024-03-26T06:42:51.456076+0000) 2024-03-26T06:42:54.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:53.831+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:54.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:53.831+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:54.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:53.831+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:54.101 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:53.831+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:42:54.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:54 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:54.005+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:54.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:54 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:54.005+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:54.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:54 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:54.005+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:42:54.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:54 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:54.160+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:54.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:54 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:54.160+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:54.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:54 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:54.160+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:54.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:54 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:54.067+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:04.346089+0000 front 2024-03-26T06:42:04.346313+0000 (oldest deadline 2024-03-26T06:42:29.646319+0000) 2024-03-26T06:42:54.450 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:54 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:54.067+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.947858+0000 front 2024-03-26T06:42:11.947941+0000 (oldest deadline 2024-03-26T06:42:35.447924+0000) 2024-03-26T06:42:54.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:54 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:54.067+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.850063+0000 front 2024-03-26T06:42:21.849964+0000 (oldest deadline 2024-03-26T06:42:46.549985+0000) 2024-03-26T06:42:54.451 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:54 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6[60476]: 2024-03-26T06:42:54.067+0000 7f75231da700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:26.550416+0000 front 2024-03-26T06:42:26.550347+0000 (oldest deadline 2024-03-26T06:42:51.850482+0000) 2024-03-26T06:42:54.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:54 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:54.623+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:54.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:54 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:54.623+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:54.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:54 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:54.623+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:55.099 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:54 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:54.813+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:55.099 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:54 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:54.813+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:55.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:54 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:54.813+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.756034+0000 front 2024-03-26T06:42:29.756123+0000 (oldest deadline 2024-03-26T06:42:51.456076+0000) 2024-03-26T06:42:55.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:54 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:54.853+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:55.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:54 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:54.853+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:55.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:54 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:54.853+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:55.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:54 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:54.853+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:42:55.392 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:55 smithi037 podman[96682]: 2024-03-26 06:42:55.135766045 +0000 UTC m=+5.824117043 container died 5a19289774109d523b235a557ed985fa85daaba0d3a8565851bb2a6f842095ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-26T06:42:55.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:54 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:54.976+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:55.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:54 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:54.976+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:55.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:54 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:54.976+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:42:55.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:55.132+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:55.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:55.132+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:55.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:55.132+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:55.700 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:55 smithi037 podman[96682]: 2024-03-26 06:42:55.488405704 +0000 UTC m=+6.176756689 container cleanup 5a19289774109d523b235a557ed985fa85daaba0d3a8565851bb2a6f842095ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:42:55.701 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:55 smithi037 bash[96682]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6 2024-03-26T06:42:55.701 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:55 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:55.581+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:55.701 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:55 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:55.581+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:55.701 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:55 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:55.581+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:55.702 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:55 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:55.581+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:42:56.099 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:55.812+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:56.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:55.812+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:56.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:55.812+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.756034+0000 front 2024-03-26T06:42:29.756123+0000 (oldest deadline 2024-03-26T06:42:51.456076+0000) 2024-03-26T06:42:56.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:55.888+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:56.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:55.888+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:56.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:55.888+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:56.101 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:55 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:55.888+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:42:56.110 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:55 smithi037 podman[96699]: 2024-03-26 06:42:55.764768019 +0000 UTC m=+0.629584028 container remove 5a19289774109d523b235a557ed985fa85daaba0d3a8565851bb2a6f842095ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:42:56.372 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:56 smithi037 podman[96791]: 2024-03-26 06:42:56.011213612 +0000 UTC m=+0.017516762 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:42:56.373 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:56 smithi037 podman[96791]: 2024-03-26 06:42:56.240994781 +0000 UTC m=+0.247297918 container create f6ead8d14accb4a4c1941bc2e5d3d9ab20478d644b0c5e94979bbd9d3794668b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-26T06:42:56.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:55.988+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:56.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:55.988+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:56.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:55 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:55.988+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:42:56.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:56 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:56.159+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:56.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:56 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:56.159+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:56.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:56 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:56.159+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:56.700 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:56 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:56.540+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:56.701 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:56 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:56.540+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:56.701 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:56 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:56.540+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:56.701 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:56 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:56.540+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:42:56.701 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:56 smithi037 podman[96791]: 2024-03-26 06:42:56.468778501 +0000 UTC m=+0.475081650 container init f6ead8d14accb4a4c1941bc2e5d3d9ab20478d644b0c5e94979bbd9d3794668b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:42:56.701 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:56 smithi037 podman[96791]: 2024-03-26 06:42:56.472241043 +0000 UTC m=+0.478544178 container start f6ead8d14accb4a4c1941bc2e5d3d9ab20478d644b0c5e94979bbd9d3794668b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:42:56.702 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:56 smithi037 podman[96791]: 2024-03-26 06:42:56.551679371 +0000 UTC m=+0.557982512 container attach f6ead8d14accb4a4c1941bc2e5d3d9ab20478d644b0c5e94979bbd9d3794668b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-26T06:42:57.065 INFO:journalctl@ceph.osd.6.smithi037.stdout:Mar 26 06:42:56 smithi037 podman[96791]: 2024-03-26 06:42:56.770086197 +0000 UTC m=+0.776389335 container died f6ead8d14accb4a4c1941bc2e5d3d9ab20478d644b0c5e94979bbd9d3794668b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-26T06:42:57.099 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:56 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:56.823+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:57.099 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:56 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:56.823+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:57.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:56 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:56.823+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.756034+0000 front 2024-03-26T06:42:29.756123+0000 (oldest deadline 2024-03-26T06:42:51.456076+0000) 2024-03-26T06:42:57.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:56 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:56.892+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:57.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:56 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:56.892+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:57.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:56 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:56.892+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:57.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:56 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:56.892+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:42:57.350 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.6.service' 2024-03-26T06:42:57.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:42:57.393 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-26T06:42:57.393 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-26T06:42:57.393 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.7 2024-03-26T06:42:57.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:56 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:56.939+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:57.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:56 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:56.939+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:57.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:56 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:56.939+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:42:57.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:57 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:57.149+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:57.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:57 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:57.149+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:57.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:57 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:57.149+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:57.615 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:57 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:57.566+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:57.615 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:57 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:57.566+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:57.615 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:57 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:57.566+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:57.616 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:57 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:57.566+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:42:57.697 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:57 smithi066 systemd[1]: Stopping Ceph osd.7 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:42:58.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:57 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:57.931+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:58.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:57 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:57.932+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:58.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:57 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:57.932+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:58.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:57 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:57.932+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:42:58.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:57 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:57.694+0000 7f00acd9e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:42:58.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:57 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:57.694+0000 7f00acd9e700 -1 osd.7 119 *** Got signal Terminated *** 2024-03-26T06:42:58.101 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:57 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:57.694+0000 7f00acd9e700 -1 osd.7 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:42:58.101 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:57 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:57.863+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:58.101 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:57 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:57.863+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:58.101 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:57 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:57.863+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.756034+0000 front 2024-03-26T06:42:29.756123+0000 (oldest deadline 2024-03-26T06:42:51.456076+0000) 2024-03-26T06:42:58.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:57 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:57.938+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:58.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:57 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:57.938+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:58.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:57 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:57.938+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:42:58.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:58 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:58.110+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:58.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:58 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:58.110+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:58.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:58 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:58.110+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:58.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:58.591+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:58.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:58.591+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:58.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:58.591+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:58.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:58 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:58.591+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:42:59.349 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:58.900+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:42:59.350 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:58.900+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:42:59.350 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:58.900+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.756034+0000 front 2024-03-26T06:42:29.756123+0000 (oldest deadline 2024-03-26T06:42:51.456076+0000) 2024-03-26T06:42:59.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:58.907+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:42:59.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:58.907+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:42:59.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:58.907+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:42:59.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:58.907+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:42:59.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:58 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:58.983+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:42:59.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:58 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:58.983+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:42:59.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:42:58 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:42:58.983+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:42:59.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:59 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:59.119+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:42:59.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:59 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:59.119+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:42:59.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:42:59 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:42:59.119+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:42:59.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:59 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:59.621+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:42:59.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:59 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:59.621+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:42:59.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:59 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:59.621+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:42:59.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:42:59 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:42:59.621+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:00.349 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:59 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:59.862+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:43:00.350 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:59 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:59.862+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:43:00.350 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:59 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:59.862+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.756034+0000 front 2024-03-26T06:42:29.756123+0000 (oldest deadline 2024-03-26T06:42:51.456076+0000) 2024-03-26T06:43:00.350 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:42:59 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:42:59.862+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:33.157122+0000 front 2024-03-26T06:42:33.157164+0000 (oldest deadline 2024-03-26T06:42:59.057310+0000) 2024-03-26T06:43:00.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:59 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:59.902+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:00.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:59 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:59.902+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:00.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:59 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:59.902+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:00.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:42:59 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:42:59.902+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:00.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:00.003+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:43:00.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:00.003+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:43:00.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:00.003+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:43:00.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:00.084+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:00.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:00.084+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:00.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:00.084+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:00.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:00.650+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:00.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:00.650+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:00.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:00.650+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:00.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:00 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:00.650+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:01.267 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:00.956+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:43:01.267 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:00.956+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:43:01.267 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:00.956+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:43:01.268 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:00 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:00.956+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.592466+0000 front 2024-03-26T06:42:34.592536+0000 (oldest deadline 2024-03-26T06:43:00.492462+0000) 2024-03-26T06:43:01.268 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:01.083+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:01.268 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:01.083+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:01.268 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:01.083+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:01.349 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:00 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:43:00.848+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:43:01.349 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:00 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:43:00.848+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:43:01.350 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:00 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:43:00.848+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.756034+0000 front 2024-03-26T06:42:29.756123+0000 (oldest deadline 2024-03-26T06:42:51.456076+0000) 2024-03-26T06:43:01.350 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:00 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:43:00.848+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:33.157122+0000 front 2024-03-26T06:42:33.157164+0000 (oldest deadline 2024-03-26T06:42:59.057310+0000) 2024-03-26T06:43:01.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:00 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:00.883+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:01.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:00 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:00.883+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:01.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:00 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:00.883+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:01.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:00 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:00.883+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:01.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:00 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:00.883+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:01.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:01 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:01.616+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:01.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:01 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:01.616+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:01.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:01 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:01.616+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:01.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:01 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:01.616+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:02.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:01 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:43:01.836+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:12.952254+0000 front 2024-03-26T06:42:12.952438+0000 (oldest deadline 2024-03-26T06:42:35.252391+0000) 2024-03-26T06:43:02.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:01 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:43:01.836+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.254190+0000 front 2024-03-26T06:42:19.254128+0000 (oldest deadline 2024-03-26T06:42:45.154110+0000) 2024-03-26T06:43:02.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:01 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:43:01.836+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.756034+0000 front 2024-03-26T06:42:29.756123+0000 (oldest deadline 2024-03-26T06:42:51.456076+0000) 2024-03-26T06:43:02.100 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:01 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7[55167]: 2024-03-26T06:43:01.836+0000 7f00a5b8b700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:33.157122+0000 front 2024-03-26T06:42:33.157164+0000 (oldest deadline 2024-03-26T06:42:59.057310+0000) 2024-03-26T06:43:02.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:01 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:01.855+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:02.101 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:01 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:01.855+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:02.101 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:01 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:01.855+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:02.101 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:01 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:01.855+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:02.101 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:01 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:01.855+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:02.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:02 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:02.115+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:02.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:02 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:02.115+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:02.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:02 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:02.115+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:02.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:01.994+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:43:02.413 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:01.994+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:43:02.413 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:01.994+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:43:02.413 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:01 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:01.995+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.592466+0000 front 2024-03-26T06:42:34.592536+0000 (oldest deadline 2024-03-26T06:43:00.492462+0000) 2024-03-26T06:43:02.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:02 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:02.579+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:02.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:02 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:02.579+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:02.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:02 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:02.579+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:02.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:02 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:02.579+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:03.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:02 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:02.842+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:03.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:02 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:02.842+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:03.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:02 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:02.842+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:03.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:02 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:02.842+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:03.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:02 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:02.842+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:03.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:03.034+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:43:03.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:03.034+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:43:03.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:03.034+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:43:03.413 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:03.034+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.592466+0000 front 2024-03-26T06:42:34.592536+0000 (oldest deadline 2024-03-26T06:43:00.492462+0000) 2024-03-26T06:43:03.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:03.135+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:03.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:03.135+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:03.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:03.135+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:03.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:03 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:03.135+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:03.524 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:03 smithi066 podman[75142]: 2024-03-26 06:43:03.25384032 +0000 UTC m=+5.653514103 container died 23ffe3128da189fff1834c3e88fd9e51a97341973486062b4b92a6ee70e6f948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-26T06:43:03.849 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:03 smithi066 podman[75142]: 2024-03-26 06:43:03.620414309 +0000 UTC m=+6.020088098 container cleanup 23ffe3128da189fff1834c3e88fd9e51a97341973486062b4b92a6ee70e6f948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:43:03.850 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:03 smithi066 bash[75142]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7 2024-03-26T06:43:03.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:03 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:03.533+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:03.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:03 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:03.533+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:03.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:03 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:03.533+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:03.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:03 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:03.533+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:03.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:03 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:03.533+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:04.258 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:03 smithi066 podman[75159]: 2024-03-26 06:43:03.92956816 +0000 UTC m=+0.667445684 container remove 23ffe3128da189fff1834c3e88fd9e51a97341973486062b4b92a6ee70e6f948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-26T06:43:04.258 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:03 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:03.852+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:04.258 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:03 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:03.852+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:04.259 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:03 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:03.852+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:04.259 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:03 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:03.852+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:04.259 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:03 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:03.852+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:04.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:04.034+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:43:04.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:04.034+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:43:04.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:04.034+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:43:04.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:04.034+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.592466+0000 front 2024-03-26T06:42:34.592536+0000 (oldest deadline 2024-03-26T06:43:00.492462+0000) 2024-03-26T06:43:04.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:04.179+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:04.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:04.179+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:04.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:04.179+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:04.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:04 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:04.179+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:04.568 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:04 smithi066 podman[75251]: 2024-03-26 06:43:04.159308431 +0000 UTC m=+0.018303223 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:43:04.568 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:04 smithi066 podman[75251]: 2024-03-26 06:43:04.263725866 +0000 UTC m=+0.122720661 container create 455eddf683fda86ddffdbe9f2dfc8a29f8d68a71841efd79982afbf58b08cca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-26T06:43:04.568 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:04 smithi066 podman[75251]: 2024-03-26 06:43:04.499336176 +0000 UTC m=+0.358330952 container init 455eddf683fda86ddffdbe9f2dfc8a29f8d68a71841efd79982afbf58b08cca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-26T06:43:04.569 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:04 smithi066 podman[75251]: 2024-03-26 06:43:04.502676606 +0000 UTC m=+0.361671390 container start 455eddf683fda86ddffdbe9f2dfc8a29f8d68a71841efd79982afbf58b08cca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:43:04.569 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:04 smithi066 podman[75251]: 2024-03-26 06:43:04.56590654 +0000 UTC m=+0.424901343 container attach 455eddf683fda86ddffdbe9f2dfc8a29f8d68a71841efd79982afbf58b08cca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-26T06:43:04.834 INFO:journalctl@ceph.osd.7.smithi066.stdout:Mar 26 06:43:04 smithi066 podman[75251]: 2024-03-26 06:43:04.803990344 +0000 UTC m=+0.662985135 container died 455eddf683fda86ddffdbe9f2dfc8a29f8d68a71841efd79982afbf58b08cca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:43:04.834 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:04.830+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:04.834 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:04.831+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:04.835 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:04.831+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:04.835 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:04.831+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:04.835 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:04 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:04.831+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:04.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:04 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:04.512+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:04.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:04 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:04.512+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:04.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:04 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:04.512+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:04.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:04 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:04.512+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:04.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:04 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:04.512+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:05.250 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.7.service' 2024-03-26T06:43:05.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:43:05.295 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-26T06:43:05.296 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-26T06:43:05.296 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.8 2024-03-26T06:43:05.324 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:05.019+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:43:05.325 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:05.019+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:43:05.325 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:05.019+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:43:05.325 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:05.019+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.592466+0000 front 2024-03-26T06:42:34.592536+0000 (oldest deadline 2024-03-26T06:43:00.492462+0000) 2024-03-26T06:43:05.325 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:05.168+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:05.325 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:05.168+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:05.325 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:05.168+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:05.325 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:05.168+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:05.589 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 systemd[1]: Stopping Ceph osd.8 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:43:05.911 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:05.586+0000 7ff16a56e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:43:05.912 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:05.586+0000 7ff16a56e700 -1 osd.8 119 *** Got signal Terminated *** 2024-03-26T06:43:05.912 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:05.586+0000 7ff16a56e700 -1 osd.8 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:43:05.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:05 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:05.554+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:05.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:05 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:05.554+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:05.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:05 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:05.554+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:05.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:05 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:05.554+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:05.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:05 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:05.554+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:06.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:05 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:05.842+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:06.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:05 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:05.842+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:06.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:05 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:05.842+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:06.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:05 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:05.842+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:06.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:05 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:05.842+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:06.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:05.978+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:43:06.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:05.978+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:43:06.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:05.978+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:43:06.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:05 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:05.978+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.592466+0000 front 2024-03-26T06:42:34.592536+0000 (oldest deadline 2024-03-26T06:43:00.492462+0000) 2024-03-26T06:43:06.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:06 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:06.129+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:06.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:06 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:06.129+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:06.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:06 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:06.129+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:06.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:06 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:06.129+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:06.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:06 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:06.518+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:06.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:06 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:06.518+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:06.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:06 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:06.518+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:06.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:06 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:06.518+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:06.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:06 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:06.518+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:07.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:06.799+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:07.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:06.799+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:07.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:06.799+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:07.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:06.799+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:07.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:06.799+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:07.314 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:07 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:07.136+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:07.314 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:07 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:07.136+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:07.314 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:07 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:07.136+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:07.315 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:07 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:07.136+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:07.315 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:06 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:06.929+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:43:07.315 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:06 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:06.929+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:43:07.315 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:06 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:06.929+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:43:07.315 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:06 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:06.929+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.592466+0000 front 2024-03-26T06:42:34.592536+0000 (oldest deadline 2024-03-26T06:43:00.492462+0000) 2024-03-26T06:43:07.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:07 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:07.533+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:07.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:07 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:07.533+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:07.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:07 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:07.533+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:07.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:07 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:07.533+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:07.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:07 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:07.533+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:08.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:07 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:07.786+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:08.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:07 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:07.786+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:08.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:07 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:07.786+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:08.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:07 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:07.786+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:08.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:07 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:07.786+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:08.162 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:07 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:07.901+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:43:08.162 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:07 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:07.901+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:43:08.162 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:07 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:07.901+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:43:08.162 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:07 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:07.901+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.592466+0000 front 2024-03-26T06:42:34.592536+0000 (oldest deadline 2024-03-26T06:43:00.492462+0000) 2024-03-26T06:43:08.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:08 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:08.165+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:08.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:08 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:08.165+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:08.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:08 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:08.165+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:08.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:08 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:08.165+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:08.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:08 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:08.569+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:08.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:08 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:08.569+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:08.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:08 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:08.569+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:08.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:08 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:08.569+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:08.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:08 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:08.569+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:09.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:08 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:08.822+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:09.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:08 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:08.822+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:09.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:08 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:08.822+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:09.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:08 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:08.822+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:09.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:08 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:08.822+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:09.190 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:08 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:08.931+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:43:09.190 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:08 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:08.931+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:43:09.190 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:08 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:08.931+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:43:09.190 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:08 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:08.931+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.592466+0000 front 2024-03-26T06:42:34.592536+0000 (oldest deadline 2024-03-26T06:43:00.492462+0000) 2024-03-26T06:43:09.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:09 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:09.187+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:09.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:09 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:09.187+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:09.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:09 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:09.187+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:09.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:09 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:09.187+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:09.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:09 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:09.537+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:09.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:09 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:09.537+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:09.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:09 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:09.537+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:09.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:09 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:09.537+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:09.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:09 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:09.537+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:10.099 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:09 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:09.822+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:10.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:09 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:09.822+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:10.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:09 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:09.822+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:10.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:09 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:09.822+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:10.100 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:09 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:09.822+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:10.240 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:09 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:09.927+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.786287+0000 front 2024-03-26T06:42:03.786376+0000 (oldest deadline 2024-03-26T06:42:26.686299+0000) 2024-03-26T06:43:10.286 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:09 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:09.927+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.690520+0000 front 2024-03-26T06:42:21.690565+0000 (oldest deadline 2024-03-26T06:42:47.590742+0000) 2024-03-26T06:43:10.286 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:09 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:09.927+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:27.591476+0000 front 2024-03-26T06:42:27.591491+0000 (oldest deadline 2024-03-26T06:42:51.091462+0000) 2024-03-26T06:43:10.286 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:09 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:09.927+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.592466+0000 front 2024-03-26T06:42:34.592536+0000 (oldest deadline 2024-03-26T06:43:00.492462+0000) 2024-03-26T06:43:10.286 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:09 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8[55822]: 2024-03-26T06:43:09.927+0000 7ff167363700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:43.993272+0000 front 2024-03-26T06:42:43.993329+0000 (oldest deadline 2024-03-26T06:43:09.793936+0000) 2024-03-26T06:43:10.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:10 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:10.237+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:10.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:10 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:10.237+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:10.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:10 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:10.237+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:10.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:10 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:10.237+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:10.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:10 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:10.541+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:10.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:10 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:10.541+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:10.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:10 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:10.541+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:10.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:10 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:10.541+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:10.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:10 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:10.541+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:11.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:10 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:10.865+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:11.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:10 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:10.865+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:11.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:10 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:10.865+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:11.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:10 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:10.865+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:11.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:10 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:10.865+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:11.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:10 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:10.865+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:11.480 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:11 smithi149 podman[77628]: 2024-03-26 06:43:11.174476131 +0000 UTC m=+5.686934744 container died 461ca5beb81ef7b7dd65d8a94cab893d81b63390e25c3742e0cc8458c2fb3e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-26T06:43:11.481 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:11 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:11.231+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:11.481 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:11 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:11.231+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:11.481 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:11 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:11.231+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:11.481 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:11 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:11.231+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:11.807 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:11 smithi149 podman[77628]: 2024-03-26 06:43:11.587039659 +0000 UTC m=+6.099498240 container cleanup 461ca5beb81ef7b7dd65d8a94cab893d81b63390e25c3742e0cc8458c2fb3e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-26T06:43:11.808 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:11 smithi149 bash[77628]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8 2024-03-26T06:43:11.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:11 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:11.587+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:11.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:11 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:11.587+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:11.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:11 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:11.587+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:11.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:11 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:11.587+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:11.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:11 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:11.587+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:12.107 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:11 smithi149 podman[77646]: 2024-03-26 06:43:11.804954048 +0000 UTC m=+0.621772159 container remove 461ca5beb81ef7b7dd65d8a94cab893d81b63390e25c3742e0cc8458c2fb3e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-26T06:43:12.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:11 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:11.887+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:12.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:11 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:11.887+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:12.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:11 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:11.887+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:12.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:11 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:11.887+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:12.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:11 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:11.887+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:12.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:11 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:11.887+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:12.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:12 smithi149 podman[77737]: 2024-03-26 06:43:12.105514773 +0000 UTC m=+0.093934857 container create 25df5ac17bf9641cae64b3009cbb98c162e31a1bfcac02a2916fe19e21f69604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-26T06:43:12.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:12 smithi149 podman[77737]: 2024-03-26 06:43:12.029542803 +0000 UTC m=+0.017962901 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:43:12.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:12 smithi149 podman[77737]: 2024-03-26 06:43:12.350922613 +0000 UTC m=+0.339342702 container init 25df5ac17bf9641cae64b3009cbb98c162e31a1bfcac02a2916fe19e21f69604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-26T06:43:12.412 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:12 smithi149 podman[77737]: 2024-03-26 06:43:12.354320446 +0000 UTC m=+0.342740538 container start 25df5ac17bf9641cae64b3009cbb98c162e31a1bfcac02a2916fe19e21f69604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-26T06:43:12.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:12 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:12.241+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:12.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:12 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:12.241+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:12.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:12 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:12.241+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:12.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:12 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:12.241+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:12.912 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:12 smithi149 podman[77737]: 2024-03-26 06:43:12.416960656 +0000 UTC m=+0.405380753 container attach 25df5ac17bf9641cae64b3009cbb98c162e31a1bfcac02a2916fe19e21f69604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:43:12.912 INFO:journalctl@ceph.osd.8.smithi149.stdout:Mar 26 06:43:12 smithi149 podman[77737]: 2024-03-26 06:43:12.641954743 +0000 UTC m=+0.630374826 container died 25df5ac17bf9641cae64b3009cbb98c162e31a1bfcac02a2916fe19e21f69604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-26T06:43:12.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:12 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:12.598+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:12.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:12 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:12.598+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:12.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:12 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:12.598+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:12.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:12 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:12.598+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:12.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:12 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:12.598+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:13.254 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.8.service' 2024-03-26T06:43:13.336 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:43:13.336 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-26T06:43:13.336 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-26T06:43:13.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.9 2024-03-26T06:43:13.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:12 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:12.933+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:13.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:12 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:12.933+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:13.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:12 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:12.933+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:13.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:12 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:12.933+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:13.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:12 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:12.933+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:13.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:12 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:12.933+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:13.508 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:13 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:13.222+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:13.508 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:13 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:13.222+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:13.508 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:13 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:13.222+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:13.508 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:13 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:13.222+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:13.508 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:13 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:13.222+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:13.637 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:13 smithi037 systemd[1]: Stopping Ceph osd.9 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:43:13.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:13.634+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:13.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:13.634+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:13.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:13.634+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:13.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:13.634+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:13.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:13.634+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:13.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:13.640+0000 7f53062c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:43:13.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:13.640+0000 7f53062c5700 -1 osd.9 119 *** Got signal Terminated *** 2024-03-26T06:43:13.952 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:13 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:13.640+0000 7f53062c5700 -1 osd.9 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:43:14.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:13 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:13.886+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:14.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:13 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:13.886+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:14.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:13 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:13.886+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:14.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:13 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:13.886+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:14.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:13 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:13.886+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:14.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:13 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:13.886+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:14.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:14 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:14.259+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:14.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:14 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:14.259+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:14.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:14 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:14.259+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:14.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:14 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:14.259+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:14.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:14 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:14.259+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:14.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:14 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:14.623+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:14.953 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:14 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:14.623+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:14.954 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:14 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:14.623+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:14.954 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:14 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:14.623+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:14.954 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:14 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:14.623+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:15.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:14 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:14.876+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:15.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:14 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:14.876+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:15.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:14 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:14.876+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:15.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:14 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:14.876+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:15.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:14 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:14.876+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:15.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:14 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:14.876+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:15.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:15 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:15.281+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:15.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:15 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:15.281+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:15.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:15 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:15.281+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:15.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:15 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:15.281+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:15.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:15 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:15.281+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:15.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:15 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:15.662+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:15.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:15 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:15.662+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:15.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:15 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:15.662+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:15.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:15 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:15.662+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:15.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:15 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:15.662+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:16.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:15 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:15.906+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:16.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:15 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:15.906+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:16.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:15 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:15.906+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:16.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:15 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:15.906+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:16.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:15 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:15.906+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:16.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:15 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:15.906+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:16.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:16 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:16.286+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:16.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:16 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:16.286+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:16.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:16 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:16.286+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:16.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:16 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:16.286+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:16.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:16 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:16.286+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:16.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:16 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:16.694+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:16.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:16 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:16.694+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:16.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:16 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:16.694+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:16.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:16 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:16.694+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:16.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:16 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:16.694+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:17.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:16 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:16.896+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:17.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:16 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:16.896+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:17.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:16 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:16.896+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:17.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:16 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:16.896+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:17.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:16 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:16.896+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:17.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:16 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:16.896+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:17.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:17 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:17.305+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:17.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:17 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:17.305+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:17.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:17 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:17.305+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:17.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:17 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:17.305+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:17.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:17 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:17.305+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:17.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:17 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:17.305+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:17.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:17 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:17.680+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:03.612828+0000 front 2024-03-26T06:42:03.613309+0000 (oldest deadline 2024-03-26T06:42:31.213667+0000) 2024-03-26T06:43:17.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:17 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:17.680+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:11.214384+0000 front 2024-03-26T06:42:11.214277+0000 (oldest deadline 2024-03-26T06:42:34.714473+0000) 2024-03-26T06:43:17.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:17 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:17.680+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:21.716748+0000 front 2024-03-26T06:42:21.716637+0000 (oldest deadline 2024-03-26T06:42:45.816759+0000) 2024-03-26T06:43:17.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:17 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:17.680+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:29.917720+0000 front 2024-03-26T06:42:29.917776+0000 (oldest deadline 2024-03-26T06:42:55.217947+0000) 2024-03-26T06:43:17.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:17 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:17.680+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.019485+0000 front 2024-03-26T06:42:38.019509+0000 (oldest deadline 2024-03-26T06:43:02.719559+0000) 2024-03-26T06:43:17.951 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:17 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9[67386]: 2024-03-26T06:43:17.680+0000 7f52ff0b2700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:54.421831+0000 front 2024-03-26T06:42:54.421857+0000 (oldest deadline 2024-03-26T06:43:16.722078+0000) 2024-03-26T06:43:18.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:17 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:17.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:18.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:17 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:17.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:18.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:17 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:17.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:18.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:17 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:17.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:18.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:17 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:17.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:18.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:17 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:17.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:18.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:18 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:18.312+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:18.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:18 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:18.312+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:18.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:18 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:18.312+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:18.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:18 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:18.312+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:18.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:18 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:18.312+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:18.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:18 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:18.312+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:19.302 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:19 smithi037 podman[96907]: 2024-03-26 06:43:19.030487205 +0000 UTC m=+5.496824175 container died 2c3cdf08385a508cc3355608ad30b6622df79bdacd2e2dacbb30b017097b767a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-26T06:43:19.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:18 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:18.965+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:19.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:18 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:18.965+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:19.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:18 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:18.965+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:19.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:18 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:18.965+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:19.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:18 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:18.965+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:19.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:18 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:18.965+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:19.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:18 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:18.965+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:53.975711+0000 front 2024-03-26T06:42:53.975721+0000 (oldest deadline 2024-03-26T06:43:18.675587+0000) 2024-03-26T06:43:19.652 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:19 smithi037 podman[96907]: 2024-03-26 06:43:19.399246897 +0000 UTC m=+5.865583858 container cleanup 2c3cdf08385a508cc3355608ad30b6622df79bdacd2e2dacbb30b017097b767a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-26T06:43:19.653 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:19 smithi037 bash[96907]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9 2024-03-26T06:43:19.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:19 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:19.333+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:19.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:19 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:19.333+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:19.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:19 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:19.333+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:19.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:19 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:19.333+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:19.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:19 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:19.333+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:19.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:19 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:19.333+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:19.950 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:19 smithi037 podman[96924]: 2024-03-26 06:43:19.650481482 +0000 UTC m=+0.621879484 container remove 2c3cdf08385a508cc3355608ad30b6622df79bdacd2e2dacbb30b017097b767a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:43:20.331 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:20 smithi037 podman[97012]: 2024-03-26 06:43:19.913702606 +0000 UTC m=+0.017591702 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:43:20.331 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:20 smithi037 podman[97012]: 2024-03-26 06:43:20.026254898 +0000 UTC m=+0.130143979 container create 30f444f8b4f8029064147b3ff8f8953d3f39aa153d27386b92c0596e5d3ecd07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-03-26T06:43:20.331 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:20 smithi037 podman[97012]: 2024-03-26 06:43:20.237074098 +0000 UTC m=+0.340963179 container init 30f444f8b4f8029064147b3ff8f8953d3f39aa153d27386b92c0596e5d3ecd07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:43:20.331 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:20 smithi037 podman[97012]: 2024-03-26 06:43:20.240268091 +0000 UTC m=+0.344157172 container start 30f444f8b4f8029064147b3ff8f8953d3f39aa153d27386b92c0596e5d3ecd07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:43:20.331 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:20 smithi037 podman[97012]: 2024-03-26 06:43:20.328541363 +0000 UTC m=+0.432430447 container attach 30f444f8b4f8029064147b3ff8f8953d3f39aa153d27386b92c0596e5d3ecd07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-26T06:43:20.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:19 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:19.952+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:20.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:19 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:19.952+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:20.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:19 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:19.952+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:20.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:19 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:19.952+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:20.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:19 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:19.952+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:20.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:19 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:19.952+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:20.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:19 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:19.952+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:53.975711+0000 front 2024-03-26T06:42:53.975721+0000 (oldest deadline 2024-03-26T06:43:18.675587+0000) 2024-03-26T06:43:20.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:20 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:20.289+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:20.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:20 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:20.289+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:20.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:20 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:20.289+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:20.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:20 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:20.289+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:20.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:20 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:20.289+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:20.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:20 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:20.289+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:20.701 INFO:journalctl@ceph.osd.9.smithi037.stdout:Mar 26 06:43:20 smithi037 podman[97012]: 2024-03-26 06:43:20.539088396 +0000 UTC m=+0.642977494 container died 30f444f8b4f8029064147b3ff8f8953d3f39aa153d27386b92c0596e5d3ecd07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-9-deactivate, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-26T06:43:21.128 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.9.service' 2024-03-26T06:43:21.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:43:21.173 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-26T06:43:21.173 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-26T06:43:21.173 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.10 2024-03-26T06:43:21.234 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:20 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:20.969+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:21.234 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:20 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:20.969+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:21.234 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:20 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:20.969+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:21.234 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:20 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:20.969+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:21.234 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:20 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:20.969+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:21.234 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:20 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:20.969+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:21.234 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:20 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:20.969+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:53.975711+0000 front 2024-03-26T06:42:53.975721+0000 (oldest deadline 2024-03-26T06:43:18.675587+0000) 2024-03-26T06:43:21.525 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:21 smithi066 systemd[1]: Stopping Ceph osd.10 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:43:21.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:21 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:21.324+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:21.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:21 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:21.324+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:21.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:21 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:21.324+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:21.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:21 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:21.324+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:21.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:21 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:21.324+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:21.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:21 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:21.324+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:21.849 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:21 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:21.522+0000 7ff130168700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:43:21.849 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:21 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:21.522+0000 7ff130168700 -1 osd.10 119 *** Got signal Terminated *** 2024-03-26T06:43:21.849 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:21 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:21.522+0000 7ff130168700 -1 osd.10 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:43:22.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.009+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:22.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.009+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:22.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.009+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:22.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.009+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:22.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.009+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:22.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.009+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:22.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.009+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:53.975711+0000 front 2024-03-26T06:42:53.975721+0000 (oldest deadline 2024-03-26T06:43:18.675587+0000) 2024-03-26T06:43:22.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:22 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:22.306+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:22.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:22 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:22.306+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:22.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:22 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:22.306+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:22.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:22 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:22.306+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:22.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:22 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:22.306+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:22.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:22 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:22.306+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:23.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.987+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:23.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.987+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:23.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.987+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:23.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.987+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:23.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.987+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:23.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.987+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:23.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:22 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:22.987+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:53.975711+0000 front 2024-03-26T06:42:53.975721+0000 (oldest deadline 2024-03-26T06:43:18.675587+0000) 2024-03-26T06:43:23.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:23 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:23.293+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:23.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:23 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:23.293+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:23.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:23 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:23.293+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:23.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:23 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:23.293+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:23.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:23 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:23.293+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:23.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:23 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:23.293+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:24.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.015+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:24.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.015+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:24.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.015+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:24.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.015+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:24.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.015+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:24.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.015+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:24.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.015+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:53.975711+0000 front 2024-03-26T06:42:53.975721+0000 (oldest deadline 2024-03-26T06:43:18.675587+0000) 2024-03-26T06:43:24.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:24 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:24.316+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:24.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:24 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:24.316+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:24.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:24 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:24.316+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:24.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:24 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:24.316+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:24.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:24 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:24.316+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:24.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:24 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:24.316+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:25.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.980+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:25.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.980+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:25.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.980+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:25.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.980+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:25.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.980+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:25.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.980+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:25.351 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:24 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:24.980+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:53.975711+0000 front 2024-03-26T06:42:53.975721+0000 (oldest deadline 2024-03-26T06:43:18.675587+0000) 2024-03-26T06:43:25.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:25 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:25.315+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:25.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:25 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:25.315+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:25.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:25 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:25.315+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:25.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:25 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:25.315+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:25.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:25 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:25.315+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:25.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:25 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:25.315+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:26.349 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:25 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:25.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:05.666498+0000 front 2024-03-26T06:42:05.666457+0000 (oldest deadline 2024-03-26T06:42:28.566474+0000) 2024-03-26T06:43:26.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:25 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:25.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:08.567060+0000 front 2024-03-26T06:42:08.566972+0000 (oldest deadline 2024-03-26T06:42:34.467148+0000) 2024-03-26T06:43:26.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:25 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:25.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:19.069312+0000 front 2024-03-26T06:42:19.069259+0000 (oldest deadline 2024-03-26T06:42:43.169415+0000) 2024-03-26T06:43:26.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:25 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:25.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-03-26T06:42:30.171355+0000 front 2024-03-26T06:42:30.171448+0000 (oldest deadline 2024-03-26T06:42:51.271488+0000) 2024-03-26T06:43:26.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:25 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:25.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:34.172597+0000 front 2024-03-26T06:42:34.172621+0000 (oldest deadline 2024-03-26T06:43:00.072759+0000) 2024-03-26T06:43:26.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:25 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:25.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:45.274384+0000 front 2024-03-26T06:42:45.274249+0000 (oldest deadline 2024-03-26T06:43:10.474701+0000) 2024-03-26T06:43:26.350 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:25 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10[61835]: 2024-03-26T06:43:25.937+0000 7ff128f55700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:53.975711+0000 front 2024-03-26T06:42:53.975721+0000 (oldest deadline 2024-03-26T06:43:18.675587+0000) 2024-03-26T06:43:26.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:26 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:26.280+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:26.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:26 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:26.280+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:26.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:26 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:26.280+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:26.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:26 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:26.280+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:26.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:26 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:26.280+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:26.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:26 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:26.280+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:27.216 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:26 smithi066 podman[75369]: 2024-03-26 06:43:26.904189662 +0000 UTC m=+5.521604444 container died f82df426b87d0cda9d734f157b56ec9c9fe954e084514221b434762e0559bdb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-03-26T06:43:27.466 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:27 smithi066 podman[75369]: 2024-03-26 06:43:27.304672888 +0000 UTC m=+5.922087639 container cleanup f82df426b87d0cda9d734f157b56ec9c9fe954e084514221b434762e0559bdb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:43:27.467 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:27 smithi066 bash[75369]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10 2024-03-26T06:43:27.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:27 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:27.327+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:27.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:27 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:27.327+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:27.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:27 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:27.327+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:27.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:27 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:27.327+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:27.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:27 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:27.327+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:27.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:27 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:27.327+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:27.849 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:27 smithi066 podman[75388]: 2024-03-26 06:43:27.464178365 +0000 UTC m=+0.556387728 container remove f82df426b87d0cda9d734f157b56ec9c9fe954e084514221b434762e0559bdb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:43:28.187 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:27 smithi066 podman[75476]: 2024-03-26 06:43:27.77792009 +0000 UTC m=+0.018193778 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:43:28.187 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:27 smithi066 podman[75476]: 2024-03-26 06:43:27.890041886 +0000 UTC m=+0.130315553 container create e8b6099ef188865ffa6da325a682d57caaf58af17dd0f8ad731d0ad00adfc816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:43:28.187 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:28 smithi066 podman[75476]: 2024-03-26 06:43:28.096010514 +0000 UTC m=+0.336284187 container init e8b6099ef188865ffa6da325a682d57caaf58af17dd0f8ad731d0ad00adfc816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-26T06:43:28.187 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:28 smithi066 podman[75476]: 2024-03-26 06:43:28.100201523 +0000 UTC m=+0.340475199 container start e8b6099ef188865ffa6da325a682d57caaf58af17dd0f8ad731d0ad00adfc816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:43:28.187 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:28 smithi066 podman[75476]: 2024-03-26 06:43:28.184079542 +0000 UTC m=+0.424353237 container attach e8b6099ef188865ffa6da325a682d57caaf58af17dd0f8ad731d0ad00adfc816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-26T06:43:28.599 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:28 smithi066 podman[75476]: 2024-03-26 06:43:28.40093049 +0000 UTC m=+0.641204163 container died e8b6099ef188865ffa6da325a682d57caaf58af17dd0f8ad731d0ad00adfc816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:43:28.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:28 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:28.360+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:28.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:28 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:28.360+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:28.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:28 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:28.360+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:28.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:28 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:28.360+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:28.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:28 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:28.360+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:28.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:28 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:28.360+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:28.952 INFO:journalctl@ceph.osd.10.smithi066.stdout:Mar 26 06:43:28 smithi066 podman[75502]: 2024-03-26 06:43:28.937028244 +0000 UTC m=+0.526018032 container remove e8b6099ef188865ffa6da325a682d57caaf58af17dd0f8ad731d0ad00adfc816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.schema-version=1.0) 2024-03-26T06:43:28.958 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.10.service' 2024-03-26T06:43:29.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:43:29.003 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-26T06:43:29.003 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-26T06:43:29.004 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.11 2024-03-26T06:43:29.350 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:29 smithi149 systemd[1]: Stopping Ceph osd.11 for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:43:29.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:29.347+0000 7fa23eb39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:43:29.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:29.347+0000 7fa23eb39700 -1 osd.11 119 *** Got signal Terminated *** 2024-03-26T06:43:29.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:29.347+0000 7fa23eb39700 -1 osd.11 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T06:43:29.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:29.398+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:29.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:29.398+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:29.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:29.398+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:29.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:29.398+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:29.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:29.398+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:29.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:29.398+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:29.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:29 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:29.398+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-03-26T06:42:58.658152+0000 front 2024-03-26T06:42:58.658135+0000 (oldest deadline 2024-03-26T06:43:28.658417+0000) 2024-03-26T06:43:30.912 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:30 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:30.434+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:30.912 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:30 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:30.434+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:30.913 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:30 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:30.434+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:30.913 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:30 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:30.434+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:30.913 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:30 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:30.434+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:30.913 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:30 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:30.434+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:30.913 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:30 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:30.434+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-03-26T06:42:58.658152+0000 front 2024-03-26T06:42:58.658135+0000 (oldest deadline 2024-03-26T06:43:28.658417+0000) 2024-03-26T06:43:31.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:31 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:31.389+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:31.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:31 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:31.389+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:31.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:31 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:31.389+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:31.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:31 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:31.389+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:31.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:31 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:31.389+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:31.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:31 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:31.389+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:31.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:31 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:31.389+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-03-26T06:42:58.658152+0000 front 2024-03-26T06:42:58.658135+0000 (oldest deadline 2024-03-26T06:43:28.658417+0000) 2024-03-26T06:43:32.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:32 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:32.399+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:32.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:32 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:32.399+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:32.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:32 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:32.399+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:32.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:32 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:32.399+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:32.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:32 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:32.399+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:32.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:32 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:32.399+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:32.663 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:32 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:32.399+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-03-26T06:42:58.658152+0000 front 2024-03-26T06:42:58.658135+0000 (oldest deadline 2024-03-26T06:43:28.658417+0000) 2024-03-26T06:43:33.912 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:33 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:33.409+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-26T06:42:02.847335+0000 front 2024-03-26T06:42:02.847239+0000 (oldest deadline 2024-03-26T06:42:26.347314+0000) 2024-03-26T06:43:33.912 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:33 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:33.409+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-03-26T06:42:10.949132+0000 front 2024-03-26T06:42:10.949092+0000 (oldest deadline 2024-03-26T06:42:35.649187+0000) 2024-03-26T06:43:33.912 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:33 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:33.409+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-03-26T06:42:20.350170+0000 front 2024-03-26T06:42:20.350132+0000 (oldest deadline 2024-03-26T06:42:44.350830+0000) 2024-03-26T06:43:33.912 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:33 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:33.409+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-03-26T06:42:38.254450+0000 front 2024-03-26T06:42:38.254477+0000 (oldest deadline 2024-03-26T06:43:02.954524+0000) 2024-03-26T06:43:33.913 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:33 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:33.409+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-03-26T06:42:44.655687+0000 front 2024-03-26T06:42:44.655661+0000 (oldest deadline 2024-03-26T06:43:12.256336+0000) 2024-03-26T06:43:33.913 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:33 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:33.409+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-03-26T06:42:52.256989+0000 front 2024-03-26T06:42:52.257038+0000 (oldest deadline 2024-03-26T06:43:16.956923+0000) 2024-03-26T06:43:33.913 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:33 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11[62438]: 2024-03-26T06:43:33.409+0000 7fa23b92e700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-03-26T06:42:58.658152+0000 front 2024-03-26T06:42:58.658135+0000 (oldest deadline 2024-03-26T06:43:28.658417+0000) 2024-03-26T06:43:35.050 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:34 smithi149 podman[77855]: 2024-03-26 06:43:34.72524159 +0000 UTC m=+5.525031719 container died 682658acdb00444f16a03c3a86c87533f52fcc40683d59d9bb6b050e0b174321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:43:35.367 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:35 smithi149 podman[77855]: 2024-03-26 06:43:35.121799723 +0000 UTC m=+5.921589855 container cleanup 682658acdb00444f16a03c3a86c87533f52fcc40683d59d9bb6b050e0b174321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-26T06:43:35.367 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:35 smithi149 bash[77855]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11 2024-03-26T06:43:35.662 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:35 smithi149 podman[77874]: 2024-03-26 06:43:35.364493047 +0000 UTC m=+0.630917245 container remove 682658acdb00444f16a03c3a86c87533f52fcc40683d59d9bb6b050e0b174321 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-26T06:43:35.962 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:35 smithi149 podman[77967]: 2024-03-26 06:43:35.579645528 +0000 UTC m=+0.017597194 image pull 10cdb6adb6155065d2fa5242628814354d43782bed55cc4f028ed36bb9d0a00c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265 2024-03-26T06:43:35.962 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:35 smithi149 podman[77967]: 2024-03-26 06:43:35.682077652 +0000 UTC m=+0.120029321 container create 5c33d2e6365351f5872f584b9fefed2607c886ac82fde822cd9e6b2fe877995d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-26T06:43:35.962 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:35 smithi149 podman[77967]: 2024-03-26 06:43:35.884283526 +0000 UTC m=+0.322235181 container init 5c33d2e6365351f5872f584b9fefed2607c886ac82fde822cd9e6b2fe877995d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11-deactivate, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-26T06:43:35.962 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:35 smithi149 podman[77967]: 2024-03-26 06:43:35.887422123 +0000 UTC m=+0.325373776 container start 5c33d2e6365351f5872f584b9fefed2607c886ac82fde822cd9e6b2fe877995d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:43:35.963 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:35 smithi149 podman[77967]: 2024-03-26 06:43:35.95913841 +0000 UTC m=+0.397090064 container attach 5c33d2e6365351f5872f584b9fefed2607c886ac82fde822cd9e6b2fe877995d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-26T06:43:36.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Mar 26 06:43:36 smithi149 podman[77967]: 2024-03-26 06:43:36.181578605 +0000 UTC m=+0.619530254 container died 5c33d2e6365351f5872f584b9fefed2607c886ac82fde822cd9e6b2fe877995d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T06:43:36.782 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@osd.11.service' 2024-03-26T06:43:36.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:43:36.823 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-26T06:43:36.823 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-26T06:43:36.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.a 2024-03-26T06:43:37.129 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:43:36 smithi037 systemd[1]: Stopping Ceph mds.a for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:43:37.450 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:43:37 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a[74242]: 2024-03-26T06:43:37.125+0000 7fc04ba96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:43:37.451 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:43:37 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a[74242]: 2024-03-26T06:43:37.125+0000 7fc04ba96700 -1 mds.a *** got signal Terminated *** 2024-03-26T06:43:41.412 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:43:41 smithi037 podman[97129]: 2024-03-26 06:43:41.13662695 +0000 UTC m=+4.118640984 container died 9e8f8e1ed77c37325bfca120a9dd4cbf6508b2c2699a8fa63fe47f8e3b0ec68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:43:41.701 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:43:41 smithi037 podman[97129]: 2024-03-26 06:43:41.50899148 +0000 UTC m=+4.491005504 container cleanup 9e8f8e1ed77c37325bfca120a9dd4cbf6508b2c2699a8fa63fe47f8e3b0ec68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:43:41.701 INFO:journalctl@ceph.mds.a.smithi037.stdout:Mar 26 06:43:41 smithi037 bash[97129]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-a 2024-03-26T06:43:41.935 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.a.service' 2024-03-26T06:43:41.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:43:41.971 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-26T06:43:41.972 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-26T06:43:41.972 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.d 2024-03-26T06:43:42.200 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:43:42 smithi037 systemd[1]: Stopping Ceph mds.d for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:43:42.700 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:43:42 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-d[73274]: 2024-03-26T06:43:42.287+0000 7f0b14af8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:43:42.701 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:43:42 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-d[73274]: 2024-03-26T06:43:42.287+0000 7f0b14af8700 -1 mds.d *** got signal Terminated *** 2024-03-26T06:43:43.758 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:43:43 smithi037 podman[97251]: 2024-03-26 06:43:43.4851144 +0000 UTC m=+1.299161535 container died 8b6b55fa840ce4af875c991aabfff34f45c43274e3b42d800d56a3339409634a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-26T06:43:44.099 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:43:43 smithi037 podman[97251]: 2024-03-26 06:43:43.838184125 +0000 UTC m=+1.652231249 container cleanup 8b6b55fa840ce4af875c991aabfff34f45c43274e3b42d800d56a3339409634a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:43:44.100 INFO:journalctl@ceph.mds.d.smithi037.stdout:Mar 26 06:43:43 smithi037 bash[97251]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-d 2024-03-26T06:43:44.275 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.d.service' 2024-03-26T06:43:44.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:43:44.309 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-26T06:43:44.309 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-26T06:43:44.310 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.g 2024-03-26T06:43:44.636 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:43:44 smithi037 systemd[1]: Stopping Ceph mds.g for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:43:44.950 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:43:44 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g[73877]: 2024-03-26T06:43:44.633+0000 7f8430755700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:43:44.951 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:43:44 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g[73877]: 2024-03-26T06:43:44.633+0000 7f8430755700 -1 mds.g *** got signal Terminated *** 2024-03-26T06:43:48.920 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:43:48 smithi037 podman[97371]: 2024-03-26 06:43:48.643301676 +0000 UTC m=+4.116995220 container died 3eccb2b6cbd6d3765d67f62b867f5c6c4ec30e2263dfca49a2efe842971d916c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS) 2024-03-26T06:43:49.200 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:43:49 smithi037 podman[97371]: 2024-03-26 06:43:49.033514347 +0000 UTC m=+4.507207898 container cleanup 3eccb2b6cbd6d3765d67f62b867f5c6c4ec30e2263dfca49a2efe842971d916c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-26T06:43:49.201 INFO:journalctl@ceph.mds.g.smithi037.stdout:Mar 26 06:43:49 smithi037 bash[97371]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-g 2024-03-26T06:43:49.413 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.g.service' 2024-03-26T06:43:49.446 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:43:49.447 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-26T06:43:49.447 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-26T06:43:49.447 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.j 2024-03-26T06:43:49.700 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:43:49 smithi037 systemd[1]: Stopping Ceph mds.j for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:43:50.200 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:43:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-j[72921]: 2024-03-26T06:43:49.795+0000 7f7555211700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:43:50.201 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:43:49 smithi037 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-j[72921]: 2024-03-26T06:43:49.795+0000 7f7555211700 -1 mds.j *** got signal Terminated *** 2024-03-26T06:43:52.700 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:43:52 smithi037 podman[97492]: 2024-03-26 06:43:52.445557767 +0000 UTC m=+2.784994673 container died 7cd9991f93c1772a92e8d0b7a8a5abb803ef82c581e4d1e504910abcb7b2aa0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-03-26T06:43:53.182 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:43:52 smithi037 podman[97492]: 2024-03-26 06:43:52.912489241 +0000 UTC m=+3.251926131 container cleanup 7cd9991f93c1772a92e8d0b7a8a5abb803ef82c581e4d1e504910abcb7b2aa0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-j, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-26T06:43:53.182 INFO:journalctl@ceph.mds.j.smithi037.stdout:Mar 26 06:43:52 smithi037 bash[97492]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-j 2024-03-26T06:43:53.347 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.j.service' 2024-03-26T06:43:53.381 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:43:53.381 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-26T06:43:53.381 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-26T06:43:53.382 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.b 2024-03-26T06:43:53.677 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:43:53 smithi066 systemd[1]: Stopping Ceph mds.b for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:43:53.949 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:43:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b[67275]: 2024-03-26T06:43:53.696+0000 7fcd82b08700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:43:53.949 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:43:53 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b[67275]: 2024-03-26T06:43:53.696+0000 7fcd82b08700 -1 mds.b *** got signal Terminated *** 2024-03-26T06:43:57.809 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:43:57 smithi066 podman[75582]: 2024-03-26 06:43:57.503218753 +0000 UTC m=+3.917807865 container died 442f7c8a9335ae8c712d1665c787d179c7d85d15abb7d917e122673501a49588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-26T06:43:58.099 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:43:57 smithi066 podman[75582]: 2024-03-26 06:43:57.904366063 +0000 UTC m=+4.318955154 container cleanup 442f7c8a9335ae8c712d1665c787d179c7d85d15abb7d917e122673501a49588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f) 2024-03-26T06:43:58.100 INFO:journalctl@ceph.mds.b.smithi066.stdout:Mar 26 06:43:57 smithi066 bash[75582]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-b 2024-03-26T06:43:58.261 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.b.service' 2024-03-26T06:43:58.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:43:58.304 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-26T06:43:58.304 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-26T06:43:58.304 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.e 2024-03-26T06:43:58.599 INFO:journalctl@ceph.mds.e.smithi066.stdout:Mar 26 06:43:58 smithi066 systemd[1]: Stopping Ceph mds.e for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:43:59.099 INFO:journalctl@ceph.mds.e.smithi066.stdout:Mar 26 06:43:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-e[66664]: 2024-03-26T06:43:58.633+0000 7f0c4f990700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:43:59.100 INFO:journalctl@ceph.mds.e.smithi066.stdout:Mar 26 06:43:58 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-e[66664]: 2024-03-26T06:43:58.633+0000 7f0c4f990700 -1 mds.e *** got signal Terminated *** 2024-03-26T06:44:01.766 INFO:journalctl@ceph.mds.e.smithi066.stdout:Mar 26 06:44:01 smithi066 podman[75716]: 2024-03-26 06:44:01.5011544 +0000 UTC m=+2.976697766 container died cda5fce98f2f98d0635581671bb06b85ff07e88b27b20272203e9a90d86f0eef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-e, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-26T06:44:02.100 INFO:journalctl@ceph.mds.e.smithi066.stdout:Mar 26 06:44:01 smithi066 podman[75716]: 2024-03-26 06:44:01.866205543 +0000 UTC m=+3.341748903 container cleanup cda5fce98f2f98d0635581671bb06b85ff07e88b27b20272203e9a90d86f0eef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-26T06:44:02.100 INFO:journalctl@ceph.mds.e.smithi066.stdout:Mar 26 06:44:01 smithi066 bash[75716]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-e 2024-03-26T06:44:02.300 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.e.service' 2024-03-26T06:44:02.340 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:44:02.340 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-26T06:44:02.340 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-26T06:44:02.341 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.h 2024-03-26T06:44:02.665 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:44:02 smithi066 systemd[1]: Stopping Ceph mds.h for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:44:03.099 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:44:02 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h[66971]: 2024-03-26T06:44:02.662+0000 7fdbdc0f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:44:03.100 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:44:02 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h[66971]: 2024-03-26T06:44:02.662+0000 7fdbdc0f1700 -1 mds.h *** got signal Terminated *** 2024-03-26T06:44:05.764 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:44:05 smithi066 podman[75836]: 2024-03-26 06:44:05.488584674 +0000 UTC m=+2.923220480 container died 47ef9a70fe8e340f9fa42c8a946a67ff157188875eb53e1a96ed18ae2f850784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:44:05.764 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:44:05 smithi066 podman[75836]: 2024-03-26 06:44:05.761654682 +0000 UTC m=+3.196290468 container cleanup 47ef9a70fe8e340f9fa42c8a946a67ff157188875eb53e1a96ed18ae2f850784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T06:44:06.099 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:44:05 smithi066 bash[75836]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h 2024-03-26T06:44:06.100 INFO:journalctl@ceph.mds.h.smithi066.stdout:Mar 26 06:44:05 smithi066 podman[75850]: 2024-03-26 06:44:05.996288741 +0000 UTC m=+0.507088184 container remove 47ef9a70fe8e340f9fa42c8a946a67ff157188875eb53e1a96ed18ae2f850784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-h, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2) 2024-03-26T06:44:06.144 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.h.service' 2024-03-26T06:44:06.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:44:06.183 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-26T06:44:06.183 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-26T06:44:06.183 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.k 2024-03-26T06:44:06.509 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:44:06 smithi066 systemd[1]: Stopping Ceph mds.k for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:44:06.849 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:44:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k[66341]: 2024-03-26T06:44:06.506+0000 7f59188a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:44:06.850 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:44:06 smithi066 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k[66341]: 2024-03-26T06:44:06.506+0000 7f59188a8700 -1 mds.k *** got signal Terminated *** 2024-03-26T06:44:09.769 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:44:09 smithi066 podman[75957]: 2024-03-26 06:44:09.481858257 +0000 UTC m=+3.071952630 container died 5ed6198eac4d0f093c4f2070be6b3c9facc2539e39332f43f731473b64a8123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T06:44:10.100 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:44:09 smithi066 podman[75957]: 2024-03-26 06:44:09.848920767 +0000 UTC m=+3.439015125 container cleanup 5ed6198eac4d0f093c4f2070be6b3c9facc2539e39332f43f731473b64a8123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T06:44:10.100 INFO:journalctl@ceph.mds.k.smithi066.stdout:Mar 26 06:44:09 smithi066 bash[75957]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-k 2024-03-26T06:44:10.275 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.k.service' 2024-03-26T06:44:10.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:44:10.314 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-26T06:44:10.315 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-26T06:44:10.315 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.c 2024-03-26T06:44:10.615 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:44:10 smithi149 systemd[1]: Stopping Ceph mds.c for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:44:10.912 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:44:10 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: 2024-03-26T06:44:10.610+0000 7fd4aa161700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:44:10.912 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:44:10 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c[67757]: 2024-03-26T06:44:10.610+0000 7fd4aa161700 -1 mds.c *** got signal Terminated *** 2024-03-26T06:44:13.866 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:44:13 smithi149 podman[78083]: 2024-03-26 06:44:13.508855177 +0000 UTC m=+3.001884412 container died 3f6908e4e203ef9e0e9530a348fc1e72921995ba02acb6d0c221ae26ab53f572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-26T06:44:14.162 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:44:13 smithi149 podman[78083]: 2024-03-26 06:44:13.961266144 +0000 UTC m=+3.454295369 container cleanup 3f6908e4e203ef9e0e9530a348fc1e72921995ba02acb6d0c221ae26ab53f572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-26T06:44:14.162 INFO:journalctl@ceph.mds.c.smithi149.stdout:Mar 26 06:44:13 smithi149 bash[78083]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-c 2024-03-26T06:44:14.327 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.c.service' 2024-03-26T06:44:14.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:44:14.367 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-26T06:44:14.367 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-26T06:44:14.367 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.f 2024-03-26T06:44:14.662 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:44:14 smithi149 systemd[1]: Stopping Ceph mds.f for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:44:15.162 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:44:14 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f[67449]: 2024-03-26T06:44:14.689+0000 7fda8aa5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:44:15.162 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:44:14 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f[67449]: 2024-03-26T06:44:14.689+0000 7fda8aa5f700 -1 mds.f *** got signal Terminated *** 2024-03-26T06:44:17.871 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:44:17 smithi149 podman[78216]: 2024-03-26 06:44:17.515518298 +0000 UTC m=+2.937681265 container died ad3a23bcf9d8f60343570f699f5daf2e02d3418d8b0ff34fc5cedccff135bc89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-26T06:44:18.135 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:44:17 smithi149 podman[78216]: 2024-03-26 06:44:17.964764971 +0000 UTC m=+3.386927936 container cleanup ad3a23bcf9d8f60343570f699f5daf2e02d3418d8b0ff34fc5cedccff135bc89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-26T06:44:18.135 INFO:journalctl@ceph.mds.f.smithi149.stdout:Mar 26 06:44:17 smithi149 bash[78216]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-f 2024-03-26T06:44:18.355 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.f.service' 2024-03-26T06:44:18.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:44:18.392 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-26T06:44:18.393 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-26T06:44:18.393 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.i 2024-03-26T06:44:18.662 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:44:18 smithi149 systemd[1]: Stopping Ceph mds.i for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:44:19.162 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:44:18 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i[68066]: 2024-03-26T06:44:18.709+0000 7f4c1d4fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:44:19.162 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:44:18 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i[68066]: 2024-03-26T06:44:18.709+0000 7f4c1d4fd700 -1 mds.i *** got signal Terminated *** 2024-03-26T06:44:21.912 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:44:21 smithi149 podman[78336]: 2024-03-26 06:44:21.508181323 +0000 UTC m=+2.895970272 container died 4f27664c4382a5d65423952cd0aabb92e36154c790e77ac83b105e74e6d045f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-26T06:44:22.319 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:44:22 smithi149 podman[78336]: 2024-03-26 06:44:22.126450231 +0000 UTC m=+3.514239177 container cleanup 4f27664c4382a5d65423952cd0aabb92e36154c790e77ac83b105e74e6d045f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-2-7e98f5f, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-26T06:44:22.319 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:44:22 smithi149 bash[78336]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i 2024-03-26T06:44:22.564 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.i.service' 2024-03-26T06:44:22.588 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:44:22 smithi149 podman[78350]: 2024-03-26 06:44:22.410838004 +0000 UTC m=+0.647882112 container remove 4f27664c4382a5d65423952cd0aabb92e36154c790e77ac83b105e74e6d045f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-i, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-2-7e98f5f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-26T06:44:22.588 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:44:22 smithi149 systemd[1]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.i.service: Deactivated successfully. 2024-03-26T06:44:22.588 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:44:22 smithi149 systemd[1]: Stopped Ceph mds.i for a86fb218-eb37-11ee-95d0-87774f69a715. 2024-03-26T06:44:22.588 INFO:journalctl@ceph.mds.i.smithi149.stdout:Mar 26 06:44:22 smithi149 systemd[1]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.i.service: Consumed 6min 6.765s CPU time. 2024-03-26T06:44:22.600 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:44:22.600 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-26T06:44:22.600 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-26T06:44:22.601 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.l 2024-03-26T06:44:22.877 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:44:22 smithi149 systemd[1]: Stopping Ceph mds.l for a86fb218-eb37-11ee-95d0-87774f69a715... 2024-03-26T06:44:23.162 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:44:22 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l[67135]: 2024-03-26T06:44:22.921+0000 7fec2c92f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T06:44:23.162 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:44:22 smithi149 ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l[67135]: 2024-03-26T06:44:22.921+0000 7fec2c92f700 -1 mds.l *** got signal Terminated *** 2024-03-26T06:44:25.824 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:44:25 smithi149 podman[78459]: 2024-03-26 06:44:25.513967098 +0000 UTC m=+2.704458033 container died 2379c8bfc0c221b16b2b1ef287ba7cd929af06257051c29d05f0d28bda9d8efa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-26T06:44:25.824 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:44:25 smithi149 podman[78459]: 2024-03-26 06:44:25.821831497 +0000 UTC m=+3.012322426 container cleanup 2379c8bfc0c221b16b2b1ef287ba7cd929af06257051c29d05f0d28bda9d8efa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=rishabh-2024mar8-2-7e98f5f, ceph=True) 2024-03-26T06:44:26.162 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:44:25 smithi149 bash[78459]: ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l 2024-03-26T06:44:26.162 INFO:journalctl@ceph.mds.l.smithi149.stdout:Mar 26 06:44:26 smithi149 podman[78473]: 2024-03-26 06:44:26.02252612 +0000 UTC m=+0.509642278 container remove 2379c8bfc0c221b16b2b1ef287ba7cd929af06257051c29d05f0d28bda9d8efa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cd565c2faf180e3895b642f4837314e69c92d3272b87470dde98807c0b926265, name=ceph-a86fb218-eb37-11ee-95d0-87774f69a715-mds-l, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-2-7e98f5f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-26T06:44:26.241 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a86fb218-eb37-11ee-95d0-87774f69a715@mds.l.service' 2024-03-26T06:44:26.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T06:44:26.276 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-26T06:44:26.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a86fb218-eb37-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T06:44:26.472 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:44:28.946 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a86fb218-eb37-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T06:44:29.149 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:44:31.877 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a86fb218-eb37-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T06:44:32.063 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:44:34.555 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T06:44:34.591 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T06:44:34.631 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T06:44:34.660 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-26T06:44:34.663 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740/remote/smithi037/crash 2024-03-26T06:44:34.663 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/crash -- . 2024-03-26T06:44:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T06:44:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2024-03-26T06:44:34.710 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740/remote/smithi066/crash 2024-03-26T06:44:34.711 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/crash -- . 2024-03-26T06:44:34.755 INFO:teuthology.orchestra.run.smithi066.stderr:tar: /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T06:44:34.755 INFO:teuthology.orchestra.run.smithi066.stderr:tar: Error is not recoverable: exiting now 2024-03-26T06:44:34.757 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740/remote/smithi149/crash 2024-03-26T06:44:34.758 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/crash -- . 2024-03-26T06:44:34.800 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T06:44:34.800 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-03-26T06:44:34.801 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-26T06:44:34.801 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-26T06:44:34.837 INFO:tasks.cephadm:Compressing logs... 2024-03-26T06:44:34.837 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T06:44:34.879 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T06:44:34.882 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T06:44:34.907 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T06:44:34.908 INFO:teuthology.orchestra.run.smithi037.stderr:: No such file or directory 2024-03-26T06:44:34.908 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.6.log 2024-03-26T06:44:34.908 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mon.a.log 2024-03-26T06:44:34.908 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T06:44:34.908 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.6.log: /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.g.log 2024-03-26T06:44:34.909 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.j.log 2024-03-26T06:44:34.910 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-26T06:44:34.910 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mgr.x.log 2024-03-26T06:44:34.911 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-26T06:44:34.912 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mgr.x.log: 90.1% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T06:44:34.913 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T06:44:34.915 INFO:teuthology.orchestra.run.smithi037.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T06:44:34.915 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.j.log.gz 2024-03-26T06:44:34.925 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T06:44:34.925 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.log 2024-03-26T06:44:34.925 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T06:44:34.925 INFO:teuthology.orchestra.run.smithi037.stderr: 83.3% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T06:44:34.927 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T06:44:34.927 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.l.log 2024-03-26T06:44:34.927 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T06:44:34.927 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.i.log 2024-03-26T06:44:34.927 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.11.log 2024-03-26T06:44:34.927 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.audit.log: /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mgr.z.log 2024-03-26T06:44:34.927 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.8.log 2024-03-26T06:44:34.928 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-26T06:44:34.928 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mgr.z.log: /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.f.log 2024-03-26T06:44:34.928 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T06:44:34.928 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.log 2024-03-26T06:44:34.928 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-26T06:44:34.928 INFO:teuthology.orchestra.run.smithi149.stderr:gzip/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.log: -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.c.log 2024-03-26T06:44:34.928 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T06:44:34.928 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mon.c.log 2024-03-26T06:44:34.929 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.log: 86.9% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T06:44:34.929 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.9.log 2024-03-26T06:44:34.929 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.d.log 2024-03-26T06:44:34.930 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mon.c.log: 92.3% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mgr.z.log.gz 2024-03-26T06:44:34.935 INFO:teuthology.orchestra.run.smithi066.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T06:44:34.936 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T06:44:34.936 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.10.log 2024-03-26T06:44:34.937 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T06:44:34.937 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.b.log 2024-03-26T06:44:34.937 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-26T06:44:34.939 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-26T06:44:34.940 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.k.log 2024-03-26T06:44:34.940 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.h.log 2024-03-26T06:44:34.940 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.7.log 2024-03-26T06:44:34.940 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.e.log 2024-03-26T06:44:34.940 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.h.log: /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mgr.y.log 2024-03-26T06:44:34.940 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T06:44:34.940 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.log 2024-03-26T06:44:34.940 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T06:44:34.941 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mon.b.log 2024-03-26T06:44:34.941 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.a.log 2024-03-26T06:44:34.946 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.log: /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mgr.y.log.gz 2024-03-26T06:44:34.948 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.d.log.gz 2024-03-26T06:44:34.951 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T06:44:35.029 INFO:teuthology.orchestra.run.smithi149.stderr: 83.0% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T06:44:35.090 INFO:teuthology.orchestra.run.smithi066.stderr: 83.0% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T06:44:35.102 INFO:teuthology.orchestra.run.smithi149.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T06:44:35.107 INFO:teuthology.orchestra.run.smithi066.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T06:44:35.114 INFO:teuthology.orchestra.run.smithi149.stderr: 86.9% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T06:44:35.120 INFO:teuthology.orchestra.run.smithi149.stderr: 90.3% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T06:44:35.293 INFO:teuthology.orchestra.run.smithi066.stderr: 90.3% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T06:44:35.346 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.a.log: 89.9% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mgr.x.log.gz 2024-03-26T06:44:35.463 INFO:teuthology.orchestra.run.smithi149.stderr: 90.9% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T06:44:35.599 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T06:44:35.906 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T06:44:36.544 INFO:teuthology.orchestra.run.smithi037.stderr: 90.2% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mon.a.log.gz 2024-03-26T06:44:38.553 INFO:teuthology.orchestra.run.smithi149.stderr: 91.8% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mon.c.log.gz 2024-03-26T06:44:40.514 INFO:teuthology.orchestra.run.smithi037.stderr: 95.2% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.g.log.gz 2024-03-26T06:44:48.689 INFO:teuthology.orchestra.run.smithi037.stderr: 93.7% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-26T06:44:48.732 INFO:teuthology.orchestra.run.smithi037.stderr: 93.8% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.9.log.gz 2024-03-26T06:44:51.286 INFO:teuthology.orchestra.run.smithi037.stderr: 93.8% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.6.log.gz 2024-03-26T06:44:52.946 INFO:teuthology.orchestra.run.smithi066.stderr: 93.6% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mon.b.log.gz 2024-03-26T06:44:54.149 INFO:teuthology.orchestra.run.smithi037.stderr: 93.7% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-26T06:45:28.793 INFO:teuthology.orchestra.run.smithi149.stderr: 95.3% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.f.log.gz 2024-03-26T06:45:29.249 INFO:teuthology.orchestra.run.smithi066.stderr: 95.3% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.k.log.gz 2024-03-26T06:45:37.565 INFO:teuthology.orchestra.run.smithi149.stderr: 95.2% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.c.log.gz 2024-03-26T06:45:39.722 INFO:teuthology.orchestra.run.smithi149.stderr: 95.3% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.l.log.gz 2024-03-26T06:45:43.983 INFO:teuthology.orchestra.run.smithi066.stderr: 93.9% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-26T06:45:47.311 INFO:teuthology.orchestra.run.smithi066.stderr: 93.7% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-26T06:45:57.733 INFO:teuthology.orchestra.run.smithi066.stderr: 93.8% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.10.log.gz 2024-03-26T06:46:09.219 INFO:teuthology.orchestra.run.smithi149.stderr: 93.8% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.11.log.gz 2024-03-26T06:46:12.740 INFO:teuthology.orchestra.run.smithi149.stderr: 93.8% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-26T06:46:14.418 INFO:teuthology.orchestra.run.smithi149.stderr: 93.8% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.8.log.gz 2024-03-26T06:46:19.640 INFO:teuthology.orchestra.run.smithi037.stderr: 93.1% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.a.log.gz 2024-03-26T06:46:19.643 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-03-26T06:46:19.643 INFO:teuthology.orchestra.run.smithi037.stderr:real 1m44.749s 2024-03-26T06:46:19.643 INFO:teuthology.orchestra.run.smithi037.stderr:user 2m50.254s 2024-03-26T06:46:19.643 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m5.008s 2024-03-26T06:46:21.943 INFO:teuthology.orchestra.run.smithi066.stderr: 93.8% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.7.log.gz 2024-03-26T06:46:22.339 INFO:teuthology.orchestra.run.smithi149.stderr: 93.9% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-26T06:47:51.935 INFO:teuthology.orchestra.run.smithi149.stderr: 93.2% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.i.log.gz 2024-03-26T06:47:51.936 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-03-26T06:47:51.937 INFO:teuthology.orchestra.run.smithi149.stderr:real 3m17.039s 2024-03-26T06:47:51.937 INFO:teuthology.orchestra.run.smithi149.stderr:user 2m41.433s 2024-03-26T06:47:51.937 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m6.893s 2024-03-26T06:53:03.496 INFO:teuthology.orchestra.run.smithi066.stderr: 93.1% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.h.log.gz 2024-03-26T06:53:16.950 INFO:teuthology.orchestra.run.smithi066.stderr: 93.1% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.e.log.gz 2024-03-26T06:54:03.008 INFO:teuthology.orchestra.run.smithi066.stderr: 93.3% -- replaced with /var/log/ceph/a86fb218-eb37-11ee-95d0-87774f69a715/ceph-mds.b.log.gz 2024-03-26T06:54:03.056 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-03-26T06:54:03.057 INFO:teuthology.orchestra.run.smithi066.stderr:real 9m28.122s 2024-03-26T06:54:03.057 INFO:teuthology.orchestra.run.smithi066.stderr:user 6m16.956s 2024-03-26T06:54:03.057 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m14.589s 2024-03-26T06:54:03.058 INFO:tasks.cephadm:Archiving logs... 2024-03-26T06:54:03.059 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740/remote/smithi037/log 2024-03-26T06:54:03.060 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T06:54:27.217 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740/remote/smithi066/log 2024-03-26T06:54:27.222 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T06:55:23.133 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740/remote/smithi149/log 2024-03-26T06:55:23.135 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T06:55:49.103 INFO:tasks.cephadm:Removing cluster... 2024-03-26T06:55:49.104 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a86fb218-eb37-11ee-95d0-87774f69a715 --force 2024-03-26T06:55:49.304 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:55:49.758 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a86fb218-eb37-11ee-95d0-87774f69a715 --force 2024-03-26T06:55:50.090 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:55:50.742 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a86fb218-eb37-11ee-95d0-87774f69a715 --force 2024-03-26T06:55:51.074 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: a86fb218-eb37-11ee-95d0-87774f69a715 2024-03-26T06:55:51.502 INFO:tasks.cephadm:Removing cephadm ... 2024-03-26T06:55:51.503 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T06:55:51.521 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T06:55:51.540 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T06:55:51.556 INFO:tasks.cephadm:Teardown complete 2024-03-26T06:55:51.556 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-26T06:55:51.568 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-26T06:55:51.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T06:55:51.570 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T06:55:51.583 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T06:55:51.706 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-26T06:55:51.717 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-26T06:55:51.717 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T06:55:51.720 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T06:55:51.723 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T06:55:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-03-26T06:55:51.736 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-03-26T06:55:51.736 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-03-26T06:55:51.739 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T06:55:51.739 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-03-26T06:55:51.739 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 +885us[ +885us] +/- 75ms 2024-03-26T06:55:51.739 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 -1021us[ -999us] +/- 74ms 2024-03-26T06:55:51.739 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +200us[ +200us] +/- 81ms 2024-03-26T06:55:51.739 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:55:51.827 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T06:55:51.828 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-03-26T06:55:51.828 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 +905us[ +905us] +/- 75ms 2024-03-26T06:55:51.828 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv02.front.sepia.ceph.com 3 6 377 28 -1005us[ -977us] +/- 74ms 2024-03-26T06:55:51.828 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +212us[ +212us] +/- 81ms 2024-03-26T06:55:51.828 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:55:51.850 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T06:55:51.851 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-03-26T06:55:51.851 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 25 +848us[ +844us] +/- 75ms 2024-03-26T06:55:51.851 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -1067us[-1070us] +/- 74ms 2024-03-26T06:55:51.851 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +165us[ +161us] +/- 81ms 2024-03-26T06:55:51.851 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T06:55:51.852 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-26T06:55:51.863 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-26T06:55:51.864 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-26T06:55:51.887 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T06:55:51.973 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T06:55:52.075 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T06:55:52.168 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T06:55:52.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:55:52.202 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T06:55:52.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:55:52.240 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T06:55:52.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:55:52.270 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-26T06:55:52.281 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-26T06:55:52.306 INFO:teuthology.task.internal:Duration was 3112.922458 seconds 2024-03-26T06:55:52.306 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-26T06:55:52.337 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-26T06:55:52.338 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T06:55:52.340 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T06:55:52.343 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T06:55:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T06:55:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T06:55:52.443 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T06:55:52.916 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-26T06:55:52.917 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-03-26T06:55:52.917 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T06:55:52.940 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-03-26T06:55:52.940 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T06:55:52.971 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-03-26T06:55:52.971 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T06:55:52.992 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-26T06:55:52.992 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T06:55:52.994 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T06:55:53.014 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T06:55:53.061 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-26T06:55:53.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T06:55:53.067 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T06:55:53.069 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T06:55:53.478 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-26T06:55:53.490 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-26T06:55:53.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T06:55:53.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T06:55:53.549 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T06:55:53.586 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-26T06:55:53.599 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T06:55:53.602 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T06:55:53.604 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T06:55:53.625 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-03-26T06:55:53.665 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-03-26T06:55:53.676 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-03-26T06:55:53.706 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T06:55:53.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:55:53.720 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T06:55:53.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:55:53.735 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T06:55:53.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T06:55:53.749 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-26T06:55:53.761 INFO:teuthology.task.internal:Transferring archived files... 2024-03-26T06:55:53.762 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740/remote/smithi037 2024-03-26T06:55:53.763 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T06:55:53.811 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740/remote/smithi066 2024-03-26T06:55:53.812 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T06:55:53.854 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740/remote/smithi149 2024-03-26T06:55:53.854 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T06:55:53.893 INFO:teuthology.task.internal:Removing archive directory... 2024-03-26T06:55:53.893 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T06:55:53.896 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T06:55:53.898 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T06:55:53.942 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-26T06:55:53.957 INFO:teuthology.task.internal:Not uploading archives. 2024-03-26T06:55:53.957 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-26T06:55:53.980 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-26T06:55:53.980 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T06:55:53.982 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T06:55:53.985 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T06:55:53.994 INFO:teuthology.orchestra.run.smithi037.stdout: 265538 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 06:55 /home/ubuntu/cephtest 2024-03-26T06:55:53.997 INFO:teuthology.orchestra.run.smithi066.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 06:55 /home/ubuntu/cephtest 2024-03-26T06:55:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 06:55 /home/ubuntu/cephtest 2024-03-26T06:55:54.000 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-26T06:55:54.012 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-26T06:55:54.080 INFO:teuthology.nuke:Checking targets against current locks 2024-03-26T06:55:54.106 DEBUG:teuthology.nuke:shortname: smithi037 2024-03-26T06:55:54.107 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T06:55:54.132 DEBUG:teuthology.nuke:shortname: smithi066 2024-03-26T06:55:54.132 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T06:55:54.137 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 05:54:25.167422', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMUJWwIeV2tDbDTKU5oQJSoans6FdnapWWLfkeVDtMx8cGd93Rk8ud8orz2qvUwStrzULIm8g8053PXmu7XgR2c='} 2024-03-26T06:55:54.159 DEBUG:teuthology.nuke:shortname: smithi149 2024-03-26T06:55:54.159 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T06:55:54.164 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 05:54:25.166051', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFwwNqCYxhAGGT+xm8s18CqV9Mc4RWhKijKL3dAJLl5sNFuuc60XCA8QAvx6bkqmHSpPku2RnzSvrx7p6ThgJto='} 2024-03-26T06:55:54.193 INFO:teuthology.orchestra.console.smithi037:Power off 2024-03-26T06:55:54.193 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T06:55:54.208 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622740', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 05:54:25.168524', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBdzvzIWPethCrJJLKbvNhiYcu7Gg3Swk7r3Ihyf8xqRLwwSJDi4QMFI1gM/s+MFEOObjxH4RrGyOvK4iYlLwsE='} 2024-03-26T06:55:54.219 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2024-03-26T06:55:54.219 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:55:54.242 INFO:teuthology.orchestra.console.smithi066:Power off 2024-03-26T06:55:54.242 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T06:55:54.257 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-03-26T06:55:54.266 DEBUG:teuthology.orchestra.console.smithi066:power off output: Chassis Power Control: Down/Off 2024-03-26T06:55:54.267 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:55:54.286 INFO:teuthology.orchestra.console.smithi149:Power off 2024-03-26T06:55:54.286 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T06:55:54.295 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-03-26T06:55:54.309 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2024-03-26T06:55:54.309 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:55:54.333 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-26T06:55:58.258 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:55:58.296 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:55:58.334 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:55:58.370 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-03-26T06:55:58.408 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-03-26T06:55:58.447 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-26T06:56:02.375 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:56:02.409 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:56:02.447 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:56:02.486 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2024-03-26T06:56:02.519 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-03-26T06:56:02.559 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-26T06:56:02.587 INFO:teuthology.orchestra.console.smithi037:Power off completed 2024-03-26T06:56:06.522 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:56:06.560 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:56:06.633 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off 2024-03-26T06:56:06.671 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-03-26T06:56:06.735 INFO:teuthology.orchestra.console.smithi066:Power off completed 2024-03-26T06:56:06.772 INFO:teuthology.orchestra.console.smithi149:Power off completed 2024-03-26T06:56:06.879 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3112.9224576950073 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi037 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7e98f5f6b535defc7fa715b456904ffc0500aa9f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5ecbdf16442b4ecf84e4ade9321961ee status: fail success: false 2024-03-26T06:56:06.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:56:06.965 INFO:teuthology.run:FAIL